A Low-carbon Optimal Operation Method for Microgrids Based on Deep Reinforcement Learning

LIN Wenshuo; LI Xin; ZHOU Rongsheng; ZHU Yiying; PENG Yiming

doi:10.19725/j.cnki.1007-2322.2023.0327

English Version Reader LoginJournal Login

您当前的位置：

首页 >

文章列表页 >

A Low-carbon Optimal Operation Method for Microgrids Based on Deep Reinforcement Learning

更新时间：2025-12-11

- A Low-carbon Optimal Operation Method for Microgrids Based on Deep Reinforcement Learning
- Issue 6, Pages: 1131-1142(2025)
- 作者机构：
  
  广东电网有限责任公司广州供电局,广东省,广州市,510620
- 作者简介：
- 基金信息：
- DOI：10.19725/j.cnki.1007-2322.2023.0327
  CLC：
- Published：2025
- 稿件说明：
移动端阅览
LIN Wenshuo, LI Xin, ZHOU Rongsheng, et al. A Low-carbon Optimal Operation Method for Microgrids Based on Deep Reinforcement Learning[J]. 2025, (6): 1131-1142.
DOI：

LIN Wenshuo, LI Xin, ZHOU Rongsheng, et al. A Low-carbon Optimal Operation Method for Microgrids Based on Deep Reinforcement Learning[J]. 2025, (6): 1131-1142. DOI： 10.19725/j.cnki.1007-2322.2023.0327.

摘要

在建设新型电力系统的远景目标下，为实现配电侧绿色低碳运行，提出了一种基于深度强化学习的计及电转气和碳捕集的微电网低碳优化运行方法。首先，考虑新能源发电设备、储能设备、电转气系统及碳捕集设备，在发电侧碳捕集–电转气–燃气轮机同新能源共同形成了多类型供能的微电网系统。其次，为实现所提微电网系统在源荷不确定性环境下的优化运行，基于深度强化学习理论，将多时段优化问题转化为马尔科夫决策问题，并提出了一种融合知识的深度强化学习求解框架。在此基础上采用分布式近端策略优化算法实现了微电网系统多类电源的低碳优化运行。仿真结果证明了所提深度强化学习框架及算法的有效性和其制定的运行方案的经济性。

Abstract

In this paper we present a low-carbon optimization strategy for microgrids with the long-term goal of developing a new type of power system. This method

rooted in deep reinforcement learning

integrates the concepts of electricity-to-gas conversion and carbon capture to attain environmentally conscious operations. Primarily

by conducting an assessment of renewable energy generation and storage apparatus

as well as incorporating electricity-to-gas conversion and carbon capture systems

the integration of a carbon-capturing electricity-to-gas turbine within the power generation framework establishes a diverse and multi-source energy microgrid system. Subsequently

to achieve optimal operation of the proposed microgrid system in an environment of source-load uncertainty

the multi-period optimization problem is transformed into a Markov decision problem based on deep reinforcement learning theory. Additionally

a knowledge fusion deep reinforcement learning framework is proposed. The utilization of the distributed proximal policy optimization algorithm facilitates the realization of optimal performance across diverse power sources within the microgrid system. The empirical findings from the simulations provide substantial evidence for both the efficacy of the envisaged deep reinforcement learning framework and algorithm

as well as the economic feasibility of the proposed operational strategy．

关键词

Keywords

references

ZHOU Renjun, SUN Hong, TANG Xiafei, et al. Low-carbon economic dispatch based on virtual power plant made up of carbon capture unit and wind power under double carbon constraint[J]. Proceedings of the CSEE, 2018, 38(6): 1675−1683(in Chinese).

周任军, 孙洪, 唐夏菲, 等. 双碳量约束下风电–碳捕集虚拟电厂低碳经济调度[J]. 中国电机工程学报, 2018, 38(6): 1675−1683.

LI Hai , ZHANG Ning , KANG Chongqing , et al. Analytics of contribution degree for renewable energy accommodation factors[J]. Proceedings of the CSEE, 2019, 39(04): 1009−1018(in Chinese).

李海, 张宁, 康重庆, 等. 可再生能源消纳影响因素的贡献度分析方法[J]. 中国电机工程学报, 2019, 39(04): 1009−1018.

MI Jianfeng, MA Xiaofang. Development trend analysis of carbon capture, utilization and storage technology in China[J]. Proceedings of the CSEE, 2019, 39(09): 2537−2544(in Chinese).

米剑锋, 马晓芳. 中国 CCUS 技术发展趋势分析[J]. 中国电机工程学报, 2019, 39(09): 2537−2544.

SCHIEBAHN S, GRUBE T, ROBINIUS M, et al. Power to gas: technological overview, systems analysis and economic assessment for a case study in Germany[J]. International Journal of Hydrogen Energy, 2015, 40(12): 4285−4294.

ZHOU Renjun, XIAO Junwen, TANG Xiafei, et al. Coordinated optimization of carbon utilization between power-to-gas renewable energy accommodation and carbon capture power plant[J]. Electric Power Automation Equipment, 2018, 38(7): 61−67(in Chinese).

周任军, 肖钧文, 唐夏菲, 等. 电转气消纳新能源与碳捕集电厂碳利用的协调优化[J]. 电力自动化设备, 2018, 38(7): 61−67.

CHEN Boda, LIN Kaidong, ZHANG Yongjun, et al. Optimal dispatching of integrated electricity and natural gas energy systems considering the coordination of carbon capture system and power-to-gas[J]. Southern Power System Technology, 2019, 13(11): 9−17(in Chinese).

陈伯达, 林楷东, 张勇军, 等. 计及碳捕集和电转气协同的电气互联系统优化调度[J]. 南方电网技术, 2019, 13(11): 9−17.

YANG J, ZHANG N, CHENG Y, et al. Modeling the operation mechanism of combined P2G and gas-fired plant with CO2 recycling[J]. IEEE Transactions on Smart Grid, 2018, 10(1): 1111−1121.

SUN Huijuan, LIU Yun, PENG Chunhua, et al. Optimization scheduling of virtual power plant with carbon capture and waste incineration considering power-to-gas coordination[J]. Power System Technology, 2021, 45(9): 3534−3545(in Chinese).

孙惠娟, 刘昀, 彭春华, 等. 计及电转气协同的含碳捕集与垃圾焚烧虚拟电厂优化调度[J]. 电网技术, 2021, 45(9): 3534−3545.

TIAN Ming, ZHANG Haifeng, LIU Kun. “Source-load” low-carbon optimal scheduling method for data center microgrids considering power to gas and carbon capture[J]. Journal of Electrical Engineering, 2022, 17(03): 85−94(in Chinese).

田明, 张海峰, 刘坤. 计及电转气和碳捕集的数据中心微电网“源–荷”低碳优化调度方法[J]. 电气工程学报, 2022, 17(03): 85−94.

GAO Han, LI Zhengshuo. Coordinated scheduling of integrated electricity-gas energy system considering response characteristic of power-to-gas and wind power uncertainty[J]. Electric Power Automation Equipment, 2021, 41(9): 24−30(in Chinese).

高晗, 李正烁. 考虑电转气响应特性与风电出力不确定性的电–气综合能源系统协调调度[J]. 电力自动化设备, 2021, 41(9): 24−30.

QADRDAN M, WU J, JENKINS N, et al. Operating strategies for a GB integrated gas and electricity network considering the uncertainty in wind power forecasts[J]. IEEE Transactions on Sustainable Energy, 2014, 5(1): 128−138.

JIANG Mingjun, LI Qiming, ZHAO Canglu, et al. Robust scheduling model of integrated energy system with carbon capture integrating extreme scenario discrimination algorithm[J]. Smart Power, 2023, 51(03): 17−24(in Chinese).

姜明军, 黎启明, 赵苍禄, 等. 融合极限场景辨别算法的含碳捕集综合能源系统鲁棒调度模型研究[J]. 智慧电力, 2023, 51(03): 17−24.

HU Weihao, CAO Di, HUANG Qi, et al. Application of deep reinforcement learning in optimal operation of distribution network[J]. Automation of Electric Power Systems, 2023, 47(14): 174−191(in Chinese).

胡维昊, 曹迪, 黄琦, 等. 深度强化学习在配电网优化运行中的应用[J]. 电力系统自动化, 2023, 47(14): 174−191.

ZHANG Youbing, LIN Yihang, HUANG Guanhong, et al. Review on applications of deep reinforcement learning in regulation of microgrid systems[J]. Power System Technology, 2023, 47(07): 2774−2788(in Chinese).

张有兵, 林一航, 黄冠弘, 等. 深度强化学习在微电网系统调控中的应用综述[J]. 电网技术, 2023, 47(07): 2774−2788.

DOMÍNGUEZ-BARBERO D, GARCÍA-GONZÁLEZ J, SANZ-BOBI M A, et al. Optimising a microgrid system by deep reinforcement learning techniques[J]. Energies, 2020, 13(11): 2830.

ZHAO Jin, LI Fangxing, MUKHERJEE S, et al. Deep reinforcement learning-based model-free on-line dynamic multi-microgrid formation to enhance resilience[J]. IEEE Transactions on Smart Grid, 2022, 13(4): 2557−2567.

LIANG Hong, LI Hongxin, ZHANG Huaying, et al. Control strategy of microgrid energy storage system based on deep reinforcement learning[J]. Power System Technology, 2021, 45(10): 3869−3876(in Chinese).

梁宏, 李鸿鑫, 张华赢, 等. 基于深度强化学习的微网储能系统控制策略研究[J]. 电网技术, 2021, 45(10): 3869−3876.

YU Liang, XIE Weiwei, XIE Di, et al. Deep reinforcement learning for smart home energy management[J]. IEEE Internet of Things Journal, 2020, 7(4): 2751−2762.

LU Xiaozhen, XIAO Xingyu, XIAO Liang, et al. Reinforcement learning-based microgrid energy trading with a reduced power plant schedule[J]. IEEE Internet of Things Journal, 2019, 6(6): 10728−10737.

CUI Shuai, TANG Xiaoning, ZHANG Bin, et al. Thermodynamic analysis for the synthesis process of methane[J]. Computers Applied Chemistry, 2015, 32(4): 419−425(in Chinese).

崔帅, 唐晓宁, 张彬, 等. 合成气甲烷化过程热力学分析[J]. 计算机与应用化学, 2015, 32(4): 419−425.

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Optimization Model of Bi-level Economic Operation of Microgrid in Electricity Market Environment

Multi-objective Distributed Secondary Control Method for Multi-VSG Independent Microgrid

Optimization Scheduling Strategies for Integrated Energy Systems Based on Improved Deep Deterministic Policy Gradient Algorithm

Double-layer Optimization Model for Microgrid System Based on Dynamic Capacity Allocation of CES

DC Self-Adapt Frequency Regulation Method for Hybrid DC Power Supply Microgrid System

Related Author

Jingdong XIE

Wenqi LU

Zhiwei LṺ

Minling HUANG

Jing CHENG

Jianxiong HU

Jinxia GONG

Chenzhou LI

Related Institution

College of Economics and Management, Shanghai University of Electric Power, Yangpu Districe

Energy and Power Science and Technology Center, Shanghai University of Electric Power, Yangpu Districe

School of Electrical Engineering, Xinjiang University, Urumqi 830047, Xinjiang Uygur Autonomous Region

Renewable Energy Power Generation and Grid-connected Control Engineering Research Center of the Ministry of Education, Urumqi 830047, Xinjiang Uygur Autonomous Region

Department of Electric Power Engineering, Shanghai University of Electric Power, Yangpu District

AI问答

⁰