Distributed Reinforcement Learning-Driven Dynamic Energy Optimization Management Strategy for Microgrid Clusters

LIU Hua; XIONG Zaibao; JIANG Taoning; GAO Yu; JIN Yuhan; GE Leijiao

doi:10.11930/j.issn.1004-9649.202503064

English Version Reader LoginJournal Login

您当前的位置：

首页 >

文章列表页 >

Distributed Reinforcement Learning-Driven Dynamic Energy Optimization Management Strategy for Microgrid Clusters

更新时间：2025-12-08

- Distributed Reinforcement Learning-Driven Dynamic Energy Optimization Management Strategy for Microgrid Clusters
- Vol. 58, Issue 10, Pages: 50-62(2025)
- 作者机构：
  
  1. 国核电力规划设计研究院有限公司,北京,100095
  2. 天津大学电气自动化与信息工程学院,天津,300072
- 作者简介：
- 基金信息：
- DOI：10.11930/j.issn.1004-9649.202503064
  CLC：
- Published Online：23 October 2025，
  
  Published：2025
- 稿件说明：
移动端阅览
柳华, 熊再豹, 蒋陶宁, 高宇, 金雨含, 葛磊蛟. 分布式强化学习驱动的微电网群动态能量优化管理策略[J]. 中国电力, 2025, 58(10): 50-62.

LIU Hua, XIONG Zaibao, JIANG Taoning, et al. Distributed Reinforcement Learning-Driven Dynamic Energy Optimization Management Strategy for Microgrid Clusters[J]. 2025, 58(10): 50-62.
柳华, 熊再豹, 蒋陶宁, 高宇, 金雨含, 葛磊蛟. 分布式强化学习驱动的微电网群动态能量优化管理策略[J]. 中国电力, 2025, 58(10): 50-62. DOI： 10.11930/j.issn.1004-9649.202503064.

LIU Hua, XIONG Zaibao, JIANG Taoning, et al. Distributed Reinforcement Learning-Driven Dynamic Energy Optimization Management Strategy for Microgrid Clusters[J]. 2025, 58(10): 50-62. DOI： 10.11930/j.issn.1004-9649.202503064.

摘要

随着全球能源需求增长和可持续发展目标推进，微电网能量管理面临高维度、复杂性和动态性挑战。因此，提出一种分布式强化学习驱动的微电网群能量优化管理策略，旨在通过智能化手段提升微电网群在能源调度和管理方面的效率。首先，针对微电网群负荷动态变化大、拓扑结构复杂等难题，构建目标优化函数，并引入一种分布式强化学习算法，实现微电网群在分布式环境下的自适应决策与协同优化。其次，将微电网中的每个电源点视为一个智能体，利用信息共享实现全局效益最大化与发电成本最小化，达到微电网群发电、储能和负荷需求管理的实时优化。最后，通过实际案例进行验证，结果表明所提策略能够维持电力供需之间的动态平衡，与传统方法技术相比，总发电成本节约了18%左右。

Abstract

With the growth of global energy demand and the advancement of sustainable development goals

energy management of microgrids

as an important means to address energy supply

improve energy efficiency and promote green energy utilization

is faced with high dimensionality

complexity and dynamic challenges. In this paper

we propose a distributed reinforcement learning-driven energy optimization and management strategy for microgrid clusters

aiming to enhance the efficiency and sustainability of microgrid clusters in energy scheduling and management through intelligent means. Aiming at the challenges of the microgrid cluster

such as large dynamic changes in load and complex topology

adaptive decision-making and collaborative optimization of the microgrid cluster in a distributed environment is achieved by constructing an objective optimization function and introducing a distributed reinforcement learning algorithm; and power generation of the microgrid cluster is achieved by treating each power point in the microgrid as an agent and utilizing information sharing to achieve the maximization of the global benefit and minimization of the power generation cost

storage and load demand management; finally

the results of the real case show that the proposed strategy is able to maintain the dynamic balance between power supply and demand

resulting in a saving of about 18% of the total power generation cost compared to the traditional methodology techniques.

关键词

Keywords

references

1王治国, 刘继荣, 郝艳军, 等. 多智能体系统微电网能源实时管理系统设计[J]. 信息技术, 2021, 45 (5): 61- 67.

WANG Zhiguo, LIU Jirong, HAO Yanjun, et al. Design of real time energy management system for multi intelligent microgrid[J]. Information Technology, 2021, 45 (5): 61- 67.

2魏震波, 张芷琪, 李银江, 等. 多主体合作模式下微电网规划运行一体化模型[J]. 电力建设, 2024, 45 (10): 47- 58. DOI

WEI Zhenbo, ZHANG Zhiqi, LI Yinjiang, et al. An integrated model of microgrid planning and operation under A multi-subject cooperation model[J]. Electric Power Construction, 2024, 45 (10): 47- 58. DOI

3FARZIN H, FOTUHI-FIRUZABAD M, MOEINI-AGHTAIE M. A stochastic multi-objective framework for optimal scheduling of energy storage systems in microgrids[J]. IEEE Transactions on Smart Grid, 2017, 8 (1): 117- 127. DOI

4樊晓伟, 王瑞妙, 杨海峰, 等. 计及源荷不确定的综合能源微电网集群优化运行[J]. 电力建设, 2024, 45 (8): 128- 139.

FAN Xiaowei, WANG Ruimiao, YANG Haifeng, et al. Optimization operation of integrated energy microgrid cluster considering source-load uncertainty[J]. Electric Power Construction, 2024, 45 (8): 128- 139.

5邱革非, 冯泽华, 沈赋, 等. 考虑车网互动的园区电网动态双层能量管理策略[J]. 上海交通大学学报, 2024, 58 (6): 916- 925.

QIU Gefei, FENG Zehua, SHEN Fu, et al. Dynamic double-layer energy management strategy for park power grid considering vehicle-to-grid[J]. Journal of Shanghai Jiao Tong University, 2024, 58 (6): 916- 925.

6ZHENG Z, SHAFIQUE M, LUO X W, et al. A systematic review towards integrative energy management of smart grids and urban energy systems[J]. Renewable and Sustainable Energy Reviews, 2024, 189, 114023. DOI

7徐明宇, 郝文波, 王盼宝, 等. 基于动态随机模型的微电网群能量管理方法[J]. 电力工程技术, 2022, 41 (5): 140- 148. DOI

XU Mingyu, HAO Wenbo, WANG Panbao, et al. Energy management method of multi-microgrids based on dynamic stochastic model[J]. Electric Power Engineering Technology, 2022, 41 (5): 140- 148. DOI

8李扬, 马文捷, 卜凡金, 等. 多智能体深度强化学习驱动的跨园区能源交互优化调度[J]. 电力建设, 2024, 45 (5): 59- 70.

LI Yang, MA Wenjie, BU Fanjin, et al. Deep reinforcement learning-driven cross-community energy interaction optimal scheduling[J]. Electric Power Construction, 2024, 45 (5): 59- 70.

9EDUSSURIYA C, MARIKKAR U, WICKRAMASINGHE S, et al. Peer-to-peer energy trading through swarm intelligent stackelberg game[J]. Energies, 2023, 16 (5): 2434. DOI

10张考, 何凯琳, 杨沛豪. 基于模糊强化学习的电力变压器故障诊断算法研究[J]. 综合智慧能源, 2024, 46 (10): 48- 55.

ZHANG Kao, HE Kailin, YANG Peihao. Research on power transformer fault diagnosis algorithm based on fuzzy reinforcement learning[J]. Integrated Intelligent Energy, 2024, 46 (10): 48- 55.

11党亚峥, 唐崇伟. 一种改进的乘子交替方向法在ℓ1-正则化分裂可行问题中的应用[J]. 上海理工大学学报, 2020, 42 (5): 460- 466,503.

DANG Yazheng, TANG Chongwei. An improved alternating direction method of multipliers for ℓ1-norm regularization splitting feasibility problem[J]. Journal of University of Shanghai for Science and Technology, 2020, 42 (5): 460- 466,503.

12YIASEMIS G, MORIAKOV N, SONKE J J, et al. vSHARP: variable splitting half-quadratic ADMM algorithm for reconstruction of inverse-problems[J]. Magnetic Resonance Imaging, 2025, 115, 110266. DOI

13VARMA A V S S, MANEPALLI K. VLSI realization of hybrid fast fourier transform using reconfigurable booth multiplier[J]. International Journal of Information Technology, 2024, 16 (7): 4323- 4333. DOI

14苏一丹, 续嘉, 覃华. 二元裂解算子交替方向乘子法的核极限学习机[J]. 电子与信息学报, 2021, 43 (9): 2586- 2593.

SU Yidan, XU Jia, QIN Hua. Kernel extreme learning machine based on alternating direction multiplier method of binary splitting operator[J]. Journal of Electronics & Information Technology, 2021, 43 (9): 2586- 2593.

15HASSAN IBRAHIM S, SALAH HAMEED I, HADI ALI H. High performance IIR filter design based on fast multiplier[J]. Diyala Journal of Engineering Sciences, 2025,192–202.

16刘洋, 薛中会, 王永全, 等. 分裂可行性问题的外推加速线性交替方向乘子法及其全局收敛性[J]. 计算机科学, 2023, 50 (6): 261- 265.

LIU Yang, XUE Zhonghui, WANG Yongquan, et al. Extrapolation accelerated linear alternating direction multiplier method for split feasibility problems and its global convergence[J]. Computer Science, 2023, 50 (6): 261- 265.

17GAJAWADA S, DEVI D N, RAO M. MOHSKM: meta-heuristic optimization driven hardware-efficient heterogeneous-split karatsuba multipliers for large-bit operations[C]//2024 IEEE Computer Society Annual Symposium on VLSI (ISVLSI). Knoxville, TN, USA. IEEE, 2024: 749-752.

18LEI W L, YE Y, XIAO M, et al. Adaptive stochastic ADMM for decentralized reinforcement learning in edge IoT[J]. IEEE Internet of Things Journal, 2022, 9 (22): 22958- 22971. DOI

19ANNESE M, FERNÁNDEZ M A, GASTALDI L. Splitting schemes for a Lagrange multiplier formulation of FSI with immersed thin-walled structure: stability and convergence analysis[J]. IMA Journal of Numerical Analysis, 2023, 43 (2): 881- 919. DOI

20ARWA E O, FOLLY K A. Reinforcement learning techniques for optimal power control in grid-connected microgrids: a comprehensive review[J]. IEEE Access, 2020, 8, 208992- 209007. DOI

21ZHU D, YANG B, LIU Y, et al. Energy management based on multi-agent deep reinforcement learning for a multi-energy industrial park[J]. Applied Energy, 2022, 311, 118636. DOI

22MATTERA G, CAGGIANO A, NELE L. Optimal data-driven control of manufacturing processes using reinforcement learning: an application to wire arc additive manufacturing[J]. Journal of Intelligent Manufacturing, 2025, 36 (2): 1291- 1310. DOI

23DAI P C, YU W W, WEN G H, et al. Distributed reinforcement learning algorithm for dynamic economic dispatch with unknown generation cost functions[J]. IEEE Transactions on Industrial Informatics, 2020, 16 (4): 2258- 2267. DOI

24XIAO G Y, ZHANG H G. Convergence analysis of value iteration adaptive dynamic programming for continuous-time nonlinear systems[J]. IEEE Transactions on Cybernetics, 2024, 54 (3): 1639- 1649. DOI

25DESHPANDE S V, HARIKRISHNAN R, IBRAHIM B S K K, et al. Mobile robot path planning using deep deterministic policy gradient with differential gaming (DDPG-DG) exploration[J]. Cognitive Robotics, 2024, 4, 156- 173. DOI

26孙国强, 殷岩岩, 卫志农, 等. 基于深度确定性策略梯度的主动配电网有功-无功协调优化调度[J]. 电力建设, 2023, 44 (11): 33- 42.

SUN Guoqiang, YIN Yanyan, WEI Zhinong, et al. Coordinated optimal dispatch of active and reactive power in active distribution networks using deep deterministic strategy gradient[J]. Electric Power Construction, 2023, 44 (11): 33- 42.

27黄堃, 付明, 梁加本. 基于融合专家知识DDPG的孤岛微电网频率调节策略[J]. 中国电力, 2024, 57 (2): 194- 201.

HUANG Kun, FU Ming, LIANG Jiaben. Frequency regulation strategy of isolated island microgrid based on fusion expert knowledge DDPG[J]. Electric Power, 2024, 57 (2): 194- 201.

28SHI X T, LI Y J, HU W X, et al. Optimal lateral path-tracking control of vehicles with partial unknown dynamics via DPG-based reinforcement learning methods[J]. IEEE Transactions on Intelligent Vehicles, 2024, 9 (1): 1701- 1710. DOI

29CARTA S, FERREIRA A, PODDA A S, et al. Multi-DQN: an ensemble of deep Q-learning agents for stock market forecasting[J]. Expert Systems with Applications, 2021, 164, 113820. DOI

30吴润泽, 霍金鑫, 郭昊博. 基于DQN的电力协同计算与缓存的任务卸载策略[J]. 电力建设, 2024, 45 (8): 149- 158.

WU Runze, HUO Jinxin, GUO Haobo. DQN-based task offloading strategy for power co-computing and caching[J]. Electric Power Construction, 2024, 45 (8): 149- 158.

31万玲玲, 陈中, 王毅, 等. 考虑能量时空转移的城市规模化共享电动汽车充放电优化调度[J]. 电力建设, 2023, 44 (6): 135- 143.

WAN Lingling, CHEN Zhong, WANG Yi, et al. Optimal charging and discharging scheduling of urban large-scale shared electric vehicles considering energy temporal and spatial transfer[J]. Electric Power Construction, 2023, 44 (6): 135- 143.

32YE Y, CHEN H, XIAO M, et al. Privacy-preserving incremental ADMM for decentralized consensus optimization[J]. IEEE Transactions on Signal Processing, 2020, 68, 5842- 5854. DOI

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

No data

Related Author

No data

Related Institution

No data

AI问答

⁰