基于近端策略优化的电力系统弹性增强多智能体框架

王新瑞; 石新聪; 陈文刚; 姬玉泽; 朱剑飞

doi:10.19725/j.cnki.1007-2322.2024.0049

欢迎来到能源电力期刊网！

English Version 期刊登录

您当前的位置：

首页 >

文章列表页 >

基于近端策略优化的电力系统弹性增强多智能体框架

智能电网 | 更新时间：2026-06-02

- 基于近端策略优化的电力系统弹性增强多智能体框架
- A Resilience-enhanced Multi-agent Framework for Power Systems Based on Proximal Policy Optimization
- 现代电力 2026年43卷第3期页码：433-443
- 作者机构：
  
  1. 国网晋城供电公司,山西省,晋城市,048000
  2. 浙江大学,浙江省,杭州市,310013
- 作者简介：
  
  王新瑞(1983)，男，学士，高级工程师，主要研究方向为电网调度运行及控制技术，E-mail：wangxr5290@163.com
  石新聪(1985)，男，博士，高级工程师，主要研究方向为电网调度运行、控制技术，E-mail：ncepu_shixincong@126.com
  陈文刚(1971)，男，学士，高级工程师，主要研究方向为自动化维护及网络管理、机器学习，E-mail：452215444@qq.com
  姬玉泽(1995)，男，硕士，工程师，主要研究方向为电网调度运行控制，E-mail：jcjiyuze@163.com
  朱剑飞(1995)，男，硕士，工程师，主要研究方向为调度自动化，E-mail：18435164021@163.com
- 基金信息：
  
  国网山西省电力有限公司科技项目(5205E0230001)。
- DOI：10.19725/j.cnki.1007-2322.2024.0049
  中图分类号： 3
- 纸质出版：2026
- 稿件说明：
移动端阅览
王新瑞, 石新聪, 陈文刚, 等. 基于近端策略优化的电力系统弹性增强多智能体框架[J]. 现代电力, 2026,43(3):433-443.

WANG Xinrui, SHI Xincong, CHEN Wengang, et al. A Resilience-enhanced Multi-agent Framework for Power Systems Based on Proximal Policy Optimization[J]. 2026, 43(3): 433-443.
王新瑞, 石新聪, 陈文刚, 等. 基于近端策略优化的电力系统弹性增强多智能体框架[J]. 现代电力, 2026,43(3):433-443. DOI： 10.19725/j.cnki.1007-2322.2024.0049.

WANG Xinrui, SHI Xincong, CHEN Wengang, et al. A Resilience-enhanced Multi-agent Framework for Power Systems Based on Proximal Policy Optimization[J]. 2026, 43(3): 433-443. DOI： 10.19725/j.cnki.1007-2322.2024.0049.

摘要

现有的电力系统弹性提升方法，如移动电源调度、主动发电重调度和网络拓扑重构等，都未深入研究无功补偿器在中断事件期间和中断事件后维持电压稳定的能力和灵活性。同时，这些方法对系统模型的精确程度要求较高，难以应用于大型综合电网。为此，该文提出一种基于深度强化学习的近端策略优化方法的多智能体框架，以解决与精确系统建模相关的计算和可扩展性问题，并为电力系统弹性增强制定无功补偿器部署计划，引入一种基于智能体的混合柔性动作评价算法，用于无功补偿器的离线定位、分级和在线控制，以提高其电压恢复能力。多智能体框架通过学习以前的经验并接受训练，最终确定无功补偿器适当的位置和大小，以避免多线路故障期间的节点电压越限状况。以暴风雨期间多线路停电引起的电压越限问题为例，在IEEE 39节点系统上进行验证，结果表明：多智能体框架能够有效规划和控制无功补偿器，消除或减少故障线路脱网状况下的节点电压越限情况，提高电力系统的弹性。

Abstract

Existing power system resilience enhancement methods

such as mobile power scheduling

active generation rescheduling

and network topology reconfiguration

have not fully explored the potential of the reactive power compensator to maintain voltage stability during and after an outage event and flexibility. Additionally

they require a high degree of accuracy in system modeling

which poses challenges for them to scale up their application to large-scale integrated power grids. To this end

in the paper we propose a multi-agent framework based on deep reinforcement learning for proximal policy optimization methods

aiming to address the computational and scalability issues associated with accurate system modeling. Moreover

we develop a reactive compensator deployment plan for power system resilience enhancement. This plan incorporates a hybrid soft action critic algorithm based on agent for offline localization

classification

and on-line control of reactive compensators to improve their voltage recovery capability. The multi-agent framework learns from previous experiences and is trained to determine the appropriate location and sizing of reactive power compensators

thereby preventing bus voltage overshoot during multi-line faults. As a case study

the voltage overshoot problem caused by a multi-line outage during a storm is validated on an IEEE 39 bus system. The results indicate that the multi-agent framework can effectively coordinate and control the reactive power compensators to eliminate or minimize the bus voltage overruns during faulted line off-grid conditions

which improves the resilience of the power system.

关键词

Keywords

references

HUANG G, WANG J, CHEN C, et al . Integration of preventive and emergency responses for power grid resilience enhancement[J ] . IEEE Transactions on Power Systems, 2017, 32(6): 4451−4463.

SCHNEIDER K P, LAVAL S, HANSEN J, et al . A distributed power system control architecture for improved distribution system resiliency[J ] . IEEE Access, 2019(7): 9957−9970.

ZHOU Yi, LI Yichen, FU Youshu, et al . Study on optimal decision of unit commitment with flexible reserve participation in power grid[J ] . Journal of Harbin University of Science and Technology, 2023, 28(3): 56−66(in Chinese).

HUANG Chunguang, SHANG Xiaofeng. A comprehensive control method for reactive power compensation in low-voltage power distribution system[J]. Electrical Engineering, 2023, 24(7): 77−82(in Chinese).

MA Biman. Research on enhancing the elasticity of power grid considering the impact of different disasters[D]. Xiangtan: Xiangtan University, 2019(in Chinese).

YAN Xiangwu, GUO Yan, PENG Weifeng, et al . Voltage source var compensator based on rotary phase shifting transformer and its control strategy[J ] . Transactions of China Electrotechnical Society, 2023, 38(16): 4448−4464(in Chinese).

SUN L, LIU W J, CHUNG C Y, et al . Improving he restorability of bulk power systems with the implementation of a WF-BESS system[J ] . IEEE Transactions on Power Systems, 2019, 3(34): 2366−2377.

WANG Q, YU Z D, YE R, et al . An ordered curtailment strategy for offshore wind power under extreme weather conditions considering the resilience of the grid[J ] . IEEE Access, 2019(7): 54824−54833.

LI J, LIU F, CHEN Y, et al . Resilience control of dc shipboard power systems[J ] . IEEE Transactions on Power Systems, 2018, 33(6): 6675−6685.

ZENG X L, LIU Z, HUI Q. Energy equipartition stabilization and cascading resilience optimization for geospatially distributed cyber-physical network systems[J]. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 2015, 45(1): 25−43.

LI Z Y, SHAHIDEHPOUR M, AMINIFAR F, et al . Networked microgrids for enhancing the power system resilience[J ] . Proceedings of the IEEE, 2017, 105(7): 1289−1310.

MUSLEH A S, KHALID H M, MUYEEN S M, et al . A prediction algorithm to enhance grid resilience toward cyber attacks in WAMCS applications[J ] . IEEE Systems Journal, 2019, 13(1): 710−719.

ALCAIDE-GODINEZ I, BAI F F, SAHA T K, et al . Contingency reserve estimation of fast frequency response for battery energy storage system[J ] . International Journal of Electrical Power Energ y Systems, 2022, 143(Dec.): 108428.1−108428.11.

YANG Peng, YU Dan, GUO Yuhan, et al . Optimization model of new energy accommodation considering demand response[J ] . Distribution Utilization, 2022, 39(11): 79−86(in Chinese).

TAHERI B, SAFDARIAN A, MOEINI-AGHTAIE M, et al . Enhancing resilience level of power distribution systems using proactive operational actions[J ] . IEEE Access, 2019(7): 137378−137389.

NAZEMI M, MOEINI-AGHTAIE M, FOTUHI-FIRUZABAD M, et al . Energy storage planning for enhanced resilience of power distribution networks against earthquakes[J ] . IEEE Transactions on Sustainable Energy, 2020, 11(2): 795−806.

YUAN Wenhai, JULAITI Abuliz, HUANG Xinmin, et al . Disaster prediction-based resilience enhancement strategy for multi-hour power supply in distribution networks[J ] . Modern Electric Power, 2024, 41(6): 1060−1071(in Chinese).

FENG Bin, HU Yijie, HUANG Gang, et al . Review on optimization methods for new power system dispatch based on deep reinforcement learning[J ] . Automation for Electric Power System, 2023, 47(17): 187−199(in Chinese).

LIU Mengqi. Elastic enhancement control of microgrid based on Q-learning algorithm[D]. Zhengzhou: North China University of Water Resources and Electric Power, 2022(in Chinese).

XU Y L, ZHANG W, LIU W X, et al . Ferrese, multiagent-based reinforcement learning for optimal reactive power dispatch[J ] . IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), 2012, 42(6): 1742−1751.

VLACHOGIANNIS J G, HATZIARGYRIOU N D. Reinforcement learning for reactive power control[J]. IEEE Transactions on Power Systems, 2004, 19(3): 1317−1325.

DUAN J J, SHI D, DIAO R S, et al . Deep-reinforcement-learning-based autonomous voltage control for power grid operations[J ] . IEEE Transactions on Power Systems, 2020, 35(1): 814−817.

WANG S Y, DUAN J J, SHI D, et al . A data-driven multi-agent autonomous voltage control framework using deep reinforcement learning[J ] . IEEE Transactions on Power Systems, 2020, 35(6): 4644-4654.

QU G N, LI N. Optimal distributed feedback voltage control under limited reactive power[J]. IEEE Transactions on Power Systems, 2020, 35(1): 315−331.

SINGHAL A, AJJARAPU V, FULLER J, et al . Real-time local volt/var control under external disturbances with high PV penetration[J ] . IEEE Transactions on Smart Grid, 2019, 10(4): 3849−3859.

ZHANG Lei, XU Gongde, SUN Huiru, et al . Discussion on the power quality of DC distribution system[J ] . Distribution Utilization, 2023, 40(10): 28−34(in Chinese).

ZHANG Haichun, CHEN Wangda, SHEN Jun, et al . A review of distribution network resilience studies considering flexibility resources[J ] . Electric Power Construction, 2023, 44(12): 66−84(in Chinese).

XUN Yixuan. Research on comprehensive configuration of reactive power compensation[J]. Electrical Technology and Economics, 2023(5): 76−78(in Chinese).

HUSSAIN A, BUI A, KIM H. Resilience-oriented optimal operation of networked hybrid microgrids[J]. IEEE Transactions on Smart Grid, 2019, 10(1): 204−215.

YANG Xiao, GUO Yinan, JI Jianjiao, et al . PPO multi-objective task allocation method for heterogeneous crowd sensing[J ] . Control Theory Applications, 2024, 41(6): 1056−1066(in Chinese).

浏览量

181

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

基于深度强化学习的微电网低碳优化运行方法

基于改进深度确定性策略梯度算法的综合能源系统优化调度策略

计及碳捕集装置及碳排放潮流理论的电力系统低碳优化学习调度

基于深度强化学习的综合能源系统优化调度

基于改进双延迟深度确定性策略梯度算法的微电网低碳经济优化调度策略