A Resilience-enhanced Multi-agent Framework for Power Systems Based on Proximal Policy Optimization

WANG Xinrui; SHI Xincong; CHEN Wengang; JI Yuze; ZHU Jianfei

doi:10.19725/j.cnki.1007-2322.2024.0049

English Version Journal Login

您当前的位置：

首页 >

文章列表页 >

A Resilience-enhanced Multi-agent Framework for Power Systems Based on Proximal Policy Optimization

更新时间：2026-06-02

- A Resilience-enhanced Multi-agent Framework for Power Systems Based on Proximal Policy Optimization
- Vol. 43, Issue 3, Pages: 433-443(2026)
- 作者机构：
  
  1. 国网晋城供电公司,山西省,晋城市,048000
  2. 浙江大学,浙江省,杭州市,310013
- 作者简介：
- 基金信息：
- DOI：10.19725/j.cnki.1007-2322.2024.0049
  CLC： 3
- Published：2026
- 稿件说明：
移动端阅览
WANG Xinrui, SHI Xincong, CHEN Wengang, et al. A Resilience-enhanced Multi-agent Framework for Power Systems Based on Proximal Policy Optimization[J]. 2026, 43(3): 433-443.
DOI：

WANG Xinrui, SHI Xincong, CHEN Wengang, et al. A Resilience-enhanced Multi-agent Framework for Power Systems Based on Proximal Policy Optimization[J]. 2026, 43(3): 433-443. DOI： 10.19725/j.cnki.1007-2322.2024.0049.

摘要

现有的电力系统弹性提升方法，如移动电源调度、主动发电重调度和网络拓扑重构等，都未深入研究无功补偿器在中断事件期间和中断事件后维持电压稳定的能力和灵活性。同时，这些方法对系统模型的精确程度要求较高，难以应用于大型综合电网。为此，该文提出一种基于深度强化学习的近端策略优化方法的多智能体框架，以解决与精确系统建模相关的计算和可扩展性问题，并为电力系统弹性增强制定无功补偿器部署计划，引入一种基于智能体的混合柔性动作评价算法，用于无功补偿器的离线定位、分级和在线控制，以提高其电压恢复能力。多智能体框架通过学习以前的经验并接受训练，最终确定无功补偿器适当的位置和大小，以避免多线路故障期间的节点电压越限状况。以暴风雨期间多线路停电引起的电压越限问题为例，在IEEE 39节点系统上进行验证，结果表明：多智能体框架能够有效规划和控制无功补偿器，消除或减少故障线路脱网状况下的节点电压越限情况，提高电力系统的弹性。

Abstract

Existing power system resilience enhancement methods

such as mobile power scheduling

active generation rescheduling

and network topology reconfiguration

have not fully explored the potential of the reactive power compensator to maintain voltage stability during and after an outage event and flexibility. Additionally

they require a high degree of accuracy in system modeling

which poses challenges for them to scale up their application to large-scale integrated power grids. To this end

in the paper we propose a multi-agent framework based on deep reinforcement learning for proximal policy optimization methods

aiming to address the computational and scalability issues associated with accurate system modeling. Moreover

we develop a reactive compensator deployment plan for power system resilience enhancement. This plan incorporates a hybrid soft action critic algorithm based on agent for offline localization

classification

and on-line control of reactive compensators to improve their voltage recovery capability. The multi-agent framework learns from previous experiences and is trained to determine the appropriate location and sizing of reactive power compensators

thereby preventing bus voltage overshoot during multi-line faults. As a case study

the voltage overshoot problem caused by a multi-line outage during a storm is validated on an IEEE 39 bus system. The results indicate that the multi-agent framework can effectively coordinate and control the reactive power compensators to eliminate or minimize the bus voltage overruns during faulted line off-grid conditions

which improves the resilience of the power system.

关键词

Keywords

references

HUANG G, WANG J, CHEN C, et al . Integration of preventive and emergency responses for power grid resilience enhancement[J ] . IEEE Transactions on Power Systems, 2017, 32(6): 4451−4463.

SCHNEIDER K P, LAVAL S, HANSEN J, et al . A distributed power system control architecture for improved distribution system resiliency[J ] . IEEE Access, 2019(7): 9957−9970.

ZHOU Yi, LI Yichen, FU Youshu, et al . Study on optimal decision of unit commitment with flexible reserve participation in power grid[J ] . Journal of Harbin University of Science and Technology, 2023, 28(3): 56−66(in Chinese).

HUANG Chunguang, SHANG Xiaofeng. A comprehensive control method for reactive power compensation in low-voltage power distribution system[J]. Electrical Engineering, 2023, 24(7): 77−82(in Chinese).

MA Biman. Research on enhancing the elasticity of power grid considering the impact of different disasters[D]. Xiangtan: Xiangtan University, 2019(in Chinese).

YAN Xiangwu, GUO Yan, PENG Weifeng, et al . Voltage source var compensator based on rotary phase shifting transformer and its control strategy[J ] . Transactions of China Electrotechnical Society, 2023, 38(16): 4448−4464(in Chinese).

SUN L, LIU W J, CHUNG C Y, et al . Improving he restorability of bulk power systems with the implementation of a WF-BESS system[J ] . IEEE Transactions on Power Systems, 2019, 3(34): 2366−2377.

WANG Q, YU Z D, YE R, et al . An ordered curtailment strategy for offshore wind power under extreme weather conditions considering the resilience of the grid[J ] . IEEE Access, 2019(7): 54824−54833.

LI J, LIU F, CHEN Y, et al . Resilience control of dc shipboard power systems[J ] . IEEE Transactions on Power Systems, 2018, 33(6): 6675−6685.

ZENG X L, LIU Z, HUI Q. Energy equipartition stabilization and cascading resilience optimization for geospatially distributed cyber-physical network systems[J]. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 2015, 45(1): 25−43.

LI Z Y, SHAHIDEHPOUR M, AMINIFAR F, et al . Networked microgrids for enhancing the power system resilience[J ] . Proceedings of the IEEE, 2017, 105(7): 1289−1310.

MUSLEH A S, KHALID H M, MUYEEN S M, et al . A prediction algorithm to enhance grid resilience toward cyber attacks in WAMCS applications[J ] . IEEE Systems Journal, 2019, 13(1): 710−719.

ALCAIDE-GODINEZ I, BAI F F, SAHA T K, et al . Contingency reserve estimation of fast frequency response for battery energy storage system[J ] . International Journal of Electrical Power Energ y Systems, 2022, 143(Dec.): 108428.1−108428.11.

YANG Peng, YU Dan, GUO Yuhan, et al . Optimization model of new energy accommodation considering demand response[J ] . Distribution Utilization, 2022, 39(11): 79−86(in Chinese).

TAHERI B, SAFDARIAN A, MOEINI-AGHTAIE M, et al . Enhancing resilience level of power distribution systems using proactive operational actions[J ] . IEEE Access, 2019(7): 137378−137389.

NAZEMI M, MOEINI-AGHTAIE M, FOTUHI-FIRUZABAD M, et al . Energy storage planning for enhanced resilience of power distribution networks against earthquakes[J ] . IEEE Transactions on Sustainable Energy, 2020, 11(2): 795−806.

YUAN Wenhai, JULAITI Abuliz, HUANG Xinmin, et al . Disaster prediction-based resilience enhancement strategy for multi-hour power supply in distribution networks[J ] . Modern Electric Power, 2024, 41(6): 1060−1071(in Chinese).

FENG Bin, HU Yijie, HUANG Gang, et al . Review on optimization methods for new power system dispatch based on deep reinforcement learning[J ] . Automation for Electric Power System, 2023, 47(17): 187−199(in Chinese).

LIU Mengqi. Elastic enhancement control of microgrid based on Q-learning algorithm[D]. Zhengzhou: North China University of Water Resources and Electric Power, 2022(in Chinese).

XU Y L, ZHANG W, LIU W X, et al . Ferrese, multiagent-based reinforcement learning for optimal reactive power dispatch[J ] . IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), 2012, 42(6): 1742−1751.

VLACHOGIANNIS J G, HATZIARGYRIOU N D. Reinforcement learning for reactive power control[J]. IEEE Transactions on Power Systems, 2004, 19(3): 1317−1325.

DUAN J J, SHI D, DIAO R S, et al . Deep-reinforcement-learning-based autonomous voltage control for power grid operations[J ] . IEEE Transactions on Power Systems, 2020, 35(1): 814−817.

WANG S Y, DUAN J J, SHI D, et al . A data-driven multi-agent autonomous voltage control framework using deep reinforcement learning[J ] . IEEE Transactions on Power Systems, 2020, 35(6): 4644-4654.

QU G N, LI N. Optimal distributed feedback voltage control under limited reactive power[J]. IEEE Transactions on Power Systems, 2020, 35(1): 315−331.

SINGHAL A, AJJARAPU V, FULLER J, et al . Real-time local volt/var control under external disturbances with high PV penetration[J ] . IEEE Transactions on Smart Grid, 2019, 10(4): 3849−3859.

ZHANG Lei, XU Gongde, SUN Huiru, et al . Discussion on the power quality of DC distribution system[J ] . Distribution Utilization, 2023, 40(10): 28−34(in Chinese).

ZHANG Haichun, CHEN Wangda, SHEN Jun, et al . A review of distribution network resilience studies considering flexibility resources[J ] . Electric Power Construction, 2023, 44(12): 66−84(in Chinese).

XUN Yixuan. Research on comprehensive configuration of reactive power compensation[J]. Electrical Technology and Economics, 2023(5): 76−78(in Chinese).

HUSSAIN A, BUI A, KIM H. Resilience-oriented optimal operation of networked hybrid microgrids[J]. IEEE Transactions on Smart Grid, 2019, 10(1): 204−215.

YANG Xiao, GUO Yinan, JI Jianjiao, et al . PPO multi-objective task allocation method for heterogeneous crowd sensing[J ] . Control Theory Applications, 2024, 41(6): 1056−1066(in Chinese).

Views

181

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

A Low-carbon Optimal Operation Method for Microgrids Based on Deep Reinforcement Learning

Optimization Scheduling Strategies for Integrated Energy Systems Based on Improved Deep Deterministic Policy Gradient Algorithm

Low Carbon Optimal Learning Scheduling for Power Systems With Carbon Catchment Devices and Carbon Flow Theory

Optimal Dispatch of Integrated Energy System Based on Deep Reinforcement Learning

A Low-carbon Economic Optimization Dispatching Strategy for Microgrids Based on Improved Twin Delayed Deep Deterministic Policy Gradient Algorithm

Related Author

PENG Yiming

ZHU Yiying

ZHOU Rongsheng

LI Xin

LIN Wenshuo

Jinxia GONG

Chenzhou LI

Hui KE

Related Institution

Guangzhou Power Supply Bureau of Guangdong Power Grid

Department of Electric Power Engineering, Shanghai University of Electric Power, Yangpu District

POWERCHINA Shanghai Electric Power Engineering Co., Ltd., Huangpu District

State Grid Liaoning Electric Power Supply Co., Ltd., Dalian Electric Power Supply Company

School of Electrical Engineering, Dalian University of Technology

AI问答

⁰