Knowledge-guided Hybrid Deep Reinforcement Learning for Transient Stability Emergency Control

Yixi Chen; Jizhong Zhu; Yun Liu; Le Zhang; Kaixin Lin

doi:10.35833/MPCE.2025.000750

欢迎来到能源电力期刊网！

English Version 期刊登录

您当前的位置：

首页 >

文章列表页 >

Knowledge-guided Hybrid Deep Reinforcement Learning for Transient Stability Emergency Control

Original Paper | 更新时间：2026-09-28

- Knowledge-guided Hybrid Deep Reinforcement Learning for Transient Stability Emergency Control
- Knowledge-guided Hybrid Deep Reinforcement Learning for Transient Stability Emergency Control
- MPCE 2026年14卷第5期页码：1634-1646
- 作者机构：
  
  1.School of Electric Power Engineering, South China University of Technology, Guangzhou, China
  2.Electric Power Research Institute, China Southern Power Grid, Guangzhou, China
- 作者简介：
  
  Yixi Chen received the B.S. degree in electrical engineering from South China University of Technology, Guangzhou, China, in 2022. He is currently pursuing a Ph.D. degree with the School of Electric Power Engineering, South China University of Technology. His current research interests include power system modeling and simulation, and data-driven power system operation optimization and stability control. Email:1402574623@qq.com
  Jizhong Zhu received the B.E., M.S., and Ph.D. degrees from Chongqing University, Chongqing, China, in 1985, 1987, and 1990, respectively. He is a Member of the European Academy of Sciences and Arts, a Foreign Corresponding Academician of the Academy of Sciences of the Institute of Bologna, and a Fellow of IEEE, IET, CSEE, AAIA, AIIA, and Alstom. He is currently a Professor in the School of Electric Power Engineering, South China University of Technology, Guangzhou, China. His research interests include power system operation and control, smart grid, microgrid, electric vehicle, and integrated smart energy system. Email:zhujz@scut.edu.cn
  Yun Liu received the B.Eng. (with First Class Hons.) and Ph.D. degrees from the College of Electrical Engineering, Zhejiang University, Hangzhou, China, in 2011 and 2016, respectively. His work experience includes the University of Central Florida, Orlando, USA, Nanyang Technological University, Singapore, and Shenzhen University, Shenzhen, China. He is currently an Associate Professor with the School of Electric Power Engineering, South China University of Technology, Guangzhou, China. His research interests include stability analysis of high power electronics-penetrated systems, and stability-constrained planning, operation, and control of inverter-based resource (IBR)-dominated power systems. Email:liuyun19881026@gmail.com
  Le Zhang received B.S., M.S., and Ph.D. degrees in electrical engineering from Beijing Jiaotong University, Beijing, China, Guangxi University, Nanning, China, and South China University of Technology, Guangzhou, China, in 2017, 2021, and 2025, respectively. He is currently working in the Electric Power Research Institute, China Southern Power Grid, Guangzhou, China. His research interests include artificial intelligence, renewable energy prediction technology, and home energy management system. Email:wjtclwwf@163.com
  Kaixin Lin received the B.S. degree in South China University of Technology, Guangzhou, China, in 2024. She is currently pursuing the M.S. degree in South China University of Technology. Her research interests include artificial intelligence, power system optimization, and home energy management system. Email:1437268348@qq.com
- 基金信息：
  
  National Natural Science Foundation of China(U25B6016)
- DOI：10.35833/MPCE.2025.000750
  中图分类号：
- 收稿：2025-08-21，
  
  修回：2025-11-23，
  
  录用：2026-02-05，
  
  网络首发：2026-03-30，
  
  纸质出版：2026-09-15
- 稿件说明：
移动端阅览
Yixi Chen, Jizhong Zhu, Yun Liu, 等. Knowledge-guided Hybrid Deep Reinforcement Learning for Transient Stability Emergency Control[J]. MPCE, 2026,14(5):1634-1646.

Chen Yixi,Zhu Jizhong,Liu Yun,et al.Knowledge-guided Hybrid Deep Reinforcement Learning for Transient Stability Emergency Control[J].Journal of Modern Power Systems and Clean Energy,2026,14(05):1634-1646.
Yixi Chen, Jizhong Zhu, Yun Liu, 等. Knowledge-guided Hybrid Deep Reinforcement Learning for Transient Stability Emergency Control[J]. MPCE, 2026,14(5):1634-1646. DOI： 10.35833/MPCE.2025.000750.

Chen Yixi,Zhu Jizhong,Liu Yun,et al.Knowledge-guided Hybrid Deep Reinforcement Learning for Transient Stability Emergency Control[J].Journal of Modern Power Systems and Clean Energy,2026,14(05):1634-1646. DOI： 10.35833/MPCE.2025.000750.

摘要

Abstract

Deep reinforcement learning (DRL) has been recognized as a promising alternative for emer

gency controls recently

for its rapid decision-making and strong policy searching capabilities. However

when applied to complex large-scale power systems

two prominent challenges emerge

①

high-dimensional discrete-continuous hybrid action space poses great adaptation difficulty for regular DRL methods

as they are designed primarily for purely discrete or continuous action spaces; and

②

unguided exploration within vast action spaces leads to inefficient convergence performance. To mitigate these limitations

this paper develops a knowledge-guided hybrid DRL-based method for transient stability emergency control. The proposed method adopts a novel hybrid policy to represent the hybrid action domain

and employs a novel derivative-free DRL for optimization

which eliminates the training instability issues associated with jointly optimizing gradients for different action types. Then

the prior domain knowledge is utilized to formulate mask rules and incorporate them into DRL training through trainable action mask (TAM) technique to guide exploration. Moreover

the proposed method is further integrated into a hierarchical DRL framework to alleviate computational complexity and enhance scalability. Case studies on the IEEE 300-bus system show that the proposed method improves solution quality by 52.9% and 42.8% compared with purely discrete and conventional hybrid action DRL methods

and achieves an 80.0% improvement in convergence efficiency compared with the method without knowledge guidance.

关键词

Keywords

references

L. Lu, Y. Zhou, H. Zeng et al. , “Robustness certification of neural networks for power system transient stability assessment,” IEEE Transactions on Power Systems , vol. 40 , no. 6 , pp. 5158 - 5170 , Nov. 2025 .

Y. Liu, R. Fan, and V. Terzija, “Power system restoration: a literature review from 2006 to 2016,” Journal of Modern Power Systems and Clean Energy , vol. 4 , no. 3 , pp. 332 - 341 , Jul. 2016 .

C. Chen, M. Cui, F. Li et al. , “Model-free emergency frequency control based on reinforcement learning,” IEEE Transactions on Industrial Informatics , vol. 17 , no. 4 , pp. 2336 - 2346 , Apr. 2021 .

P. Bhui and N. Senroy, “Real-time prediction and control of transient stability using transient energy function,” IEEE Transactions on Power Systems , vol. 32 , no. 2 , pp. 923 - 934 , Mar. 2017 .

Q. Jiang, Y. Wang, and G. Geng, “A parallel reduced-space interior point method with orthogonal collocation for first-swing stability constrained emergency control,” IEEE Transactions on Power Systems , vol. 29 , no. 1 , pp. 84 - 92 , Jan. 2014 .

T. Heins, M. Joševski, S. K. Gurumurthy et al. , “Centralized model predictive control for transient frequency control in islanded inverter-based microgrids,” IEEE Transactions on Power Systems , vol. 38 , no. 3 , pp. 2641 - 2652 , May 2023.

Z. Li, G. Yao, G. Geng et al. , “An efficient optimal control method for open-loop transient stability emergency control,” IEEE Transactions on Power Systems , vol. 32 , no. 4 , pp. 2704 - 2713 , Jul. 2017 .

G. Gan, Z. Zhu, G. Geng et al. , “An efficient parallel sequential approach for transient stability emergency control of large-scale power system,” IEEE Transactions on Power Systems , vol. 33 , no. 6 , pp. 5854 - 5864 , Nov. 2018 .

S. Yang, Z. Hao, B. Zhang et al. , “An accurate and fast start-up scheme for power system real-time emergency control,” IEEE Transactions on Power Systems , vol. 34 , no. 5 , pp. 3562 - 3572 , Sept. 2019.

A. R. R. Matavalam and V. Ajjarapu, “PMU-based monitoring and mitigation of delayed voltage recovery using admittances,” IEEE Transactions on Power Systems , vol. 34 , no. 6 , pp. 4451 - 4463 , Nov. 2019 .

V. Pradhan, A. M. Kulkarni, and S. A. Khaparde, “A model-free approach for emergency damping control using wide area measurements,” IEEE Transactions on Power Systems , vol. 33 , no. 5 , pp. 4902 - 4912 , Sept. 2018.

Q. Zong, W. Yao, H. Zhou et al. , “Transient stability assessment of large-scale power system using predictive maximal Lyapunov exponent approach,” IEEE Transactions on Power Systems , vol. 39 , no. 3 , pp. 5163 - 5176 , May 2024.

Y. Li, W. Gao, S. Huang et al. , “Data-driven optimal control strategy for virtual synchronous generator via deep reinforcement learning approach,” Journal of Modern Power Systems and Clean Energy , vol. 9 , no. 4 , pp. 919 - 929 , Jul. 2021 .

Q. Huang, R. Huang, W. Hao et al. , “Adaptive power system emergency control using deep reinforcement learning,” IEEE Transactions on Smart Grid , vol. 11 , no. 2 , pp. 1171 - 1182 , Mar. 2020 .

R. R. Hossain, Q. Huang, and R. Huang, “Graph convolutional network-based topology embedded deep reinforcement learning for voltage stability control,” IEEE Transactions on Power Systems , vol. 36 , no. 5 , pp. 4848 - 4851 , Sept. 2021.

R. Huang, Y. Chen, T. Yin et al. , “Accelerated derivative-free deep reinforcement learning for large-scale grid emergency voltage control,” IEEE Transactions on Power Systems , vol. 37 , no. 1 , pp. 14 - 25 , Jan. 2022 .

R. Huang, Y. Chen, T. Yin et al. , “Learning and fast adaptation for grid emergency control via deep meta reinforcement learning,” IEEE Transactions on Power Systems , vol. 37 , no. 6 , pp. 4168 - 4178 , Nov. 2022 .

Y. Du, Q. Huang, R. Huang et al. , “Physics-informed evolutionary strategy based control for mitigating delayed voltage recovery,” IEEE Transactions on Power Systems , vol. 37 , no. 5 , pp. 3516 - 3527 , Sept. 2022.

Y. Zhang, M. Yue, J. Wang et al. , “Multi-agent graph-attention deep reinforcement learning for post-contingency grid emergency voltage control,” IEEE Transactions on Neural Networks and Learning Systems , vol. 35 , no. 3 , pp. 3340 - 3350 , Mar. 2024 .

Y. Ma, Z. Hu, and Y. Song, “Dynamic nonlinear droop-based fast frequency regulation for power systems with flexible resources using meta-reinforcement learning approach,” Journal of Modern Power Systems and Clean Energy , vol. 13 , no. 2 , pp. 379 - 390 , Mar. 2025 .

Y. Chen, J. Zhu, and L. Zhang, “Distributed hybrid deep reinforcement learning for grid emergency control,” in Proceedings of the IEEE PES General Meeting , Seattle, USA, Jul. 2024 , pp. 1 - 5 .

Y. Chen, J. Zhu, Y. Liu et al. , “Distributed hierarchical deep reinforcement learning for large-scale grid emergency control,” IEEE Transactions on Power Systems , vol. 39 , no. 2 , pp. 4446 - 4458 , Mar. 2024 .

T. Schaul, J. Quan, I. Antonoglou et al. , “Prioritized experience replay,” in Proceedings of the International Conference on Learning Representations , San Juan, Puerto Rico, May 2016, pp. 1 - 21 .

P. Chou, D. Maturana, and S. Scherer, “Improving stochas tic policy gradients in continuous control with deep reinforcement learning using the beta distribution,” in Proceedings of the International Conference on Machine Learning , Sydney, Australia, Aug. 2017 , pp. 834 - 843 .

Y. Chen, J. Zhu, H. Dong et al. , “Knowledge-augmented population-based deep reinforcement learning for real-time network-constrained economic dispatch of large-scale power grid,” IEEE Transactions on Power Systems , vol. 40 , no. 6 , pp. 5369 - 5384 , Nov. 2025 .

L. Liu and G.-H. Yang, “Distributed dynamic event-triggered optimal economic dispatch for multienergy systems with prescribed-time convergence,” IEEE Transactions on Industrial Informatics , vol. 21 , no. 6 , pp. 4533 - 4542 , Jun. 2025 .

L. Liu, Y. Huang, and C. Deng. (2025, Mar.). Accelerated distributed cooperative energy management for integrated energy systems. [Online]. Available: https://www.ieee-jas.net/en/article/doi/10.1109/JAS. 2025.125489

S. Huang and S. Ontanon. (2020, Jun.). A closer look at invalid action masking in policy gradient algorithms. [Online]. Available: https://arxiv.org/abs/2006.14171

F. Milano, “An open source power system a nalysis toolbox,” IEEE Transactions on Power Systems , vol. 20 , no. 3 , pp. 1199 - 1206 , Aug. 2005 .

P. Moritz, R. Nishihara, S. Wang et al. , “Ray: a distributed framework for emerging AI applications,” in Proceedings of the 13th USENIX Symposium on Operating Systems Design and Implementation , Carlsbad, USA, Oct. 2018 , pp. 561 - 577 .

A. Farraj, E. Hammad, and D. Kundur, “A cyber-enabled stabilizing control scheme for resilient smart grid systems,” IEEE Transactions on Power Systems , vol. 7 , no. 4 , pp. 1856 - 1865 , Jul. 2016 .

D. E. Bernal, Q. Chen, F. Gong et al. , “Mixed-integer nonlinear decomposition toolbox for Pyomo (MindtPy),” Computer Aided Chemical Engineering , vol. 44 , pp. 895 - 900 , Aug. 2018 .

T. Xiao, Y. Chen, H. Diao et al. , “Fast-converging deep reinforcement learning for optimal dispatch of large-scale power systems under transient security constraints,” Journal of Modern Power Systems and Clean Energy , vol. 13 , no. 5 , pp. 1495 - 1506 , Sept. 2025.

C. Bi, D. Liu, L. Zhu et al. , “Short-term voltage stability emergency control strategy pre-formulation for massive operating scenarios via adversarial reinforcement learning,” Applied Energy , vol. 389 , p. 125751 , Jul. 2025 .

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

Transient Stability Enhancement of Grid-forming Inverter-based Resources by Optimizing Angle of Saturated Current

Poincaré-type Virtual Oscillator Control and Passivity-based Transient Stability Enhancement Method for Grid-forming Inverters

Tree-partitioning Based Emergency Control for Mitigating Cascading Failures in Power Systems

Transient Stability Analysis and Control of VSG-controlled Multi-paralleled Photovoltaic Generation Systems Under Asymmetrical Grid Faults

Active and Reactive Power Coordination Control for Voltage Control in Active Distribution Networks Based on LCCIN-SAC Algorithm