Knowledge-guided Hybrid Deep Reinforcement Learning for Transient Stability Emergency Control

Yixi Chen; Jizhong Zhu; Yun Liu; Le Zhang; Kaixin Lin

doi:10.35833/MPCE.2025.000750

English Version Journal Login

您当前的位置：

首页 >

文章列表页 >

Knowledge-guided Hybrid Deep Reinforcement Learning for Transient Stability Emergency Control

Original Paper | 更新时间：2026-09-28

- Knowledge-guided Hybrid Deep Reinforcement Learning for Transient Stability Emergency Control
- Journal of Modern Power Systems and Clean Energy Vol. 14, Issue 5, Pages: 1634-1646(2026)
- 作者机构：
  
  1.School of Electric Power Engineering, South China University of Technology, Guangzhou, China
  2.Electric Power Research Institute, China Southern Power Grid, Guangzhou, China
- 作者简介：
  
  Yixi Chen received the B.S. degree in electrical engineering from South China University of Technology, Guangzhou, China, in 2022. He is currently pursuing a Ph.D. degree with the School of Electric Power Engineering, South China University of Technology. His current research interests include power system modeling and simulation, and data-driven power system operation optimization and stability control. Email:1402574623@qq.com
  Jizhong Zhu received the B.E., M.S., and Ph.D. degrees from Chongqing University, Chongqing, China, in 1985, 1987, and 1990, respectively. He is a Member of the European Academy of Sciences and Arts, a Foreign Corresponding Academician of the Academy of Sciences of the Institute of Bologna, and a Fellow of IEEE, IET, CSEE, AAIA, AIIA, and Alstom. He is currently a Professor in the School of Electric Power Engineering, South China University of Technology, Guangzhou, China. His research interests include power system operation and control, smart grid, microgrid, electric vehicle, and integrated smart energy system. Email:zhujz@scut.edu.cn
  Yun Liu received the B.Eng. (with First Class Hons.) and Ph.D. degrees from the College of Electrical Engineering, Zhejiang University, Hangzhou, China, in 2011 and 2016, respectively. His work experience includes the University of Central Florida, Orlando, USA, Nanyang Technological University, Singapore, and Shenzhen University, Shenzhen, China. He is currently an Associate Professor with the School of Electric Power Engineering, South China University of Technology, Guangzhou, China. His research interests include stability analysis of high power electronics-penetrated systems, and stability-constrained planning, operation, and control of inverter-based resource (IBR)-dominated power systems. Email:liuyun19881026@gmail.com
  Le Zhang received B.S., M.S., and Ph.D. degrees in electrical engineering from Beijing Jiaotong University, Beijing, China, Guangxi University, Nanning, China, and South China University of Technology, Guangzhou, China, in 2017, 2021, and 2025, respectively. He is currently working in the Electric Power Research Institute, China Southern Power Grid, Guangzhou, China. His research interests include artificial intelligence, renewable energy prediction technology, and home energy management system. Email:wjtclwwf@163.com
  Kaixin Lin received the B.S. degree in South China University of Technology, Guangzhou, China, in 2024. She is currently pursuing the M.S. degree in South China University of Technology. Her research interests include artificial intelligence, power system optimization, and home energy management system. Email:1437268348@qq.com
- 基金信息：
- DOI：10.35833/MPCE.2025.000750
  CLC：
- Received：21 August 2025，
  
  Revised：2025-11-23，
  
  Accepted：05 February 2026，
  
  Online First：30 March 2026，
  
  Published：15 September 2026
- 稿件说明：
移动端阅览
Chen Yixi,Zhu Jizhong,Liu Yun,et al.Knowledge-guided Hybrid Deep Reinforcement Learning for Transient Stability Emergency Control[J].Journal of Modern Power Systems and Clean Energy,2026,14(05):1634-1646.
DOI：

Chen Yixi,Zhu Jizhong,Liu Yun,et al.Knowledge-guided Hybrid Deep Reinforcement Learning for Transient Stability Emergency Control[J].Journal of Modern Power Systems and Clean Energy,2026,14(05):1634-1646. DOI： 10.35833/MPCE.2025.000750.

摘要

Abstract

Deep reinforcement learning (DRL) has been recognized as a promising alternative for emer

gency controls recently

for its rapid decision-making and strong policy searching capabilities. However

when applied to complex large-scale power systems

two prominent challenges emerge

①

high-dimensional discrete-continuous hybrid action space poses great adaptation difficulty for regular DRL methods

as they are designed primarily for purely discrete or continuous action spaces; and

②

unguided exploration within vast action spaces leads to inefficient convergence performance. To mitigate these limitations

this paper develops a knowledge-guided hybrid DRL-based method for transient stability emergency control. The proposed method adopts a novel hybrid policy to represent the hybrid action domain

and employs a novel derivative-free DRL for optimization

which eliminates the training instability issues associated with jointly optimizing gradients for different action types. Then

the prior domain knowledge is utilized to formulate mask rules and incorporate them into DRL training through trainable action mask (TAM) technique to guide exploration. Moreover

the proposed method is further integrated into a hierarchical DRL framework to alleviate computational complexity and enhance scalability. Case studies on the IEEE 300-bus system show that the proposed method improves solution quality by 52.9% and 42.8% compared with purely discrete and conventional hybrid action DRL methods

and achieves an 80.0% improvement in convergence efficiency compared with the method without knowledge guidance.

关键词

Keywords

references

L. Lu, Y. Zhou, H. Zeng et al. , “Robustness certification of neural networks for power system transient stability assessment,” IEEE Transactions on Power Systems , vol. 40 , no. 6 , pp. 5158 - 5170 , Nov. 2025 .

Y. Liu, R. Fan, and V. Terzija, “Power system restoration: a literature review from 2006 to 2016,” Journal of Modern Power Systems and Clean Energy , vol. 4 , no. 3 , pp. 332 - 341 , Jul. 2016 .

C. Chen, M. Cui, F. Li et al. , “Model-free emergency frequency control based on reinforcement learning,” IEEE Transactions on Industrial Informatics , vol. 17 , no. 4 , pp. 2336 - 2346 , Apr. 2021 .

P. Bhui and N. Senroy, “Real-time prediction and control of transient stability using transient energy function,” IEEE Transactions on Power Systems , vol. 32 , no. 2 , pp. 923 - 934 , Mar. 2017 .

Q. Jiang, Y. Wang, and G. Geng, “A parallel reduced-space interior point method with orthogonal collocation for first-swing stability constrained emergency control,” IEEE Transactions on Power Systems , vol. 29 , no. 1 , pp. 84 - 92 , Jan. 2014 .

T. Heins, M. Joševski, S. K. Gurumurthy et al. , “Centralized model predictive control for transient frequency control in islanded inverter-based microgrids,” IEEE Transactions on Power Systems , vol. 38 , no. 3 , pp. 2641 - 2652 , May 2023.

Z. Li, G. Yao, G. Geng et al. , “An efficient optimal control method for open-loop transient stability emergency control,” IEEE Transactions on Power Systems , vol. 32 , no. 4 , pp. 2704 - 2713 , Jul. 2017 .

G. Gan, Z. Zhu, G. Geng et al. , “An efficient parallel sequential approach for transient stability emergency control of large-scale power system,” IEEE Transactions on Power Systems , vol. 33 , no. 6 , pp. 5854 - 5864 , Nov. 2018 .

S. Yang, Z. Hao, B. Zhang et al. , “An accurate and fast start-up scheme for power system real-time emergency control,” IEEE Transactions on Power Systems , vol. 34 , no. 5 , pp. 3562 - 3572 , Sept. 2019.

A. R. R. Matavalam and V. Ajjarapu, “PMU-based monitoring and mitigation of delayed voltage recovery using admittances,” IEEE Transactions on Power Systems , vol. 34 , no. 6 , pp. 4451 - 4463 , Nov. 2019 .

V. Pradhan, A. M. Kulkarni, and S. A. Khaparde, “A model-free approach for emergency damping control using wide area measurements,” IEEE Transactions on Power Systems , vol. 33 , no. 5 , pp. 4902 - 4912 , Sept. 2018.

Q. Zong, W. Yao, H. Zhou et al. , “Transient stability assessment of large-scale power system using predictive maximal Lyapunov exponent approach,” IEEE Transactions on Power Systems , vol. 39 , no. 3 , pp. 5163 - 5176 , May 2024.

Y. Li, W. Gao, S. Huang et al. , “Data-driven optimal control strategy for virtual synchronous generator via deep reinforcement learning approach,” Journal of Modern Power Systems and Clean Energy , vol. 9 , no. 4 , pp. 919 - 929 , Jul. 2021 .

Q. Huang, R. Huang, W. Hao et al. , “Adaptive power system emergency control using deep reinforcement learning,” IEEE Transactions on Smart Grid , vol. 11 , no. 2 , pp. 1171 - 1182 , Mar. 2020 .

R. R. Hossain, Q. Huang, and R. Huang, “Graph convolutional network-based topology embedded deep reinforcement learning for voltage stability control,” IEEE Transactions on Power Systems , vol. 36 , no. 5 , pp. 4848 - 4851 , Sept. 2021.

R. Huang, Y. Chen, T. Yin et al. , “Accelerated derivative-free deep reinforcement learning for large-scale grid emergency voltage control,” IEEE Transactions on Power Systems , vol. 37 , no. 1 , pp. 14 - 25 , Jan. 2022 .

R. Huang, Y. Chen, T. Yin et al. , “Learning and fast adaptation for grid emergency control via deep meta reinforcement learning,” IEEE Transactions on Power Systems , vol. 37 , no. 6 , pp. 4168 - 4178 , Nov. 2022 .

Y. Du, Q. Huang, R. Huang et al. , “Physics-informed evolutionary strategy based control for mitigating delayed voltage recovery,” IEEE Transactions on Power Systems , vol. 37 , no. 5 , pp. 3516 - 3527 , Sept. 2022.

Y. Zhang, M. Yue, J. Wang et al. , “Multi-agent graph-attention deep reinforcement learning for post-contingency grid emergency voltage control,” IEEE Transactions on Neural Networks and Learning Systems , vol. 35 , no. 3 , pp. 3340 - 3350 , Mar. 2024 .

Y. Ma, Z. Hu, and Y. Song, “Dynamic nonlinear droop-based fast frequency regulation for power systems with flexible resources using meta-reinforcement learning approach,” Journal of Modern Power Systems and Clean Energy , vol. 13 , no. 2 , pp. 379 - 390 , Mar. 2025 .

Y. Chen, J. Zhu, and L. Zhang, “Distributed hybrid deep reinforcement learning for grid emergency control,” in Proceedings of the IEEE PES General Meeting , Seattle, USA, Jul. 2024 , pp. 1 - 5 .

Y. Chen, J. Zhu, Y. Liu et al. , “Distributed hierarchical deep reinforcement learning for large-scale grid emergency control,” IEEE Transactions on Power Systems , vol. 39 , no. 2 , pp. 4446 - 4458 , Mar. 2024 .

T. Schaul, J. Quan, I. Antonoglou et al. , “Prioritized experience replay,” in Proceedings of the International Conference on Learning Representations , San Juan, Puerto Rico, May 2016, pp. 1 - 21 .

P. Chou, D. Maturana, and S. Scherer, “Improving stochas tic policy gradients in continuous control with deep reinforcement learning using the beta distribution,” in Proceedings of the International Conference on Machine Learning , Sydney, Australia, Aug. 2017 , pp. 834 - 843 .

Y. Chen, J. Zhu, H. Dong et al. , “Knowledge-augmented population-based deep reinforcement learning for real-time network-constrained economic dispatch of large-scale power grid,” IEEE Transactions on Power Systems , vol. 40 , no. 6 , pp. 5369 - 5384 , Nov. 2025 .

L. Liu and G.-H. Yang, “Distributed dynamic event-triggered optimal economic dispatch for multienergy systems with prescribed-time convergence,” IEEE Transactions on Industrial Informatics , vol. 21 , no. 6 , pp. 4533 - 4542 , Jun. 2025 .

L. Liu, Y. Huang, and C. Deng. (2025, Mar.). Accelerated distributed cooperative energy management for integrated energy systems. [Online]. Available: https://www.ieee-jas.net/en/article/doi/10.1109/JAS. 2025.125489

S. Huang and S. Ontanon. (2020, Jun.). A closer look at invalid action masking in policy gradient algorithms. [Online]. Available: https://arxiv.org/abs/2006.14171

F. Milano, “An open source power system a nalysis toolbox,” IEEE Transactions on Power Systems , vol. 20 , no. 3 , pp. 1199 - 1206 , Aug. 2005 .

P. Moritz, R. Nishihara, S. Wang et al. , “Ray: a distributed framework for emerging AI applications,” in Proceedings of the 13th USENIX Symposium on Operating Systems Design and Implementation , Carlsbad, USA, Oct. 2018 , pp. 561 - 577 .

A. Farraj, E. Hammad, and D. Kundur, “A cyber-enabled stabilizing control scheme for resilient smart grid systems,” IEEE Transactions on Power Systems , vol. 7 , no. 4 , pp. 1856 - 1865 , Jul. 2016 .

D. E. Bernal, Q. Chen, F. Gong et al. , “Mixed-integer nonlinear decomposition toolbox for Pyomo (MindtPy),” Computer Aided Chemical Engineering , vol. 44 , pp. 895 - 900 , Aug. 2018 .

T. Xiao, Y. Chen, H. Diao et al. , “Fast-converging deep reinforcement learning for optimal dispatch of large-scale power systems under transient security constraints,” Journal of Modern Power Systems and Clean Energy , vol. 13 , no. 5 , pp. 1495 - 1506 , Sept. 2025.

C. Bi, D. Liu, L. Zhu et al. , “Short-term voltage stability emergency control strategy pre-formulation for massive operating scenarios via adversarial reinforcement learning,” Applied Energy , vol. 389 , p. 125751 , Jul. 2025 .

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Transient Stability Enhancement of Grid-forming Inverter-based Resources by Optimizing Angle of Saturated Current

Poincaré-type Virtual Oscillator Control and Passivity-based Transient Stability Enhancement Method for Grid-forming Inverters

Tree-partitioning Based Emergency Control for Mitigating Cascading Failures in Power Systems

Transient Stability Analysis and Control of VSG-controlled Multi-paralleled Photovoltaic Generation Systems Under Asymmetrical Grid Faults

Active and Reactive Power Coordination Control for Voltage Control in Active Distribution Networks Based on LCCIN-SAC Algorithm

Related Author

Ali Arjomandi-Nezhad

Bikash C. Pal

Xiangyu Wu

Shunyu Yao

Yin Xu

Yafan Shi

Josep M. Guerrero

Jian Xu

Related Institution

Department of Electrical and Electronics Engineering at Imperial College London, London SW7 2AZ, United Kingdom

AAU Energy, Aalborg University, Aalborg

Electrical Engineering Department, University of Valladolid

Huanjiang Laboratory, Zhejiang University

School of Electrical Engineering, Beijing Jiaotong University

AI问答

⁰