李柏堉, 赵津蔓, 韩肖清, 杨晶. 基于双智能体深度强化学习的电力系统静态安全预防控制方法[J]. 中国电机工程学报, 2023, 43(5): 1818-1830. DOI: 10.13334/j.0258-8013.pcsee.220005
引用本文: 李柏堉, 赵津蔓, 韩肖清, 杨晶. 基于双智能体深度强化学习的电力系统静态安全预防控制方法[J]. 中国电机工程学报, 2023, 43(5): 1818-1830. DOI: 10.13334/j.0258-8013.pcsee.220005
LI Baiyu, ZHAO Jinman, HAN Xiaoqing, YANG Jing. Static Security Oriented Preventive Control of Power System Based on Double Deep Reinforcement Learning[J]. Proceedings of the CSEE, 2023, 43(5): 1818-1830. DOI: 10.13334/j.0258-8013.pcsee.220005
Citation: LI Baiyu, ZHAO Jinman, HAN Xiaoqing, YANG Jing. Static Security Oriented Preventive Control of Power System Based on Double Deep Reinforcement Learning[J]. Proceedings of the CSEE, 2023, 43(5): 1818-1830. DOI: 10.13334/j.0258-8013.pcsee.220005

基于双智能体深度强化学习的电力系统静态安全预防控制方法

Static Security Oriented Preventive Control of Power System Based on Double Deep Reinforcement Learning

  • 摘要:N-1”静态安全校验是电力系统安全稳定分析的重要内容,当系统不满足静态安全性时,需要采取预防控制,而调整发电机出力是最重要的预防控制措施。传统的方法是依据专家知识和经验做尝试性进行发电机功率调整,需耗费较多时间;深度强化学习具有“离线训练、在线端对端形成策略”优点,在电力系统预防控制中有很好的应用前景,但如何缩小搜索空间、提高训练速度,是需要解决的问题。该文提出一种基于柔性动作-评价深度强化学习算法的双智能体发电机调整方法。考虑到输电网具有PQ可解耦这一特点,设计了集中式训练的合作型双智能体结构,由两个智能体分别承担发电机有功功率调整和电压调整任务,相互合作,有效减少了搜索空间,提高了模型的稳定性,并根据不同运行方式下全网“N-1”校验时线路负载和节点电压判断系统安全性,且结合效用理论设计了奖励函数,进一步提高了收敛速度。IEEE 39节点系统算例表明,所提方法得到的智能体在多种运行方式下可快速有效生成预防控制策略,验证了所提方法的有效性。

     

    Abstract: The N-1 static security verification is an important part of the power system security and stability analysis. When the system can not meet the requirements of static security, appropriate preventive control measures should be taken. It is the most important preventive control measure to adjust generators' power. The traditional method for the control strategy decision is based on experts' knowledge and experience in combination with digital simulation, which usually takes long time. Deep reinforcement learning, having the advantage of "offline training, online end-to-end strategy formation", is suitable for application in power system preventive control decision. However, how to reduce the search space and improve the training speed is a problem that needs to be solved. In this paper, a double-agents generator power adjustment method is proposed based on soft actor-critic (SAC) deep reinforcement learning algorithm. Considering that power transmission network has the feature of PQ decoupling, a cooperative double-agents structure with centralized training is designed. In the structure, two agents take on the tasks of active power adjustment and voltage regulation of generators respectively and cooperate with each other according to the N-1 verification of the whole power system under different operation states, and the search space for the control strategy can be reduced and the model stability can be improved effectively. Case studies on the IEEE 39-bus system show that the proposed method can quickly and effectively generate preventive control strategies under various operation states, which verifies the effectiveness of the proposed method.

     

/

返回文章
返回