基于Actor-Critic框架的风机换流器参数优化策略

李强; 邹小明; 任必兴; 何宇帆; 杜文娟

doi:10.19725/j.cnki.1007-2322.2023.0310

欢迎来到能源电力期刊网！

English Version 读者登录期刊登录

您当前的位置：

首页 >

文章列表页 >

基于Actor-Critic框架的风机换流器参数优化策略

电力装备制造技术 | 更新时间：2025-10-22

- 基于Actor-Critic框架的风机换流器参数优化策略
- Parameter Optimization Strategy for Wind Generator Converters Based on Actor-Critic Framework
- 现代电力 2025年42卷第5期页码：1101-1110
- 作者机构：
  
  1. 国网江苏省电力有限公司电力科学研究院,江苏省,南京市,211103
  2. 四川大学电气工程学院,四川省,成都市,610065
- 作者简介：
  
  [ "李强(1981)，男，博士，研究员级高级工程师，研究方向为新能源运行分析与并网控制，E-mail：35830342@qq.com" ]
  [ "邹小明(1992)，男，博士，工程师，研究方向为新能源建模分析、并网控制，E-mail：zouxming@126.com" ]
  [ "任必兴(1988)，男，博士，工程师，通信作者，研究方向为大规模新能源、柔性输电接入电力系统的稳定分析与控制、电力系统宽频振荡机理分析与抑制等，E-mail：renbixing@126.com" ]
  [ "何宇帆(1998)，男，博士研究生，通信作者，研究方向为新能源发电并网控制与稳定性分析以及人工智能技术在电力系统中的应用，E-mail：919357741@qq.com" ]
  [ "杜文娟(1979)，女，博士，教授，研究方向为电力系统稳定性分析与控制，E-mail：ddwenjuan@qq.com" ]
- 基金信息：
  
  国网江苏省电力有限公司科技项目(海上风电并网诱发宽频振荡的溯源及抑制技术研究，J2022003)。
- DOI：10.19725/j.cnki.1007-2322.2023.0310
  中图分类号： 5
- 纸质出版：2025
- 稿件说明：
移动端阅览
李强, 邹小明, 任必兴, 等. 基于Actor-Critic框架的风机换流器参数优化策略[J]. 现代电力, 2025,42(5):1101-1110.

LI Qiang, ZOU Xiaoming, REN Bixing, et al. Parameter Optimization Strategy for Wind Generator Converters Based on Actor-Critic Framework[J]. 2025, 42(5): 1101-1110.
李强, 邹小明, 任必兴, 等. 基于Actor-Critic框架的风机换流器参数优化策略[J]. 现代电力, 2025,42(5):1101-1110. DOI： 10.19725/j.cnki.1007-2322.2023.0310.

LI Qiang, ZOU Xiaoming, REN Bixing, et al. Parameter Optimization Strategy for Wind Generator Converters Based on Actor-Critic Framework[J]. 2025, 42(5): 1101-1110. DOI： 10.19725/j.cnki.1007-2322.2023.0310.

摘要

随着可再生能源并网发电量的不断增加，由电力电子设备引发的电力系统次同步振荡问题逐渐凸显，给电力系统的安全稳定运行带来了新的挑战。除此之外，当目标电力系统规模较大时，常用的基于线性化模型的分析方法面临着维数灾难。为了解决上述问题，根据强化学习原理，通过动作-评价(Actor-Critic)学习框架提出一种对风机换流器控制参数的优化策略。通过搜集永磁直驱风机(permanent magnetic synchronous generator，PMSG)运行状态数据，训练强化学习代理(Agent)，以此评估风机运行状态及其稳定性，并寻找优化风机换流器参数的最优策略。该训练方法得到的代理能够基于时域采样数据对风机换流器参数进行优化，从而有效抑制由于换流器诱发的振荡现象，在没有建立线性化分析模型的情况下，能够有效优化并增强电力系统的稳定性。经实验验证，该优化策略在采样数据有噪声干扰的情况下仍然具有良好的优化性能。

Abstract

With the increasing power generation capacity of renewable energy connected to the grid

the issue of power system sub-synchronous oscillation caused by power electronic equipment becomes prominent

posing new challenges to the safe and stable operation of the power system. The commonly used analysis methods based on linearized models encounter the problem of dimensionality disaster when dealing with large-scale target power systems. To address the above issues

in this paper we propose an optimization strategy for the control parameters of wind generator converters based on the principle of reinforcement learning and utilizes the actor-critic learning framework. With the collected operating status data of permanent magnet synchronous generator (PMSG)

a reinforcement learning agent (Agent) is trained to evaluate the PMSG's operating status and stability based on the proposed return function and determines an optimal strategy for wind generator converter parameters optimization. The agent obtained by this training method can optimize the parameters of wind generator converters based on time-domain sampling data

effectively suppressing the oscillation phenomenon induced by the converters. Moreover

it is not only able to optimize but also enhance the stability of the power system without the need for a linearized analysis model. The optimization strategy demonstrates superior optimization performance in the presence of noise interference in the sampled data

as confirmed through verification.

关键词

Keywords

references

DEMELLO F P, ONCORDIA C. Concepts of synchronous machine stability as affected by excitation control[J]. IEEE Transactions on Power Apparatus and Systems, 1969, PAS-88(4): 316−329.

LIU Shiyu, XIE Xiaorong, WANG Zhonghong. SSR problem in compensated transmission system of thermal power bases in China[J]. Power System Technology, 2008, 32(1): 5−8(in Chinese).

刘世宇, 谢小荣, 王仲鸿. 我国火电基地串补输电系统的次同步谐振问题[J]. 电网技术, 2008, 32(1): 5−8.

CHEN Lujie, XU Shiyun, SUN Huadong, et al. A survey on wide-frequency oscillation for power systems with high penetration of power electronics[J]. Proceedings of the CSEE, 2021, 41(07): 2297−2310 (in Chinese).

陈露洁, 徐式蕴, 孙华东, 等. 高比例电力电子电力系统宽频带振荡研究综述[J]. 中国电机工程学报, 2021, 41(07): 2297−2310.

WANG Yuhong, MIN Yangxi, YU Guangyuan, et al. Parameters coordinated optimization of subsynchronous oscillation of doubly fed induction generator system based on impedance sensitivity analysis with Sobol method[J]. High Voltage Engineering, 2023, 49(4): 1703−1713(in Chinese).

王渝红, 闵杨晰, 于光远, 等. 基于Sobol方法阻抗灵敏度分析的双馈风机系统次同步振荡参数协调优化[J]. 高电压技术, 2023, 49(4): 1703−1713.

JIN Guobin, YANG Mingcheng, LI Guoqing, et al. Optimization design of controller parameters for an offshore wind power DC output system by diode rectifier considering small signal stability[J]. Power System Protection and Control, 2022, 50(21): 65−74(in Chinese).

金国彬, 杨明城, 李国庆, 等. 考虑小干扰稳定的海上风电经不控整流直流送出系统控制器参数优化设计[J]. 电力系统保护与控制, 2022, 50(21): 65−74.

WANG Xiang, SU Yuxiang, SHEN Xiaoqun. Intelligent optimization of parameters for wind power SSCI suppression based on MVO algorithm[J]. Automation and Instrumentation, 2021, 36(03): 6−10(in Chinese).

王翔, 苏玉香, 沈晓群. 基于MVO算法针对风电SSCI抑制的参数智能优化[J]. 自动化与仪表, 2021, 36(03): 6−10.

WU Xi, WANG Mengting, SHI Xingyu, et al. SSO supplementary damping control method for DFIG based on model-free adaptive control[J]. Proceedings of the CSEE, 2022, 42(10): 3601−3614(in Chinese).

吴熙, 王梦婷, 施星宇, 等. 基于无模型自适应控制的双馈风机次同步振荡附加阻尼控制方法[J]. 中国电机工程学报, 2022, 42(10): 3601−3614.

VARMA R K, SALEHI R. SSR mitigation with a new control of PV solar farm as STATCOM (PV-STATCOM)[J]. IEEE Transactions on Sustainable Energy, 2017, 8(4): 1473−1483.

LIU Wei, LIU Jie, WU Xiaodan, et al. Identifying impedance models of wind farms with field tests[J]. Proceedings of the CSEE, 2020, 40(S1): 91−97(in Chinese).

刘威, 刘杰, 吴小丹, 等. 风电场阻抗模型的现场测试[J]. 中国电机工程学报, 2020, 40(S1): 91−97.

REN Bixing, DU Wenjuan, WANG Haifeng. Analysis on sub-synchronous control interaction between static synchronous compensator and permanent magnet synchronous generator[J]. Transactions of China Electrotechnical Society, 2018, 33(24): 5884−5896(in Chinese).

任必兴, 杜文娟, 王海风. 静止同步补偿器与直驱永磁风机的次同步控制交互研究[J]. 电工技术学报, 2018, 33(24): 5884−5896.

DONG Wenkai, DU Wenjuan, WANG Haifeng. Reduced-order modal computation method for small-signal stability examination of a wind farm[J]. Transactions of China Electrotechnical Society, 2021, 36(07): 1468−1479(in Chinese).

董文凯, 杜文娟, 王海风. 用于风电场小干扰稳定性检验的降阶模式计算方法[J]. 电工技术学报, 2021, 36(07): 1468−1479.

WANG Linguang, XIE Xiaorong, XU Tao, et al. Oscillatory stability analysis method based on the logarithmic derivative calculation of aggregated impedance determinant[J]. Proceedings of the CSEE, 2023, 43(11): 4254−4261(in Chinese).

汪林光, 谢小荣, 许涛, 等. 基于聚合阻抗行列式对数导数计算的振荡稳定性分析方法[J]. 中国电机工程学报, 2023, 43(11): 4254−4261.

卢强, 梅生伟, 孙元章. 电力系统非线性控制[M]. 2版. 北京: 清华大学出版社, 2008.

SUN Yuwei, ZHU Jilin, FU Chao. Nonlinear integrated control strategy of cascaded power electronic transformer[J]. Proceedings of the CSEE, 2020, 40(09): 2891−2903(in Chinese).

孙玉巍, 朱吉林, 付超. 级联式电力电子变压器非线性综合控制策略[J]. 中国电机工程学报, 2020, 40(09): 2891−2903.

BISHOP C. Pattern recognition and machine learning[M]. New York: Springer, 2006.

RICHARD S S, ANDREW G B. Reinforcement learning: an introduction[M]. 2nd Edition. Cambridge: The MIT Press, 2018.

YANG Siming, SHAN Zheng, DING Yu, et al. Survey of research on reinforcement learning[J]. Computer Engineering, 2021(12): 047(in Chinese).

杨思明, 单征, 丁煜, 等. 深度强化学习研究综述[J]. 计算机工程, 2021(12): 047.

BANNA H U, SOLANKI S K, SOLANKI J. Data-driven disturbance source identification for power system oscillations using credibility search ensemble learning[J]. IET Smart Grid, 2019, 2(2): 293−300.

CHEN Jian, DU Wenjuan, WANG Haifeng. A method of locating the power system subsynchronous oscillation source unit with grid-connected PMSG using deep transfer learning[J]. Transactions of China Electrotechnical Society, 2021, 36(1): 179−190(in Chinese).

陈剑, 杜文娟, 王海风. 采用深度迁移学习定位含直驱风机次同步振荡源机组的方法[J]. 电工技术学报, 2021, 36(1): 179−190.

LI Peiping, ZENG Lingkang, WEN Jianfeng, et al. Adaptive wide-area coordinated damping control considering bidirectional random communication time-delay compensation[J]. Power System Technology, 2020, 44(10): 3937−3948(in Chinese).

李培平, 曾令康, 文剑峰, 等. 考虑双向随机通信时滞补偿的自适应广域协调阻尼控制[J]. 电网技术, 2020, 44(10): 3937−3948.

HUANG Renke, CHEN Yujiao, YIN Tianzhixi, et al. Learning and fast adaptation for grid emergency control via deep meta reinforcement learning[J]. IEEE Transactions on Power Systems, 2022, 37(6): 4168−4178.

LI S, HASKEW T A, SWATLOSKI R P, et al. Optimal and direct-current vector control of direct-driven PMSG wind turbines[J]. IEEE Trans. on Power Electronics, 2012, 27(5): 2325−2337.

程时杰, 曹一家, 江全元. 电力系统次同步振荡的理论与方法[M]. 北京: 科学出版社, 2009.

WANG Yuhong, LI Chenxin, ZHOU Xu, et al. Localization method of wide-band oscillation disturbance sources based on compressed sensing and graph convolutional neural networks[J]. High Voltage Engineering, 2024, 50(3): 1080−1089(in Chinese).

王渝红, 李晨鑫, 周旭, 等. 压缩感知和图卷积神经网络相结合的宽频振荡扰动源定位方法[J]. 高电压技术, 2024, 50(3): 1080−1089.

LI Gaowang, ZHANG Zhi, LI Da, et al. Coherency clustering method based on Prony analysis feature extraction[J]. Power System Protection and Control, 2020, 48(22): 91−99(in Chinese).

李高望, 张智, 李达, 等. 基于Prony分析特征提取的同调机组分群方法[J]. 电力系统保护与控制, 2020, 48(22): 91−99.

KRIZHEVSKY A, SUTSKEVER I, HINTON G E. Imagenet classification with deep convolutional neural networks[C]//Advances in Neural Information Processing Systems. 2012: 1097-1105.

GUPTA A, GURRALA G, SASTRY P S. An online power system stability monitoring system using convolutional neural networks[J]. IEEE Trans. on Power Systems, 2019, 34(2): 864−872.

杜文娟, 王海风. 电力系统宽频振荡开环模式谐振原理与分析方法[M]. 北京: 科学出版社, 2019.

ZHOU Boying, TANG Kexuan, SHEN Chen. Research on injection method of gaussian noise in EMTP[J]. System Simulation Technology, 2023, 19(01): 1−8(in Chinese).

周博英, 唐可翾, 沈沉. 电磁暂态仿真程序中高斯噪声的注入方法研究[J]. 系统仿真技术, 2023, 19(01): 1−8.

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

基于孪生延迟DDPG强化学习的电−热耦合系统低碳经济调度

柔性直流引发火电机组次同步振荡的电气负阻尼关键因素分析

基于多主体博弈和强化学习的微电网群与配电网协调优化调度方法

基于多智体强化学习的多风氢系统联合优化运行

双馈风电场串补送出系统次同步振荡及参数调整分析