A Multi-layer Coordinated Control Strategy for Power System Generation Power Based on  Multi-experience Replay Pool MATD3 Algorithm

FU Zhiwei; SHEN Xiaoyang; FANG Chaoying; TANG Jinqiang; CHEN Shaokang; CHENG Yunchu; LIANG Ning

doi:10.20204/j.sp.2025.11007

English Version Reader LoginJournal Login

您当前的位置：

首页 >

文章列表页 >

A Multi-layer Coordinated Control Strategy for Power System Generation Power Based on Multi-experience Replay Pool MATD3 Algorithm

更新时间：2025-12-10

- A Multi-layer Coordinated Control Strategy for Power System Generation Power Based on Multi-experience Replay Pool MATD3 Algorithm
- Vol. 53, Issue 11, Pages: 56-64(2025)
- 作者机构：
  
  1. 国网福建省电力有限公司,福建,福州,350003
  2. 国网福建省电力有限公司电力科学研究院,福建,福州,350003
  3. 厦门亿力吉奥信息科技有限公司,福建,厦门,361008
  4. 昆明理工大学电力学院,云南,昆明,650000
- 作者简介：
- 基金信息：
- DOI：10.20204/j.sp.2025.11007
  CLC：
- Published：2025
- 稿件说明：
移动端阅览
傅智为,沈晓阳,方超颖,唐锦强,陈少康,程韵初,梁宁.基于多经验回放池MATD3算法的电力系统发电功率多层协调控制策略[J].智慧电力,2025,53(11):56-64.

FU Zhiwei, SHEN Xiaoyang, FANG Chaoying, et al. A Multi-layer Coordinated Control Strategy for Power System Generation Power Based on Multi-experience Replay Pool MATD3 Algorithm[J]. 2025, 53(11): 56-64.
傅智为,沈晓阳,方超颖,唐锦强,陈少康,程韵初,梁宁.基于多经验回放池MATD3算法的电力系统发电功率多层协调控制策略[J].智慧电力,2025,53(11):56-64. DOI： 10.20204/j.sp.2025.11007.

FU Zhiwei, SHEN Xiaoyang, FANG Chaoying, et al. A Multi-layer Coordinated Control Strategy for Power System Generation Power Based on Multi-experience Replay Pool MATD3 Algorithm[J]. 2025, 53(11): 56-64. DOI： 10.20204/j.sp.2025.11007.

摘要

提出了一种基于多经验回放池-多智能体双延迟深度确定性策略梯度算法的电力系统发电功率多层协调控制策略。首先，基于调频辅助服务市场的多区域互联框架，构建了含多种调节机组的电力系统多层发电控制架构；其次，基于“分布式训练，集中式控制”的思想，设计了多智能体控制器的互联结构；该结构以兼顾碳排放成本的综合奖励函数为优化目标，实现了对各区域机组发电功率的绿色经济性优化分配。仿真结果表明，该模型与策略能有效降低区域控制偏差与频率偏差，并降低调控成本，同时满足了低碳排放的要求。

Abstract

This paper proposes a multi-layer coordinated control strategy for power system generation power based on the multi-experience replay pool multi-agent twin delayed deep deterministic policy gradient （MER-MATD3） algorithm. Firstly， based on a multi-area interconnection framework within the frequency regulation ancillary service market，a multi-layer generation control architecture for the power system incorporating various regulating units is established. Secondly， guided by the principle of “distributed training and centralized execution，” an interconnected structure for multi-agent controllers is designed. This structure utilizes a comprehensive reward function that considers carbon emission costs as the optimization objective， achieving a green and economical optimal allocation of generation output for units across different areas. Simulation results demonstrate that the proposed model and strategy can effectively reduce both area control error and frequency deviation， lower regulation costs， while simultaneously meeting the requirement for low carbon emissions.

关键词

Keywords

references

崔文倩，魏军强，赵云灏，等.双碳目标下含重力储能的配电网多目标运行优化[J].电力建设，2023，44（4）:45-53.

卢毓东，陈益.“双碳”目标下绿色人工智能技术研究综述[J].浙江电力，2023，42（10）:45-56.

邱明石，陈俊儒，刘牧阳，等.考虑纳网系统和微网运营商电价机制的主从博弈优化调度策略[J].电力系统保护与控制，2024，52（19）:15-34.

缪蔡然，朱姚培，王琦，等.考虑气热惯性的综合能源系统参与辅助服务策略[J].电力建设，2023，44（11）:128-137.

赵霞，张荣荣，赵瑞锋，等.CPS标准下AGC机组动态优化调度的改进模型[J].电工技术学报，2016，31（5）:99-106.

XU Y，HUANG C，LI X，et al.A novel automatic generation control for thermal and gas power plants[C]//2018 IEEE Power & Energy Society General Meeting（PESGM）. Portland，USA，2018：1-5.

XU J F，HE J，LIU Y C，et al.Automatic generation control performance evaluation method for thermal power units based on improved linear segmentation method[C]//2022 Power System and Green Energy Conference.Shanghai，China，2022：722-726.

陆秋瑜，杨银国，谢平平，等.适应储能参与的调频辅助服务市场机制设计及调度策略[J].电网技术，2023，47（12）:4971-4989.

XIONG J，CHEN D，LI Z，et al.An assessment method for the contribution of electrochemical energy storage to the fm auxiliary service market[C]//2022 Asian Conference on Frontiers of Power and Energy（ACFPE）.Chengdu，China，2022：573-577.

王楠，李振，周喜超，等.发电厂AGC与储能联合调频特性及仿真[J].热力发电，2021，50（8）:148-156.

WANG Z，ZHONG J，LI J.Design of performance-based frequency regulation market and its implementations in real-time operation[J].International Journal of Electrical Power & Energy Systems，2017，87：187-197.

ZHANG J，WEI P，QIN B.Automatic generation control design of doubly-fed induction generator integrated power system based on distributed model predictive control[C]//2023 5th International Conference on Power and Energy Technology.Tianjin，China，2023：736-741.

MAO J，LIU R，WU A，et al.An improved whale optimization algorithm based PIDF-（1+ PI） cascade automatic generation control for multi-area multi-source power system with capacitive energy storage[J].IEEE Access，2023：72418-72435.

TABASSUM N，JAHAN E，GOSWAMI N，et al.Analysis of genetic algorithm-optimized PID controllers for a two-area automatic generation control system[C]//2023 10th IEEE International Conference on Power Systems（ICPS）.Cox’s Bazar，Bangladesh，2023: 736-741.

赵霞，梁钰，孙名轶，等.AGC机组动态优化调度模型的凸松弛及其双层迭代算法[J].电力系统自动化，2022，46（17）:228-238.

刘晨，郭凯凯，张乃峰，等.定子永磁型双凸极电机多参数多目标优化设计[J].重庆工商大学学报（自然科学版），2024（2）：1-8.

吕帅，龚晓宇，张正昊，等.结合进化算法的深度强化学习方法研究综述[J].计算机学报，2022，45（7）:1478-1499.

ZHANG B，HU W，GHIAS AMYM，et al.Multi-agent deep reinforcement learning based distributed control architecture for interconnected multi-energy microgrid energy management and optimization[J].Energy Conversion and Management，2023，277：116647.

YANG J，SUN Z，HU W，et al.Joint control of manufacturing and onsite microgrid system via novel neural-network integrated reinforcement learning algorithms[J].Applied Energy，2022，315：118982.

马庆，邓长虹.基于单/多智能体简化强化学习的电力系统无功电压控制[J].电工技术学报，2024，39（5）:1300-1312.

FAN P，KE S，YANG J，et al.A load frequency coordinated control strategy for multimicrogrids with V2G based on improved MA-DDPG[J].International Journal of Electrical Power & Energy Systems，2023，146：108765.

范培潇，杨军，温裕鑫，等.基于可进化模型预测控制的含电动汽车多微电网智能发电控制策略[J].电工技术学报，2024，39（3）:699-713.

李敬如，白宇，王旭阳，等.考虑主配协同的分布式光伏接入电网安全经济承载力计算方法[J].电力建设，2025，46（10）：113-121.

刘伟，高嵩，宋宗勋，等.基于深度强化学习的空气源热泵供热系统温度控制策略[J].山东电力技术，2025，52（1）：54-61.

李子凯，杨波，周忠堂，等.基于强化学习算法的微电网优化策略[J].山东电力技术，2024，51（6）：27-35.

母欢欢，余凌，夏凡，等.基于深度强化学习的机组组合智能求解算法[J].高压电器，2025，61（5）：197-207.

马涛，周飞飞，何迎利，等.基于强化学习的风电消纳随机鲁棒混合优化调度系统[J].电子设计工程，2024，32（15）：25-29.

周良才，周毅，沈维健，等.基于深度强化学习的新型电力系统无功电压优化控制[J].电测与仪表，2024，61（9）：182-189.

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

No data

Related Author

No data

Related Institution

No data

AI问答

⁰