基于改进双延迟深度确定性策略梯度算法的微电网低碳经济优化调度策略

宋金威; 何悠悠; 孙大帅; 董臣臣; 王景龙

doi:10.19725/j.cnki.1007-2322.2024.0063

欢迎来到能源电力期刊网！

English Version 期刊登录

您当前的位置：

首页 >

文章列表页 >

基于改进双延迟深度确定性策略梯度算法的微电网低碳经济优化调度策略

智能电网 | 更新时间：2026-06-02

- 基于改进双延迟深度确定性策略梯度算法的微电网低碳经济优化调度策略
- A Low-carbon Economic Optimization Dispatching Strategy for Microgrids Based on Improved Twin Delayed Deep Deterministic Policy Gradient Algorithm
- 现代电力 2026年43卷第3期页码：455-463
- 作者机构：
  
  上海采日能源科技有限公司, 嘉定区,上海市,201800
- 作者简介：
  
  宋金威(1997)，女，硕士，研究方向为微电网能量管理与协调控制方法， E-mail：jw.song@sermatec-ess.com
  何悠悠(1992)，女，硕士，研究方向为微电网能量管理与协调控制方法，E-mail：yy.he@sermatec-ess.com
  孙大帅(1981)，男，硕士，通信作者，研究方向为电池故障预警、热失控、EMS和储能控制等， E-mail：sds@sermatec-ess.com
  董臣臣(1992)，男，工程师，研究方向为微电网能量管理与协调控制方法、电池故障预警，E-mail：dcc@sermatec-ess.com
  王景龙(2001)，男，工程师，研究方向为微电网能量管理与协调控制方法、电池故障预警， E-mail：jl.wang@sermatec-ess.com
- 基金信息：
- DOI：10.19725/j.cnki.1007-2322.2024.0063
  中图分类号： 3
- 纸质出版：2026
- 稿件说明：
移动端阅览
宋金威, 何悠悠, 孙大帅, 等. 基于改进双延迟深度确定性策略梯度算法的微电网低碳经济优化调度策略[J]. 现代电力, 2026,43(3):455-463.

SONG Jinwei, HE Youyou, SUN Dashuai, et al. A Low-carbon Economic Optimization Dispatching Strategy for Microgrids Based on Improved Twin Delayed Deep Deterministic Policy Gradient Algorithm[J]. 2026, 43(3): 455-463.
宋金威, 何悠悠, 孙大帅, 等. 基于改进双延迟深度确定性策略梯度算法的微电网低碳经济优化调度策略[J]. 现代电力, 2026,43(3):455-463. DOI： 10.19725/j.cnki.1007-2322.2024.0063.

SONG Jinwei, HE Youyou, SUN Dashuai, et al. A Low-carbon Economic Optimization Dispatching Strategy for Microgrids Based on Improved Twin Delayed Deep Deterministic Policy Gradient Algorithm[J]. 2026, 43(3): 455-463. DOI： 10.19725/j.cnki.1007-2322.2024.0063.

摘要

随着中国“双碳”战略的大力实施，新能源逐渐进入公众视野，微电网的出现有利于新能源的快速消纳，微电网调度与优化问题也逐渐成为研究热点。采用传统方法进行微电网低碳经济调度会出现数据维度高、建模难度大、灵活性低、运行时间长等问题。针对这些问题，提出一种基于改进双延迟深度确定性策略梯度算法(twin delayed deep deterministic policy gradient，TD3)的微电网低碳经济优化调度策略。首先，对模型预测控制(model predictive control，MPC)的策略结果进行统计分析，将得出的储能电池运行规律用于TD3算法自学习规则的改进，有效缩减模型探索区域；其次，引入双缓存区技术，改善样本失衡问题，实现效率和鲁棒性的双重提升；最后，通过对某示范微电网的算例进行分析，验证所提算法模型的有效性。

Abstract

With the vigorous implementation of China’s carbon peaking and carbon neutrality goals

renewable energy has gradually come into the public spotlight. The emergence of microgrids is conducive to the rapid absorption of renewable energy

making microgrid dispatching and optimization a research focal point. Traditional methods for low-carbon economic dispatching of microgrids encounter various challenges such as high data dimensionality

complex modeling

limited flexibility

and extended operation times. To address these challenges

a low-carbon economic optimization dispatching strategy for microgrids is proposed based on an improved twin delayed deep deterministic (TD3) policy gradient algorithm. Firstly

a statistical analysis on the strategy results of model predictive control (MPC) is conducted. The obtained operation rules of energy storage batteries are utilized to refine the self-learning rules of the TD3 algorithm

thereby effectively reducing the model’s exploration area. Secondly

the dual buffer technique is introduced to address the issue of sample imbalance

achieving a dual enhancement in efficiency and robustness. Finally

the effectiveness of the proposed algorithm model is verified through the analysis of a case study of a demonstration microgrid.

关键词

Keywords

references

XIAO Hao, PEI Wei, KONG Li. Multi-time scale coordinated optimal dispatch of microgrid based on model predictive control[J]. Automation of Electric Power Systems, 2016, 40(18): 7−14(in Chinese).

CHEN Jingchuan, CAI Zexiang, MA Guolong. Power capacity optimization of islanded microgrid considering uncertainty coupling and load demand management[J]. Southern Power System Technology, 2020, 14(7): 46−55(in Chinese).

WANG Changhao, LIU Yang, XU Lixiong. Day-ahead economic dispatch for a combined cooling, heat and power microgrid system considering wind power and load uncertainty[J]. Electric Power, 2020, 53(8): 50−59(in Chinese).

ASENSIO M, MENESES P, MUNOZ-DELGADO G, et al . Joint distribution network and renewable energy expansion planning considering demand response and energy storage—part i: stochastic programming model[J ] . IEEE Transactions on Smart Grid, 2018, 9(2): 655−666.

ASENSIO M, MENESES P, MUNOZ-DELGADO G, et al . Joint distribution network and renewable energy expansion planning considering demand response and energy storage—part ii: numerical results[J ] . IEEE Transactions on Smart Grid, 2018, 9(2): 667−675.

JIANG Hongling. Economic optimization model of island micro-grid considering energy storage charge and discharge[J]. Research and Exploration in Laboratory, 2022(1): 138−143(in Chinese).

ALHUMAID Y, KHAN K, ALISMAIL F, et al . Multi-input nonlinear programming based deterministic optimization framework for evaluating microgrids with optimal renewable-storage energy mix[J ] . Sustainability, 2021, 13(11): 5878.

YU Wenying, WANG Wenli, MA Xinxiu, et al . Optimization design of hybrid wind-pv system based on improved genetic algorithm[J ] . Research and Exploration in Laboratory, 2018, 37(1): 125−129,185(in Chinese).

ZHANG Jing, QIN Dain, YE Yongchun, et al . Multi-time scale economic sche duling method based on day-ahead robust optimization and intraday mpc rolling optimization for microgrid[J ] . IEEE Access, 2021, 9: 140315−140324.

SUN Peng, YUN Teng, CHEN Zhe. Multi-objective robust optimization of multi-energy microgrid with waste treatment[J]. IRenewable Energy, 2021, 178: 1198−1210.

ZHU Qunbo, HAO Sipeng. Collaborative optimal dispatching of integrated energy systems in industrial parks[J]. Electrical Measurement Instrumentation, 2024, 61(2): 34−40(in Chinese).

LI Haitao, SHEN Baochen, YANG Yanhong, et al . Energy management and optimization strategy for microgrid based on improved dueling deep Q network algorithm[J ] . Automation of Electric Power Systems, 2022, 46(7): 42−49(in Chinese).

MNIH V, KAVUKCUOGLU K, SILVER D, et al . Human-level control through deep reinforcement learning[J ] . Nature, 2015, 518(7540): 529−533.

SUN Qiuye, WANG Danlu, MA Dazhong, et al . Multi-objective energy management for we-energy in Energy Internet using reinforcement learning[C ] //2017 IEEE Symposium Series on Computational Intelligence (SSCI). Honolulu, HI, USA: IEEE, 2017: 1-6.

LIU Junfeng, CHEN Jianlong, WANG Xiaosheng, et al . Energy management and optimization of multi-energy grid based on deep reinforcement learning[J ] . Power System Technology, 2020, 44(10): 3798−3803(in Chinese).

SHU Yuankai, BI Wenzheng, DONG Wei, et al . Dueling double q-learning based real-time energy dispatch in grid-connected microgrids[C ] //2020 19th International Symposium on Distributed Computing and Applications for Business Engineering and Science (DCABES). Xuzhou, China: CAST, 2020: 42-45.

SUN Zeyue, ESKANDARI M, ZHENG Chaoran, et al . Handling computation hardness and time complexity issue of battery energy storage scheduling in microgrids by deep reinforcement learning[J ] . Energies, 2023, 16(1): 90.

CHEN Siwei, LI Jianjun, ZOU Xinxun, et al . Low-carbon economic dispatch of electric-thermal coupling system based on twin delayed DDPG reinforcement learning[J ] . Modern Electric Power, 2025, 42(2): 314−321(in Chinese).

GONG Jinxia, LI Chenzhou, KE Hui. Optimization scheduling strategies for integrated energy systems based on improved deep deterministic policy gradient algorithm[J]. Modern Electric Power, 2025, 42(2): 322−332(in Chinese).

CHEN Shuai. Energy management and scheduling of microgrids based on reinforcement learning[D]. Beijing: Beijing University of Science and Technology, 2023(in Chinese).

SUTTON R S, BARTO A G. Reinforcement learning: an introduction[M]. Cambridge: MIT Press, 2018.

HENRI G, LEVENT T, HALEV A, et al . pymgrid: an open-source python microgrid simulator for applied artificial intelligence research[J ] . arXiv preprint arXiv: 2011.08004: 2020.

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

基于深度强化学习的微电网低碳优化运行方法

多虚拟同步发电机独立微网的多目标分布式二次控制方法

基于改进深度确定性策略梯度算法的综合能源系统优化调度策略

基于云储能容量动态分配的微电网系统双层优化模型

混合直流供电微电网系统的直流自适应调频方法