A Low-carbon Economic Optimization Dispatching Strategy for Microgrids Based on Improved Twin Delayed Deep Deterministic Policy Gradient Algorithm

SONG Jinwei; HE Youyou; SUN Dashuai; DONG Chenchen; WANG Jinglong

doi:10.19725/j.cnki.1007-2322.2024.0063

English Version Journal Login

您当前的位置：

首页 >

文章列表页 >

A Low-carbon Economic Optimization Dispatching Strategy for Microgrids Based on Improved Twin Delayed Deep Deterministic Policy Gradient Algorithm

更新时间：2026-06-02

- A Low-carbon Economic Optimization Dispatching Strategy for Microgrids Based on Improved Twin Delayed Deep Deterministic Policy Gradient Algorithm
- Vol. 43, Issue 3, Pages: 455-463(2026)
- 作者机构：
  
  上海采日能源科技有限公司, 嘉定区,上海市,201800
- 作者简介：
- 基金信息：
- DOI：10.19725/j.cnki.1007-2322.2024.0063
  CLC： 3
- Published：2026
- 稿件说明：
移动端阅览
SONG Jinwei, HE Youyou, SUN Dashuai, et al. A Low-carbon Economic Optimization Dispatching Strategy for Microgrids Based on Improved Twin Delayed Deep Deterministic Policy Gradient Algorithm[J]. 2026, 43(3): 455-463.
DOI：

SONG Jinwei, HE Youyou, SUN Dashuai, et al. A Low-carbon Economic Optimization Dispatching Strategy for Microgrids Based on Improved Twin Delayed Deep Deterministic Policy Gradient Algorithm[J]. 2026, 43(3): 455-463. DOI： 10.19725/j.cnki.1007-2322.2024.0063.

摘要

随着中国“双碳”战略的大力实施，新能源逐渐进入公众视野，微电网的出现有利于新能源的快速消纳，微电网调度与优化问题也逐渐成为研究热点。采用传统方法进行微电网低碳经济调度会出现数据维度高、建模难度大、灵活性低、运行时间长等问题。针对这些问题，提出一种基于改进双延迟深度确定性策略梯度算法(twin delayed deep deterministic policy gradient，TD3)的微电网低碳经济优化调度策略。首先，对模型预测控制(model predictive control，MPC)的策略结果进行统计分析，将得出的储能电池运行规律用于TD3算法自学习规则的改进，有效缩减模型探索区域；其次，引入双缓存区技术，改善样本失衡问题，实现效率和鲁棒性的双重提升；最后，通过对某示范微电网的算例进行分析，验证所提算法模型的有效性。

Abstract

With the vigorous implementation of China’s carbon peaking and carbon neutrality goals

renewable energy has gradually come into the public spotlight. The emergence of microgrids is conducive to the rapid absorption of renewable energy

making microgrid dispatching and optimization a research focal point. Traditional methods for low-carbon economic dispatching of microgrids encounter various challenges such as high data dimensionality

complex modeling

limited flexibility

and extended operation times. To address these challenges

a low-carbon economic optimization dispatching strategy for microgrids is proposed based on an improved twin delayed deep deterministic (TD3) policy gradient algorithm. Firstly

a statistical analysis on the strategy results of model predictive control (MPC) is conducted. The obtained operation rules of energy storage batteries are utilized to refine the self-learning rules of the TD3 algorithm

thereby effectively reducing the model’s exploration area. Secondly

the dual buffer technique is introduced to address the issue of sample imbalance

achieving a dual enhancement in efficiency and robustness. Finally

the effectiveness of the proposed algorithm model is verified through the analysis of a case study of a demonstration microgrid.

关键词

Keywords

references

XIAO Hao, PEI Wei, KONG Li. Multi-time scale coordinated optimal dispatch of microgrid based on model predictive control[J]. Automation of Electric Power Systems, 2016, 40(18): 7−14(in Chinese).

CHEN Jingchuan, CAI Zexiang, MA Guolong. Power capacity optimization of islanded microgrid considering uncertainty coupling and load demand management[J]. Southern Power System Technology, 2020, 14(7): 46−55(in Chinese).

WANG Changhao, LIU Yang, XU Lixiong. Day-ahead economic dispatch for a combined cooling, heat and power microgrid system considering wind power and load uncertainty[J]. Electric Power, 2020, 53(8): 50−59(in Chinese).

ASENSIO M, MENESES P, MUNOZ-DELGADO G, et al . Joint distribution network and renewable energy expansion planning considering demand response and energy storage—part i: stochastic programming model[J ] . IEEE Transactions on Smart Grid, 2018, 9(2): 655−666.

ASENSIO M, MENESES P, MUNOZ-DELGADO G, et al . Joint distribution network and renewable energy expansion planning considering demand response and energy storage—part ii: numerical results[J ] . IEEE Transactions on Smart Grid, 2018, 9(2): 667−675.

JIANG Hongling. Economic optimization model of island micro-grid considering energy storage charge and discharge[J]. Research and Exploration in Laboratory, 2022(1): 138−143(in Chinese).

ALHUMAID Y, KHAN K, ALISMAIL F, et al . Multi-input nonlinear programming based deterministic optimization framework for evaluating microgrids with optimal renewable-storage energy mix[J ] . Sustainability, 2021, 13(11): 5878.

YU Wenying, WANG Wenli, MA Xinxiu, et al . Optimization design of hybrid wind-pv system based on improved genetic algorithm[J ] . Research and Exploration in Laboratory, 2018, 37(1): 125−129,185(in Chinese).

ZHANG Jing, QIN Dain, YE Yongchun, et al . Multi-time scale economic sche duling method based on day-ahead robust optimization and intraday mpc rolling optimization for microgrid[J ] . IEEE Access, 2021, 9: 140315−140324.

SUN Peng, YUN Teng, CHEN Zhe. Multi-objective robust optimization of multi-energy microgrid with waste treatment[J]. IRenewable Energy, 2021, 178: 1198−1210.

ZHU Qunbo, HAO Sipeng. Collaborative optimal dispatching of integrated energy systems in industrial parks[J]. Electrical Measurement Instrumentation, 2024, 61(2): 34−40(in Chinese).

LI Haitao, SHEN Baochen, YANG Yanhong, et al . Energy management and optimization strategy for microgrid based on improved dueling deep Q network algorithm[J ] . Automation of Electric Power Systems, 2022, 46(7): 42−49(in Chinese).

MNIH V, KAVUKCUOGLU K, SILVER D, et al . Human-level control through deep reinforcement learning[J ] . Nature, 2015, 518(7540): 529−533.

SUN Qiuye, WANG Danlu, MA Dazhong, et al . Multi-objective energy management for we-energy in Energy Internet using reinforcement learning[C ] //2017 IEEE Symposium Series on Computational Intelligence (SSCI). Honolulu, HI, USA: IEEE, 2017: 1-6.

LIU Junfeng, CHEN Jianlong, WANG Xiaosheng, et al . Energy management and optimization of multi-energy grid based on deep reinforcement learning[J ] . Power System Technology, 2020, 44(10): 3798−3803(in Chinese).

SHU Yuankai, BI Wenzheng, DONG Wei, et al . Dueling double q-learning based real-time energy dispatch in grid-connected microgrids[C ] //2020 19th International Symposium on Distributed Computing and Applications for Business Engineering and Science (DCABES). Xuzhou, China: CAST, 2020: 42-45.

SUN Zeyue, ESKANDARI M, ZHENG Chaoran, et al . Handling computation hardness and time complexity issue of battery energy storage scheduling in microgrids by deep reinforcement learning[J ] . Energies, 2023, 16(1): 90.

CHEN Siwei, LI Jianjun, ZOU Xinxun, et al . Low-carbon economic dispatch of electric-thermal coupling system based on twin delayed DDPG reinforcement learning[J ] . Modern Electric Power, 2025, 42(2): 314−321(in Chinese).

GONG Jinxia, LI Chenzhou, KE Hui. Optimization scheduling strategies for integrated energy systems based on improved deep deterministic policy gradient algorithm[J]. Modern Electric Power, 2025, 42(2): 322−332(in Chinese).

CHEN Shuai. Energy management and scheduling of microgrids based on reinforcement learning[D]. Beijing: Beijing University of Science and Technology, 2023(in Chinese).

SUTTON R S, BARTO A G. Reinforcement learning: an introduction[M]. Cambridge: MIT Press, 2018.

HENRI G, LEVENT T, HALEV A, et al . pymgrid: an open-source python microgrid simulator for applied artificial intelligence research[J ] . arXiv preprint arXiv: 2011.08004: 2020.

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

A Low-carbon Optimal Operation Method for Microgrids Based on Deep Reinforcement Learning

Multi-objective Distributed Secondary Control Method for Multi-VSG Independent Microgrid

Optimization Scheduling Strategies for Integrated Energy Systems Based on Improved Deep Deterministic Policy Gradient Algorithm

Double-layer Optimization Model for Microgrid System Based on Dynamic Capacity Allocation of CES

DC Self-Adapt Frequency Regulation Method for Hybrid DC Power Supply Microgrid System

Related Author

LIN Wenshuo

LI Xin

ZHOU Rongsheng

ZHU Yiying

PENG Yiming

Minling HUANG

Jing CHENG

Jianxiong HU

Related Institution

Guangzhou Power Supply Bureau of Guangdong Power Grid

Renewable Energy Power Generation and Grid-connected Control Engineering Research Center of the Ministry of Education, Urumqi 830047, Xinjiang Uygur Autonomous Region

School of Electrical Engineering, Xinjiang University, Urumqi 830047,Xinjiang Uygur Autonomous Region

POWERCHINA Shanghai Electric Power Engineering Co., Ltd., Huangpu District

Department of Electric Power Engineering, Shanghai University of Electric Power, Yangpu District

AI问答

⁰