基于改进深度确定性策略梯度算法的综合能源系统优化调度策略

龚锦霞; 李琛舟; 柯慧

doi:10.19725/j.cnki.1007-2322.2023.0026

欢迎来到能源电力期刊网！

English Version 读者登录期刊登录

您当前的位置：

首页 >

文章列表页 >

基于改进深度确定性策略梯度算法的综合能源系统优化调度策略

能源互联网 | 更新时间：2025-10-13

- 基于改进深度确定性策略梯度算法的综合能源系统优化调度策略
- Optimization Scheduling Strategies for Integrated Energy Systems Based on Improved Deep Deterministic Policy Gradient Algorithm
- 现代电力 2025年42卷第2期页码：322-332
- 作者机构：
  
  1.上海电力大学电气工程学院, 上海市杨浦区 200090
  2.上海电力设计院有限公司, 上海市黄浦区 200025
- 作者简介：
  
  [ "龚锦霞(1984)，女，博士，讲师，硕士生导师，通信作者，研究方向为新能源并网、智能电网调度，E-mail：jxgong2015@163.com" ]
  [ "李琛舟(1998)，男，硕士研究生，研究方向为综合能源的优化与智能算法" ]
  [ "柯慧(1983)，女，硕士，高级工程师，主要研究方向为电网规划" ]
- 基金信息：
- DOI：10.19725/j.cnki.1007-2322.2023.0026
  中图分类号： TM73
- 收稿：2023-02-03，
  
  录用：2023-12-19，
  
  纸质出版：2025-04-10
- 稿件说明：
移动端阅览
龚锦霞, 李琛舟, 柯慧. 基于改进深度确定性策略梯度算法的综合能源系统优化调度策略[J]. 现代电力, 2025,42(2):322-332.

Jinxia GONG, Chenzhou LI, Hui KE. Optimization Scheduling Strategies for Integrated Energy Systems Based on Improved Deep Deterministic Policy Gradient Algorithm[J]. Modern electric power, 2025, 42(2): 322-332.
龚锦霞, 李琛舟, 柯慧. 基于改进深度确定性策略梯度算法的综合能源系统优化调度策略[J]. 现代电力, 2025,42(2):322-332. DOI： 10.19725/j.cnki.1007-2322.2023.0026.

Jinxia GONG, Chenzhou LI, Hui KE. Optimization Scheduling Strategies for Integrated Energy Systems Based on Improved Deep Deterministic Policy Gradient Algorithm[J]. Modern electric power, 2025, 42(2): 322-332. DOI： 10.19725/j.cnki.1007-2322.2023.0026.

摘要

针对综合能源系统优化调度问题中存在的决策空间庞大、算法难以收敛等问题，提出一种基于改进深度确定性策略梯度算法(deep deterministic policy gradient，DDPG)的优化调度策略。通过增设第二个经验池，解决算法难以收敛，甚至寻优失败的问题。针对综合能源系统优化调度问题，优化算法中网络参数更新流程，提高算法训练效率。同时，对奖励函数进行重新设计，采用非线性奖励函数进一步提高算法稳定性。最后，通过对一个包含光伏、储能系统、制冷机组、电加热机组和燃气锅炉组成的综合能源系统进行仿真，并对比算法改进前后的性能。算例表明，基于改进深度确定性策略梯度算法的优化调度策略具有较好的收敛性、稳定性和高效的训练效率，可以实现综合能源系统的灵活高效调度。

Abstract

To address the issues of large decision space and difficulty in convergence in the optimization scheduling of integrated energy systems

in this paper we propose an optimized scheduling strategy based on the improved deep deterministic policy gradient (DDPG) algorithm. The difficulty in convergence and even failure in optimization is solved by adding a second experience pool. In order to address the optimization scheduling challenge of integrated energy systems

the algorithm is optimized by improving the network parameter update process

resulting in an increase in the efficiency of the training process. In addition

the reward function is redesigned and a non-linear reward function is adopted to further improve the stability of the algorithm. Finally

an integrated energy system composed of photovoltaic

energy storage systems

refrigeration units

electric heating units and gas boilers is simulated

and the performance of the algorithm is compared before and after the improvement. The case study indicates that the optimization scheduling strategy based on the improved deep deterministic policy gradient algorithm exhibits excellent convergence

stability and high training efficiency. Moreover

it enables flexible and efficient scheduling of the integrated energy system.

关键词

Keywords

references

程乐峰, 余涛, 张孝顺, 等. 机器学习在能源与电力系统领域的应用和展望[J]. 电力系统自动化, 2019, 43(1): 15−31.

CHENG Lefeng, YU Tao, ZHANG Xiaoshun, et al . Machine learning for energy and electric power systems: state of the art and prospects[J ] . Automation of Electric Power Systems, 2019, 43(1): 15−31(in Chinese).

刘振亚. 全球能源互联网跨国跨洲互联研究及展望[J]. 中国电机工程学报, 2016, 36(19): 5103−5110.

LIU Zhenya. Research of global clean energy resource and power grid interconnection[J]. Proceedings of the CSEE, 2016, 36(19): 5103−5110(in Chinese).

WANG Y, ZHANG N, KANG C, et al . Standardized matrix modeling of multiple energy systems[J ] . IEEE Transactions on Smart Grid, 2017, 10(1): 257−270.

余晓丹, 徐宪东, 陈硕翼, 等. 综合能源系统与能源互联网简述[J]. 电工技术学报, 2016, 31(1): 1−13.

YU Xiaodan, XU Xiandong, CHEN Shouyi, et al . A brief review to integrated energy system and energy internet[J ] . Transactions of China Electrotechnical Society, 2016, 31(1): 1−13(in Chinese).

GEIDL M, KOEPPEL G, FAVRE-PERROD P, et al . Energy hubs for the future[J ] . IEEE Power & Energy Magazine, 2006, 5(1): 24−30.

赵海彭, 苗世洪, 李超, 等. 考虑冷热电需求耦合响应特性的园区综合能源系统优化运行策略研究[J]. 中国电机工程学报, 2022, 42(2): 573−589.

ZHAO Haipeng, MIAO Shihong, LI Chao, et al . Research on optimal operation strategy for park-level integrated energy system considering cold-heat-electric demand coupling response characteristics[J ] . Proceedings of the CSEE, 2022, 42(2): 573−589(in Chinese).

刘涤尘, 马恒瑞, 王波,等. 含冷热电联供及储能的区域综合能源系统运行优化[J]. 电力系统自动化, 2018, 42(4): 113−120.

LIU Diche n, MA Hengrui, WANG Bo, et al . Operation optimization of regional integrated energy system with cchp and energy storage system[J ] . Automation of Electric Power Systems, 2018, 42(4): 113−120(in Chinese).

程杉, 黄天力, 魏荣宗. 含冰蓄冷空调的冷热电联供型微网多时间尺度优化调度[J]. 电力系统自动化, 2019, 43(5): 30−38.

CHENG Shan, HUANG Tianli, WEI Rongzong. Multi time-scale optimal scheduling of CCHP microgrid with ice-storage air-conditioning[J]. Automation of Electric Power Systems, 2019, 43(5): 30−38(in Chinese).

王成山, 吕超贤, 李鹏, 等. 园区型综合能源系统多时间尺度模型预测优化调度[J]. 中国电机工程学报, 2019, 39(23): 6791−6803.

WANG Chengshan, LV Chaoxian, LI Peng, et al . Multiple time-scale optimal scheduling of community integrated energy system based on model predictive control[J ] . Proceedingsof the CSEE, 2019, 39(23): 6791−6803(in Chinese).

CHEN L, ZHU X, CAI J, et al . Multi-time scale coordinated optimal dispatch of microgrid cluster based on MAS[J ] . Electric Power Systems Research, 2019, 177(5): 105976.

彭刘阳, 孙元章, 徐箭, 等. 基于深度强化学习的自适应不确定性经济调度[J]. 电力系统自动化, 2020, 44(9): 33−42.

PENG Liuyang, SUN Yuanzhang, XU Jian, et al . Self-adapt ive uncertainty economic dispatch based on deep reinforcement learning[J ] . Automation of Electric Power Systems, 2020, 44(9): 33−42(in Chinese).

ABEDI S, YOON S W, KWON S. Battery energy storage control using a reinforcement learning approach with cyclic time-dependent Markov process[J]. International Journal of Electrical Power&Energy Systems, 2022, 134: 107368.

YU L, XIE W, XIE D, et al . Deep reinforcement learning for smart home energy management[J ] . IEEE Internet of Things Journal, 2019, 7(4): 2751−2762.

杨挺, 赵黎媛, 刘亚闯, 等. 基于深度强化学习的综合-能源系统动态经济调度[J]. 电力系统自动化, 2021, 45(5): 39−47.

YANG Tang, ZHAO Liyuan, LIU Yachuang, et al . Dynamic economic-dispatch for integrated energy system based on deep reinforcement learning[J ] . Automation of Electric Power Systems, 2021, 45(5): 39−47(in Chinese).

KHALID J, RAMLI M A, KHAN M S, et al . Efficient load frequency control of renewable integrated power system: A twin delayed DDPG-based deep reinforcement learning approach[J ] . IEEE Access, 2022(10): 51561−51574.

蒋明喆, 成贵学, 赵晋斌. 基于改进DDPG的多能园区典型日调度研究[J]. 电网技术, 2022, 46(5): 1867−1880.

JIANG Mingzhe, CHENG Guixue, ZHAO Jinbin. Research on the improvement of DDPG multifunctional industrial park typical daily scheduling[J]. Power System Technology, 2022, 46(5): 1867−1880(in Chinese).

MNIH V, KAVUKCUOGLU K, SILVER D, et al . Human-level control through deep reinforcement learning[J ] . Nature, 2015, 518(7540): 529−533.

ZHANG Z, CHONG A, PAN Y, et al . A deep reinforcement learning approach to using whole building energy model for HVAC optimal control[C ] //2018 ASHRAE/IBPSA-USA Building Performance Analysis Conference and SimBuild. Chicago, IL, USA: 110-121.

WATKINS C J, DAYAN P. Q-learning[J]. Machine Learning, 1992, 8(3): 279−292.

LILLICRAP T P, HUNT J J, PRITZEL A, et al . Continuous control with deep reinforcement learning[C ] //4th International Conference on Learning Representation (ICLR 2016). 2016: 1-14.

董雷, 刘雨, 乔骥, 等. 基于多智能体深度强化学习的电热联合系统优化运行[J]. 电网技术, 2021, 45(12): 4729−4738.

DONG Lei, LIU Yu, QIAO Ji, et al . Optimal dispatch of combined heat and power system based on multi-agent deep reinforcement learning[J ] . Power System Technology, 2021, 45(12): 4729−4738(in Chinese).

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

基于深度强化学习的综合能源系统优化调度

考虑综合需求响应和阶梯型碳交易的综合能源系统协调优化调度

考虑需求响应与碳价不确定性的综合能源系统优化调度

基于改进麻雀搜索算法的电–热–气–氢综合能源系统优化

考虑供能可靠性的综合能源系统双层容量配置优化