Wide Load Flexible Control Strategy of Supercritical Combined Heat and Power Unit Based on Deep Reinforcement Learning and Data-driven Approach

ZHU Keyan; ZHANG Guangming; WANG Qinghua; WANG Wei; NIU Yuguang; LIU Jizhen

doi:10.19805/j.cnki.jcspe.2025.240114

English Version Reader LoginJournal Login

您当前的位置：

首页 >

文章列表页 >

Wide Load Flexible Control Strategy of Supercritical Combined Heat and Power Unit Based on Deep Reinforcement Learning and Data-driven Approach

更新时间：2025-10-16

- Wide Load Flexible Control Strategy of Supercritical Combined Heat and Power Unit Based on Deep Reinforcement Learning and Data-driven Approach
- Chinese Journal of Power Engineering Vol. 45, Issue 5, Pages: 785-795(2025)
- 作者机构：
  
  1. 华北电力大学控制与计算机工程学院,北京,102206
  2. 北京怀柔实验室,北京,101400
- 作者简介：
- 基金信息：
- DOI：10.19805/j.cnki.jcspe.2025.240114
  CLC： TP18;TM621
- Published Online：19 May 2025，
  
  Published：2025
- 稿件说明：
移动端阅览
朱珂言,张光明,王庆华,王玮,牛玉广,刘吉臻. 基于深度强化学习与数据驱动的超临界热电联产机组宽负荷灵活控制策略动力工程学报, 2025, 45(5): 785-795 https://doi.

org/10.19805/j.cnki.jcspe.2025.240114
朱珂言,张光明,王庆华,王玮,牛玉广,刘吉臻. 基于深度强化学习与数据驱动的超临界热电联产机组宽负荷灵活控制策略动力工程学报, 2025, 45(5): 785-795 https://doi. DOI： 10.19805/j.cnki.jcspe.2025.240114.

org/10.19805/j.cnki.jcspe.2025.240114 DOI：

摘要

为满足传统火电机组灵活调节性能的需求

提出了一种数据驱动的超临界热电联产(S-CHP)非线性环境建模方法和基于双延迟深度确定性策略梯度(TD3)的强化学习控制方法。首先

基于多层感知机(MLP)的深度学习算法与S-CHP机组动态特性

建立了数据驱动的非线性模型环境;进而基于深度强化学习算法

设计了契合S-CHP机组动态特性的演员-评委策略价值网络和S-CHP特定状态值与奖励函数

提出了一种TD3灵活控制策略以实现快速响应、保证供热及稳定运行的控制目标。结果表明:MLP模型环境与单层网络相比

在额定功率52%~93%负荷宽度下

均方根误差降低了51.7%;与传统协调控制方法相比

TD3控制策略具有更好的跟踪效果和更高的响应速率。

Abstract

To meet the demand for flexible regulation performance of traditional thermal power units

a data-driven nonlinear environmental modeling method for supercritical combined heat and power (S-CHP) and a reinforcement learning control method based on double-delay deep deterministic strategy gradient (TD3) were proposed. Firstly

a data-driven nonlinear model environment was established based on the deep learning algorithm of multi-layer perceptron (MLP) and the dynamic characteristics of the S-CHP unit. Furthermore

based on deep reinforcement learning algorithms

an actor critic strategy value network and S-CHP specific state values and reward functions were designed to conform the dynamic characteristics of the S-CHP unit. A flexible control strategy of TD3 was proposed to achieve the control objectives of rapid response

ensuring heating supply and stable operation. Results show that compared with single-layer networks

the MLP model reduces the root mean square error by 51.7% at the rated power of 52%-93%. Compared with the traditional coordinated control method

the TD3 control strategy has better tracking effect and higher response rate.

关键词

Keywords

references

许朋江, 徐睿, 邓佳, 等. 330 MW机组采暖抽汽对发电热经济性的影响分析[J]. 中国电机工程学报, 2020, 40(19): 6257-6264. XU Pengjiang, XU Rui, DENG Jia, et al. Analysis of the influence of heating steam extraction of 330 MW unit on the economy of electric generation[J]. Proceedings of the CSEE, 2020, 40(19): 6257-6264.

王玮, 申朋玉, 李雪寒, 等. 提升直流炉供热机组灵活性的电热协调控制策略[J]. 中国电机工程学报, 2023, 43(6): 2100-2109. WANG Wei, SHEN Pengyu, LI Xuehan, et al. Electricity-heat coordinated control strategy for improving the flexibility of a once-through CHP unit[J]. Proceedings of the CSEE, 2023, 43(6): 2100-2109.

李玲, 刘鑫屏. 新能源大规模并网条件下火电机组深度调峰控制策略优化[J]. 中国电力, 2020, 53(1): 155-161. LI Ling, LIU Xinping. Control strategy optimization for thermal power unit adapted to deep peak shaving for large-scale new energy source integration[J]. Electric Power, 2020, 53(1): 155-161.

刘鑫屏, 田亮, 王琪, 等. 供热机组发电负荷-机前压力-抽汽压力简化非线性动态模型[J]. 动力工程学报, 2014, 34(2): 115-121. LIU Xinping, TIAN Liang, WANG Qi, et al. Simplified nonlinear dynamic model of generating load-throttle pressure-extraction pressure for heating units[J]. Journal of Chinese Society of Power Engineering, 2014, 34(2): 115-121.

周超, 谢七月, 左毅, 等. 基于多模型切换的锅炉主蒸汽温度预测控制[J]. 电力科学与技术学报, 2020, 35(4): 154-160. ZHOU Chao, XIE Qiyue, ZUO Yi, et al. Multi-model switching based predictive control for main steam temperature in boiler[J]. Journal of Electric Power Science and Technology, 2020, 35(4): 154-160.

牛玉广, 何青波, 李永生, 等. 超临界火电机组多变量前馈预见预测控制[J]. 动力工程学报, 2020, 40(10): 801-807. NIU Yuguang, HE Qingbo, LI Yongsheng, et al. Multi-variable feedforward foreseeable predictive control for supercritical thermal power units[J]. Journal of Chinese Society of Power Engineering, 2020, 40(10): 801-807.

王玮, 孙阳, 刘吉臻, 等. 适应电网快速调频的热电联产机组新型变负荷控制策略[J]. 电力系统自动化, 2018, 42(21): 63-69. WANG Wei, SUN Yang, LIU Jizhen, et al. Load-change control strategy for combined heat and power units adapted to rapid frequency regulation of power grid[J]. Automation of Electric Power Systems, 2018, 42(21): 63-69.

WANG Wei, JING Sitong, SUN Yang, et al. Combined heat and power control considering thermal inertia of district heating network for flexible electric power regulation[J]. Energy, 2019, 169: 988-999.

严家政, 专祥涛. 基于强化学习的参数自整定及优化算法[J]. 智能系统学报, 2022, 17(2): 341-347. YAN Jiazheng, ZHUAN Xiangtao. Parameter self-tuning and optimization algorithm based on reinforcement learning[J]. CAAI Transactions on Intelligent Systems, 2022, 17(2): 341-347.

刘勇, 李祥, 蒋沛阳, 等. 基于DDPG-PID的机器人动态目标跟踪与避障控制研究[J]. 南京航空航天大学学报, 2022, 54(1): 41-50. LIU Yong, LI Xiang, JIANG Peiyang, et al. Research on robot dynamic target tracking and obstacle avoidance control based on DDPG-PID[J]. Journal of Nanjing University of Aeronautics & Astronautics, 2022, 54(1): 41-50.

ZHANG Guangming, ZHANG Chao, WANG Wei, et al. Offline reinforcement learning control for electricity and heat coordination in a supercritical CHP unit[J]. Energy, 2023, 266: 126485.

叶婧, 蔡霞, 张磊, 等. 基于深度强化学习的热电联产机组多工况自适应控制[J]. 热力发电, 2023, 52(4): 104-112. YE Jing, CAI Xia, ZHANG Lei, et al. Multi-condition adaptive control of combined heat and power unit based on deep reinforcement learning[J]. Thermal Power Generation, 2023, 52(4): 104-112.

肖白, 周文凯, 姜卓. 基于孤立森林、模态分解和神经网络的空间负荷态势感知[J]. 电力系统自动化, 2022, 46(18): 190-198. XIAO Bai, ZHOU Wenkai, JIANG Zhuo. Spatial load situation awareness based on isolation forest, mode decomposition and neural networks[J]. Automation of Electric Power Systems, 2022, 46(18): 190-198.

董文康, 吴雨芯, 姚琦, 等. 基于深度强化学习的海上风电机组状态维护与备件库存联合优化[J]. 太阳能学报, 2023, 44(12): 190-199. DONG Wenkang, WU Yuxin, YAO Qi, et al. Joint optimization of state maintenance and spare parts inventory of offshore wind turbines based on deep reinforcement learning[J]. Acta Energiae Solaris Sinica, 2023, 44(12): 190-199.

李彦营, 席磊, 郭宜果, 等. 基于权重双Q-时延更新学习算法的自动发电控制[J]. 中国电机工程学报, 2022, 42(15): 5459-5471. LI Yanying, XI Lei, GUO Yiguo, et al. Automatic generation control based on the weighted double Q-delayed update learning algorithm[J]. Proceedings of the CSEE, 2022, 42(15): 5459-5471.

江昌旭, 刘晨曦, 林铮, 等. 基于深度强化学习的电力系统暂态稳定控制策略研究综述[J]. 高电压技术, 2023, 49(12): 5171-5186. JIANG Changxu, LIU Chenxi, LIN Zheng, et al. Review of power system transient stability control strategies based on deep reinforcement learning[J]. High Voltage Engineering, 2023, 49(12): 5171-5186.

LILLICRAP T P, HUNT J J, PRITZEL A, et al. Continuous control with deep reinforcement learning[C]//4th International Conference on Learning Representations, ICLR 2016. [s.l.]: Israel Institute of Technology, 2016.

李建标, 陈建福, 高滢, 等. 基于RG-DDPG的直流微网能量管理策略[J]. 中国电力, 2023, 56(7): 85-94. LI Jianbiao, CHEN Jianfu, GAO Ying, et al. Strategy for DC microgrid energy management based on RG-DDPG[J]. Electric Power, 2023, 56(7): 85-94.

王轩, 陈嘉宝, 舒歌群, 等. 基于深度强化学习的有机朗肯循环内燃机余热回收系统瞬态优化控制[J]. 中国电机工程学报, 2023, 43(11): 4169-4177. WANG Xuan, CHEN Jiabao, SHU Gequn, et al. Transient optimal control of a transcritical organic rankine cycle for waste heat recovery of internal combustion engine based on deep reinforcement learning[J]. Proceedings of the CSEE, 2023, 43(11): 4169-4177.

张佳志. 强化学习中的高估与安全性问题研究[D]. 徐州: 中国矿业大学, 2023.

FUJIMOTO S, HOOF H, MEGER D. Addressing function approximation error in actor-critic methods[C]//Proceedings of the 35th International Conference on Machine Learning. Sweden: PMLR 80, 2018: 1587-1596.

陈皓炜, 贾新春, 孙小明, 等. SCR脱硝系统的强化学习复合串级控制[J]. 动力工程学报, 2022, 42(5): 421-428. CHEN Haowei, JIA Xinchun, SUN Xiaoming, et al. Composite cascade control based on reinforcement learning for a SCR denitration system[J]. Journal of Chinese Society of Power Engineering, 2022, 42(5): 421-428.

ZHANG Tianhao, DONG Zhe, HUANG Xiaojin. Multi-objective optimization of thermal power and outlet steam temperature for a nuclear steam supply system with deep reinforcement learning[J]. Energy, 2024, 286: 129526.

张红昌, 薛小军, 徐钢, 等. 热电联产机组热电解耦改造方案的调峰特性及能耗分析[J]. 动力工程学报, 2023, 43(10): 1382-1390. ZHANG Hongchang, XUE Xiaojun, XU Gang, et al. Peak shaving characteristics and energy consumption analysis of the thermoelectric-decoupling retrofit scheme for cogeneration units[J]. Journal of Chinese Society of Power Engineering, 2023, 43(10): 1382-1390.

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Adaptive Frequency Regulation Control Method for Wind Farms Based on Deep Reinforcement Learning

Related Author

TIAN Sheng

LIU Yongqian

TIAN Xinshou

CHI Yongning

CHEN Haowei

JIA Xinchun

SUN Xiaoming

HOU Pengfei

Related Institution

School of Renewable Energy, North China Electric Power University

China Institute of Energy and Transportation Integrated Development, North China Electric Power University

China Electric Power Research Institute

School of Automation and Software Engineering, Shanxi University

AI问答

⁰