Deep reinforcement learning-driven decision-making paradigm for electricity-carbon-hydrogen collaboration

ZHANG Fuchun; CHEN Wenjun; ZENG Tianze; LIU Nian; GUO Hongzhen; LIU Dunnan; WANG Peng; XU Chuanbo

doi:10.11930/j.issn.1004-9649.202602036

English Version Journal Login

您当前的位置：

首页 >

文章列表页 >

Deep reinforcement learning-driven decision-making paradigm for electricity-carbon-hydrogen collaboration

更新时间：2026-07-01

- Deep reinforcement learning-driven decision-making paradigm for electricity-carbon-hydrogen collaboration
- Vol. 59, Issue 6, Pages: 24-36(2026)
- 作者机构：
  
  1. 华北电力大学经济与管理学院,北京,102206
  2. 澳大利亚新南威尔士大学计算机科学与工程学院，澳大利亚悉尼 NSW,2052
  3. 新能源电力系统全国重点实验室（华北电力大学）,北京,102206
  4. 华北电力大学国家能源发展战略研究院,北京,102206
- 作者简介：
- 基金信息：
- DOI：10.11930/j.issn.1004-9649.202602036
  CLC：
- Online First：22 June 2026，
  
  Published：2026
- 稿件说明：
移动端阅览
张富春, 陈文君, 曾天泽, 刘念, 郭红珍, 刘敦楠, 王鹏, 许传博. 深度强化学习驱动的电-碳-氢协同决策范式[J]. 中国电力, 2026, 59(6): 24-36.

ZHANG Fuchun, CHEN Wenjun, ZENG Tianze, et al. Deep reinforcement learning-driven decision-making paradigm for electricity-carbon-hydrogen collaboration[J]. 2026, 59(6): 24-36.
张富春, 陈文君, 曾天泽, 刘念, 郭红珍, 刘敦楠, 王鹏, 许传博. 深度强化学习驱动的电-碳-氢协同决策范式[J]. 中国电力, 2026, 59(6): 24-36. DOI： 10.11930/j.issn.1004-9649.202602036.

ZHANG Fuchun, CHEN Wenjun, ZENG Tianze, et al. Deep reinforcement learning-driven decision-making paradigm for electricity-carbon-hydrogen collaboration[J]. 2026, 59(6): 24-36. DOI： 10.11930/j.issn.1004-9649.202602036.

摘要

为实现低碳能源系统协同优化，电-碳-氢协同成为关键路径之一，但其高维、非线性、强不确定等特征制约传统优化方法应用。深度强化学习具备从数据中自主学习、适应动态环境与多目标决策的能力，为破解该协同优化难题提供了新途径。系统梳理了电-碳-氢协同机理，阐述了深度强化学习应用于该领域的必要性，综述了其在电力市场、碳市场、电-碳协同、电-氢协同及三者集成中的研究进展。结果表明，深度强化学习在提升新能源消纳、优化碳交易与多能流协同等方面潜力显著，但仍面临模型复杂性、策略可解释性与安全性、多目标权衡等挑战。未来应聚焦深度强化学习与大语言模型融合、鲁棒与安全机制、可解释性提升及跨尺度协同等方向，以推动其实际应用。

Abstract

To achieve synergistic optimization of low-carbon energy systems

electricity-carbon-hydrogen synergy has become one of the critical pathways. However

its high dimensionality

nonlinearity

and strong uncertainties limit traditional optimization methods. Deep reinforcement learning (DRL)

with its ability to learn from data

adapt to dynamic environments

and support multi-objective decision-making

offers a promising solution. This paper reviews the mechanisms of electricity-carbon-hydrogen synergy and the necessity of applying DRL

summarizing recent progress in electricity markets

carbon markets

electricity-carbon synergy

electricity-hydrogen synergy

and their integration. The results show that DRL holds significant potential for enhancing renewable energy integration

optimizing carbon trading

and coordinating multi-energy flows

though challenges remain in model complexity

interpretability

safety

and multi-objective trade-offs. Future research should focus on integrating DRL with large language models

improving robustness

safety

and interpretability

and enabling cross-scale coordination to facilitate practical deployment.

关键词

Keywords

references

1邱忠涛, 格根敖其, 贾跃龙, 等. 新型电力系统供需协同全要素理论框架[J]. 中国电力, 2025, 58 (10): 147- 162.

QIU Zhongtao, GEGEN Aoqi, JIA Yuelong, et al. All-factor theoretical framework for supply-demand synergy in new-type power systems[J]. Electric Power, 2025, 58 (10): 147- 162.

2张丝钰, 张宁, 代红才, 等. 可再生能源电解水制氢系统规划优化与生产模拟[J]. 中国电力, 2024, 57 (4): 52- 60.

ZHANG Siyu, ZHANG Ning, DAI Hongcai, et al. Optimization and simulation on hydrogen production system using water electrolysis powered by renewable energy[J]. Electric Power, 2024, 57 (4): 52- 60.

3谭玲玲, 汤伟, 楚冬青, 等. 考虑电-氢一体化的微电网低碳-经济协同优化调度[J]. 中国电力, 2024, 57 (5): 137- 148.

TAN Lingling, TANG Wei, CHU Dongqing, et al. Low-carbon-economic collaborative optimal dispatching of microgrid considering electricity-hydrogen integration[J]. Electric Power, 2024, 57 (5): 137- 148.

4李江南, 程韧俐, 周保荣, 等. 含碳捕集及电转氢设备的低碳园区综合能源系统随机优化调度[J]. 中国电力, 2024, 57 (5): 149- 156.

LI Jiangnan, CHENG Renli, ZHOU Baorong, et al. Stochastic optimal of integrated energy system in low-carbon parks considering carbon capture storage and power to hydrogen[J]. Electric Power, 2024, 57 (5): 149- 156.

5许传博, 刘建国. 氢储能在我国新型电力系统中的应用价值、挑战及展望[J]. 中国工程科学, 2022, 24 (3): 89- 99.

XU Chuanbo, LIU Jianguo. Hydrogen energy storage in China's new-type power system: application value, challenges, and prospects[J]. Strategic Study of CAE, 2022, 24 (3): 89- 99.

6奚洪磊, 孙景钌, 瞿合祚, 等. 基于SAO-NSGA-Ⅱ算法的海岛微电网电氢碳协同优化调度[J]. 浙江电力, 2025, 44 (6): 58- 68.

XI Honglei, SUN Jingliao, QU Hezuo, et al. Coordinated optimal scheduling of island microgrid for power-hydrogen-carbon integration based on SAO-NSGA-II algorithm[J]. Zhejiang Electric Power, 2025, 44 (6): 58- 68.

7杨胡萍, 龚家宁, 程明, 等. 计及多重不确定性的综合能源系统两阶段鲁棒低碳优化调度[J]. 中国电力, 2025, 58 (11): 101- 110, 121.

YANG Huping, GONG Jianing, CHENG Ming, et al. Two-stage robust low-carbon optimal scheduling for integrated energy systems considering for multiple uncertainties[J]. Electric Power, 2025, 58 (11): 101- 110, 121.

8YANG Z, REN Z, LI H, et al. A multi-stage stochastic dispatching method for electricity-hydrogen integrated energy systems driven by model and data[J]. Applied Energy, 2024, 371, 123668.

9HAN C W, WU Z M. Cooperative robust optimization of hydrogen integrated energy systems for economic dispatch under uncertainty with bidirectional power conversion and progressive stepwise carbon emission trading mechanisms[J]. Energy, 2025, 335, 137959.

10YE J, WANG X L, HUA Q S, et al. Deep reinforcement learning based energy management of a hybrid electricity-heat-hydrogen energy system with demand response[J]. Energy, 2024, 305, 131874.

11于万水, 易俊, 杨文丽, 等. 考虑边际碳排放和需求响应的用户侧动态碳责任核算方法[J]. 中国电力, 2025, 58 (12): 86- 95.

YU Wanshui, YI Jun, YANG Wenli, et al. User-side dynamic carbon responsibility accounting method considering marginal carbon emissions and demand response[J]. Electric Power, 2025, 58 (12): 86- 95.

12WEI D, SANG R, SPONHOLZ P, et al. Reversible hydrogenation of carbon dioxide to formic acid using a Mn-pincer complex in the presence of lysine[J]. Nature Energy, 2022, 7 (5): 438- 447.

13许传博, 李沛遥, 张文座, 等. 面向电-碳-氢-醇协同的海上综合能源岛规划[J]. 南方能源建设, 2025, 12 (3): 52- 66.

XU Chuanbo, LI Peiyao, ZHANG Wenzuo, et al. Planning for offshore integrated energy islands with electricity-carbon-hydrogen-methanol synergy[J]. Southern Energy Construction, 2025, 12 (3): 52- 66.

14周原冰, 杨方, 余潇潇, 等. 中国能源电力碳中和实现路径及实施关键问题[J]. 中国电力, 2022, 55 (5): 1- 11.

ZHOU Yuanbing, YANG Fang, YU Xiaoxiao, et al. Realization pathways and key problems of carbon neutrality in China's energy and power system[J]. Electric Power, 2022, 55 (5): 1- 11.

15JIA Q R, ZHANG T, ZHU Z H, et al. Harnessing hydrogen energy storage for renewable energy stability in China: a path to carbon neutrality[J]. International Journal of Hydrogen Energy, 2025, 118, 93- 101.

16MENGESHA I, ROY D. Carbon pricing drives critical transition to green growth[J]. Nature Communications, 2025, 16, 1321.

18王辉, 周珂锐, 吴作辉, 等. 含电转气和碳捕集耦合的综合能源系统多时间尺度优化调度[J]. 中国电力, 2024, 57 (8): 214- 226.

WANG Hui, ZHOU Kerui, WU Zuohui, et al. Multi-time scale optimal scheduling of integrated energy system coupling power-to-gas and carbon capture system[J]. Electric Power, 2024, 57 (8): 214- 226.

19WANG X J, ZHAO M R, SIQIN Z, et al. System scheduling of the zero-carbon hydrogen-based industrial park via cGAN-distributionally robust optimization: a case study in Ordos, China[J]. Energy, 2025, 336, 138343.

20ZHANG Z N, SHEN X J, CHANG M J. Dual stochastic multi-stage robust optimization of integrated energy system considering long medium and short-term uncertainties[J]. Applied Energy, 2025, 400, 126573.

21WANG J, JIN X L, JIA H J, et al. Electricity-hydrogen-heat coupled sharing for multi-energy buildings: a game theoretic approach[J]. Applied Energy, 2025, 398, 126350.

22MAO Y D, LI X W, LIU J Y, et al. Multi-objective optimization of integrated energy systems in natural gas industrial zones: Enhancing performance through variable hydrogen blending ratios[J]. Applied Thermal Engineering, 2025, 268, 125942.

23崔先迤, 曾伟杰, 崔泽政, 等. 考虑氢负荷需求的风光氢热联产型虚拟电厂日前调度优化运行策略[J]. 湖南电力, 2024, 44 (6): 1- 9.

CUI Xianyi, ZENG Weijie, CUI Zezheng, et al. Day-head scheduling optimal operation strategy for wind-solar-hydrogen-heat cogeneration virtual power plants considering hydrogen load requirements[J]. Hunan Electric Power, 2024, 44 (6): 1- 9.

24VAN OTTERLO M, WIERING M. Reinforcement learning and Markov decision processes[M]//Reinforcement Learning. Berlin, Heidelberg: Springer, 2012: 3-42.

25HERNANDEZ-LEAL P, KARTAL B, TAYLOR M E. A survey and critique of multiagent deep reinforcement learning[J]. Autonomous Agents and Multi-Agent Systems, 2019, 33 (6): 750- 797.

26宋雨桐, 陈涛, 高赐威, 等. 基于深度强化学习技术的光伏–固体氧化物燃料电池混合能源系统多场景控制[J]. 中国电机工程学报, 2022, 42 (22): 8129- 8139.

SONG Yutong, CHEN Tao, GAO Ciwei, et al. Deep reinforcement learning-based multiple scenario control strategy for PV-SOFC hybrid system[J]. Proceedings of the CSEE, 2022, 42 (22): 8129- 8139.

27WERON R. Electricity price forecasting: a review of the state-of-the-art with a look into the future[J]. International Journal of Forecasting, 2014, 30 (4): 1030- 1081.

28HU F, ZHAO Y, YU Y W, et al. Deep reinforcement learning-based strategic bidding in electricity markets via variational autoencoder-assisted competitor behavior learning[J]. Engineering Applications of Artificial Intelligence, 2025, 161, 112205.

29ZHU Z Q, BU S Q, CHAN K W, et al. Designing the future electricity spot market with high renewables via reliable simulations[J]. Nature Reviews Electrical Engineering, 2025, 2 (5): 320- 337.

30YE Y J, QIU D W, SUN M Y, et al. Deep reinforcement learning for strategic bidding in electricity markets[J]. IEEE Transactions on Smart Grid, 2020, 11 (2): 1343- 1355.

31WENG H E, HU Y L, LIANG M, et al. Optimizing bidding strategy in electricity market based on graph convolutional neural network and deep reinforcement learning[J]. Applied Energy, 2025, 380, 124978.

32ZHONG S Y, WANG X Y, ZHAO J, et al. Deep reinforcement learning framework for dynamic pricing demand response of regenerative electric heating[J]. Applied Energy, 2021, 288, 116623.

33XIE J H, AJAGEKAR A, YOU F Q. Multi-Agent attention-based deep reinforcement learning for demand response in grid-responsive buildings[J]. Applied Energy, 2023, 342, 121162.

34李祥光, 谭青博, 李帆琪, 等. 电碳耦合对煤电机组现货市场结算电价影响分析模型[J]. 中国电力, 2024, 57 (5): 113- 125.

LI Xiangguang, TAN Qingbo, LI Fanqi, et al. Analysis model to study the influence of electrocarbon coupling on settlement price of coal power units in spot market[J]. Electric Power, 2024, 57 (5): 113- 125.

35CAO Z J, LIU H. A novel carbon price forecasting method based on model matching, adaptive decomposition, and reinforcement learning ensemble strategy[J]. Environmental Science and Pollution Research, 2023, 30 (13): 36044- 36067.

36SHI S Y, YU T. A deep reinforcement learning based carbon trading strategy for generation companies under uncertainties[C]//Proceedings of the 9th PURPLE MOUNTAIN FORUM on Smart Grid Protection and Control (PMF2024). Singapore: Springer, 2025: 279–293.

37WANG M, LIU S Y, LIU J G, et al. Agent-based modeling of firm's heterogeneous preferences: Implications for trading and technology adoption in electricity-carbon markets[J]. Energy Economics, 2025, 148, 108688.

38ZHANG J Y, CHEN K J. Research on carbon asset trading strategy based on PSO-VMD and deep reinforcement learning[J]. Journal of Cleaner Production, 2024, 435, 140322.

39LI L, ZHAO J, DENG K. Edge-based real-time tracking of carbon emission flow in power systems with dynamic network pruning[J]. Applied Energy, 2026, 402, 126970.

40LI J F, HE X T, LI W D, et al. Low-carbon optimal learning scheduling of the power system based on carbon capture system and carbon emission flow theory[J]. Electric Power Systems Research, 2023, 218, 109215.

41LEE S, PRABAWA P, CHOI D H. Joint peak power and carbon emission shaving in active distribution systems using carbon emission flow-based deep reinforcement learning[J]. Applied Energy, 2025, 379, 124944.

42LIANG T, CHAI L L, TAN J X, et al. Dynamic optimization of an integrated energy system with carbon capture and power-to-gas interconnection: a deep reinforcement learning-based scheduling strategy[J]. Applied Energy, 2024, 367, 123390.

43LI Y J, LENG D, CAO Y J, et al. Strategic bidding and trading in joint electricity-carbon market with deep reinforcement learning[C]//2023 IEEE 7th Conference on Energy Internet and Energy System Integration (EI2). Hangzhou, China: IEEE, 2024: 5183–5189.

44ZHOU Y T, MA Z J, WANG T Y, et al. Joint energy and carbon trading for multi-microgrid system based on multi-agent deep reinforcement learning[J]. IEEE Transactions on Power Systems, 2024, 39 (6): 7376- 7388.

45YANG T, DANG Z S, WANG X J, et al. Comprehensive carbon accounting for power systems considering hybrid power trading mode[J]. Energy, 2025, 333, 136897.

46YE T, HUANG Y P, YANG W J, et al. Safe multi-agent deep reinforcement learning for decentralized low-carbon operation in active distribution networks and multi-microgrids[J]. Applied Energy, 2025, 387, 125609.

47杨彪, 谭洪, 朱建川, 等. 面向电-氢本地能源市场的综合能源系统优化运行模型[J]. 电力系统自动化, 2026, 50 (10): 182- 190.

YANG Biao, TAN Hong, ZHU Jianchuan, et al. Optimal operation model of integrated energy systems for electricity-hydrogen local energy markets[J]. Automation of Electric Power Systems, 2026, 50 (10): 182- 190.

48ZHAO Y P, HUANG J S, XU E D, et al. A data-driven scheduling approach for integrated electricity-hydrogen system based on improved DDPG[J]. IET Renewable Power Generation, 2024, 18 (3): 442- 455.

49DREHER A, BEXTEN T, SIEKER T, et al. AI agents envisioning the future: Forecast-based operation of renewable energy storage systems using hydrogen with Deep Reinforcement Learning[J]. Energy Conversion and Management, 2022, 258, 115401.

50LAN P H, CHEN S, LI Q H, et al. Intelligent hydrogen-ammonia combined energy storage system with deep reinforcement learning[J]. Renewable Energy, 2024, 237, 121725.

51WANG J, ZHOU J H, ZHAO W Z. Deep reinforcement learning based energy management strategy for fuel cell/battery/supercapacitor powered electric vehicle[J]. Green Energy and Intelligent Transportation, 2022, 1 (2): 100028.

52WU J J, SONG D F, ZHANG X M, et al. Multi-objective reinforcement learning-based energy management for fuel cell vehicles considering lifecycle costs[J]. International Journal of Hydrogen Energy, 2023, 48 (95): 37385- 37401.

53LU X J, ZHANG Y H, FAN D J, et al. Low-carbon economic dispatch of integrated energy systems for electricity, gas, and heat based on deep reinforcement learning[J]. Sustainability, 2025, 17 (20): 9040.

54LAN P H, CHEN S, WANG F. Carbon and electricity trading for the green hydrogen-based integrated energy system: a deep reinforcement learning-based scheduling optimization[J]. Renewable Energy, 2026, 256, 124176.

55ZHANG L, HE Y, HATZIARGYRIOU N D, et al. An optimization scheduling strategy for hydrogen-based integrated energy systems using multi-agent deep reinforcement learning[J]. Energy Conversion and Management, 2025, 326, 119483.

56蔡木良, 赖信辉, 李赢正, 等. 基于主动配电网灵活性支撑机制的电-碳-绿证分布式协同优化方法[J]. 中国电力, 2026, 59 (1): 10- 19.

CAI Muliang, LAI Xinhui, LI Yingzheng, et al. Power-carbon-green certificate distributed cooperative optimization method based on flexibility support mechanism of active distribution networks[J]. Electric Power, 2026, 59 (1): 10- 19.

57王芊瑞, 阮景昕, 王跃社. 考虑风、光出力时空相关性的电-氢协同储能系统经济性优化调度研究[J]. 综合智慧能源, 2025, 47 (12): 34- 45.

WANG Qianrui, RUAN Jingxin, WANG Yueshe. Economic optimal scheduling of electricity-hydrogen coordinated energy storage system considering spatiotemporal correlation of wind and photovoltaic power outputs[J]. Integrated Intelligent Energy, 2025, 47 (12): 34- 45.

58程川原, 穆士才, 梁晔, 等. 电碳市场经济作用关系研究[J]. 湖南电力, 2025, 45 (6): 76- 83.

CHENG Chuanyuan, MU Shicai, LIANG Ye, et al. Research on the economic relationship between power and carbon market[J]. Hunan Electric Power, 2025, 45 (6): 76- 83.

59夏旭华, 杨建迪, 施永涛. 基于深度强化学习的多能流建筑综合能源系统优化调度[J]. 浙江电力, 2025, 44 (5): 101- 111.

XIA Xuhua, YANG Jiandi, SHI Yongtao. Optimal scheduling of BIES with multi-energy flow coupling based on deep RL[J]. Zhejiang Electric Power, 2025, 44 (5): 101- 111.

60邢作霞, 赵子逸, 孙浩, 等. 基于电制氢多级利用的综合能源系统低碳经济运行优化研究[J]. 综合智慧能源, 2025, 47 (7): 71- 81.

XING Zuoxia, ZHAO Ziyi, SUN Hao, et al. Research on low-carbon economic operation optimization of integrated energy systems based on multi-level utilization of hydrogen production from electricity[J]. Integrated Intelligent Energy, 2025, 47 (7): 71- 81.

61李健, 张诤杰, 杜龙, 等. 考虑“电-碳-绿”联合交易的虚拟电厂博弈优化调度[J]. 浙江电力, 2025, 44 (2): 64- 74.

LI Jian, ZHANG Zhengjie, DU Long, et al. Game optimization scheduling of virtual power plant considering electricity, carbon and green certificate markets[J]. Zhejiang Electric Power, 2025, 44 (2): 64- 74.

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Comprehensive Cost Analysis of Multiple Entities under the Coupling Mechanism of Electricity and Carbon

Electricity Carbon Coupled Market Modeling Method and Market Optimization Mechanism Based on Dynamic Carbon Emission Intensity

Carbon Allocation Throughout the Entire Process of Electric Energy Trading Considering Spot Trading

The design of government-driven renewable energy contracts for difference in the UK electricity market and their implications for China

An optimization method for imbalance funds in a multi-agent game-theoretic electricity market

Related Author

Yanyang WANG

Jianzhao LI

Jingqing LIU

Cheng TANG

Jiantao WANG

Jinling LU

Hui REN

Tong ZHAO

Related Institution

Hebei Electric Power Trading Center Co., Ltd.

Department of Electrical Engineering, North China Electric Power University

State Grid Jiangsu Electric Power Co., Ltd.

School of Electrical and Electronic Engineering, North China Electric Power University

Key Laboratory of Control of Power Transmission and Conversion, Ministry of Education (Shanghai Jiao Tong University)

AI问答

⁰