Virtual power plant market trading strategy based on hybrid game reinforcement learning

ZHENG Feng; SUN Dian; HUANG Lili; YANG Feng; NI Yun

doi:10.11930/j.issn.1004-9649.202506047

您当前的位置：

首页 >

文章列表页 >

Virtual power plant market trading strategy based on hybrid game reinforcement learning

更新时间：2026-03-17

- Virtual power plant market trading strategy based on hybrid game reinforcement learning
- Vol. 59, Issue 3, Pages: 94-102(2026)
- 作者机构：
  
  1. 国家能源集团长源电力股份有限公司,湖北,武汉,430000
  2. 国能长源能源销售有限公司,湖北,武汉,430000
- 作者简介：
- 基金信息：
- DOI：10.11930/j.issn.1004-9649.202506047
  CLC：
- Online First：16 March 2026，
  
  Published：16 March 2026
- 稿件说明：
移动端阅览
郑峰, 孙电, 黄丽丽, 杨峰, 倪芸. 基于混合博弈强化学习的虚拟电厂市场交易策略[J]. 中国电力, 2026, 59(3): 94-102.

ZHENG Feng, SUN Dian, HUANG Lili, et al. Virtual power plant market trading strategy based on hybrid game reinforcement learning[J]. 2026, 59(3): 94-102.
郑峰, 孙电, 黄丽丽, 杨峰, 倪芸. 基于混合博弈强化学习的虚拟电厂市场交易策略[J]. 中国电力, 2026, 59(3): 94-102. DOI： 10.11930/j.issn.1004-9649.202506047.

ZHENG Feng, SUN Dian, HUANG Lili, et al. Virtual power plant market trading strategy based on hybrid game reinforcement learning[J]. 2026, 59(3): 94-102. DOI： 10.11930/j.issn.1004-9649.202506047.

摘要

随着地区分布式能源快速发展，其单机装机容量小和出力随机性强的问题愈发凸显，导致分布式能源在单独参与市场交易时竞争力不足。为提升其市场参与能力，整合分布式能源形成虚拟电厂（virtual power plant，VPP）已成为一种有效途径。因此，针对含分布式能源的VPP市场交易策略进行研究，提出一种基于混合博弈强化学习的交易策略。首先，根据虚拟电厂内部单元的运行特性构建能源供应商和负荷聚合商的收益模型；然后，为了保证虚拟电厂内部运营商的整体收益建立社会福利最大化模型；最后，基于Stackelberg博弈和演化博弈的混合博弈强化学习算法求解该交易模型。算例分析表明，基于混合博弈强化学习算法的双层模型求解效果优于其他传统智能算法，求解时间减小近50%；此外，VPP同时参与能量市场和辅助服务市场时，可获得更高的收益。

Abstract

With the rapid development of regional distributed energy

the issues of small installed capacity and strong output variability have become increasingly prominent

resulting in insufficient competitiveness when distributed energy participates in market transactions independently. To enhance its market participation capabilities

integrating distributed energy resources into virtual power plant has emerged as an effective approach. Therefore

this study investigates market trading strategies for virtual power plant incorporating distributed energy resources and proposes a trading strategy based on hybrid game-based reinforcement learning. First

establish revenue models for energy suppliers and load aggregators based on the operational characteristics of internal units within the virtual power plant. Then

to ensure the overall profitability of operators within the virtual power plant

a social welfare maximization model is established. Finally

the transaction model is solved using a hybrid game-based reinforcement learning algorithm combining Stackelberg and evolutionary game theory. Case studies demonstrate that the two-layer model based on hybrid game-theoretic reinforcement learning algorithms outperforms traditional intelligent algorithms

reducing computation time by nearly 50%. Furthermore

when virtual power plants participate in both energy markets and ancillary service markets

they can achieve higher returns.

关键词

Keywords

references

1杨胡萍, 龚家宁, 程明, 等. 计及多重不确定性的综合能源系统两阶段鲁棒低碳优化调度[J]. 中国电力, 2025, 58 (11): 101- 110, 121.

YANG Huping, GONG Jianing, CHENG Ming, et al. Two-stage robust low-carbon optimal scheduling for integrated energy systems considering for multiple uncertainties[J]. Electric Power, 2025, 58 (11): 101- 110, 121.

2赵峰, 孙超, 寇凌岳, 等. 分布式储能聚合商参与调频市场竞价策略与协调调度研究[J]. 浙江电力, 2025, 44 (3): 30- 41.

ZHAO Feng, SUN Chao, KOU Lingyue, et al. Research on a bidding strategy and coordinated scheduling of distributed energy storage aggregators in frequency regulation market[J]. Zhejiang Electric Power, 2025, 44 (3): 30- 41.

3刘洋, 陆秋瑜, 徐展鹏, 等. 适用于大规模海上风电集中送电的能源基地运行调度策略[J]. 广东电力, 2025, 38 (5): 16- 29.

LIU Yang, LU Qiuyu, XU Zhanpeng, et al. Energy base operation and scheduling strategy suitable for large-scale centralized transmission of offshore wind power[J]. Guangdong Electric Power, 2025, 38 (5): 16- 29.

4周洋, 黄德志, 李培栋, 等. 考虑平衡端点相位不对称及光伏接入的低压配电网三相潮流模型[J]. 中国电力, 2024, 57 (10): 190- 198.

ZHOU Yang, HUANG Dezhi, LI Peidong, et al. A three-phase power flow model for low-voltage distribution networks considering balanced bus phase asymmetry and photovoltaic access[J]. Electric Power, 2024, 57 (10): 190- 198.

5段玉, 陈军, 刘震, 等. 面向新型电力系统动态分析的多时间尺度构网型储能系统仿真建模方法[J]. 广东电力, 2024, 37 (12): 27- 38.

DUAN Yu, CHEN Jun, LIU Zhen, et al. Modelling of multi-time scale grid-forming energy storage systems for dynamic analysis of new power systems[J]. Guangdong Electric Power, 2024, 37 (12): 27- 38.

6黄灿兵, 熊妮, 吴伟, 等. 虚拟电厂参与一次调频的优化调度策略研究[J]. 浙江电力, 2025, 44 (2): 42- 52.

HUANG Canbing, XIONG Ni, WU Wei, et al. Research on an optimal scheduling strategy for a virtual power plant participating in primary frequency regulation[J]. Zhejiang Electric Power, 2025, 44 (2): 42- 52.

7王宇绅, 陈皓勇, 黄宇翔, 等. 多重不确定性下的虚拟电厂参与电能量和需求响应市场鲁棒优化调度策略[J]. 发电技术, 2024, 45 (6): 1173- 1185.

WANG Yushen, CHEN Haoyong, HUANG Yuxiang, et al. Robust optimal scheduling strategy for virtual power plant participation in electric energy and demand response markets under multiple uncertainties[J]. Power Generation Technology, 2024, 45 (6): 1173- 1185.

8江海燕, 李晨阳, 孟诗语, 等. 考虑配电网分区的虚拟电厂分布式协同优化运行模型[J]. 浙江电力, 2025, 44 (2): 53- 63.

JIANG Haiyan, LI Chenyang, MENG Shiyu, et al. A distributed cooperative optimal operating model for virtual power plants considering distribution network partitions[J]. Zhejiang Electric Power, 2025, 44 (2): 53- 63.

9钟永洁, 汤成俊, 王紫东, 等. 我国虚拟电厂的发展演进和关键技术及难点分析[J]. 浙江电力, 2025, 44 (2): 13- 31.

ZHONG Yongjie, TANG Chengjun, WANG Zidong, et al. Analysis of the evolution, key technologies, and challenges of virtual power plants in China[J]. Zhejiang Electric Power, 2025, 44 (2): 13- 31.

10杨彪, 李佳蓉, 魏子轲. 美国分布式能源发展对我国的启示[J]. 中国电力企业管理, 2024 (25): 94- 96.

11张燕, 乔松博, 徐奇锋, 等. 基于纳什议价理论的分布式绿色电力交易优化分析[J]. 中国电力, 2022, 55 (12): 168- 178.

ZHANG Yan, QIAO Songbo, XU Qifeng, et al. Analysis of distributed green power transaction optimization based on Nash bargaining theory[J]. Electric Power, 2022, 55 (12): 168- 178.

12WU S, LIN J, LI J, et al. Multi-timescale trading strategy for renewable power to ammonia virtual power plant in the electricity, hydrogen, and ammonia markets[J]. Policy and Regulation IEEE Transactions on Energy Markets, 2023, 1 (4): 322- 335.

13GONG X, LI X M, ZHONG Z M. Strategic bidding of virtual power plants in integrated electricity-carbon-green certificate market with renewable energy uncertainties[J]. Sustainable Cities and Society, 2025, 121, 106176.

14许星原, 陈皓勇, 黄宇翔, 等. 虚拟电厂市场化交易中的挑战、策略与关键技术[J]. 发电技术, 2023, 44 (6): 745- 757.

XU Xingyuan, CHEN Haoyong, HUANG Yuxiang, et al. Challenges, strategies and key technologies for virtual power plants in market trading[J]. Power Generation Technology, 2023, 44 (6): 745- 757.

15曾垂辉, 李文正, 李宝伟, 等. 含多分布式能源聚合商的本地电力市场最优能源交易策略[J]. 中国电力, 2023, 56 (11): 236- 245.

ZENG Chuihui, LI Wenzheng, LI Baowei, et al. Optimal energy trading strategies for local electricity markets with multiple distributed energy resource aggregators[J]. Electric Power, 2023, 56 (11): 236- 245.

16杨谭洪, 杨彪, 张延旭, 等. 虚拟电厂设计与运营[M]. 北京: 中国电力出版社, 2025.

17何斯强, 张俊岭, 顾宗奇, 等. 面向分布式资源聚合管控的虚拟电厂建模与优化控制综述[J]. 山东电力技术, 2024, 51 (2): 11- 24, 45.

HE Siqiang, ZHANG Junling, GU Zongqi, et al. Overview of virtual power plant modeling and optimization control for distributed resource aggregation and control[J]. Shandong Electric Power, 2024, 51 (2): 11- 24, 45.

18崔一阳, 黎灿兵, 潘斗南, 等. 高比例新能源电力系统中机组组合研究综述[J]. 湖南电力, 2025, 45 (1): 37- 45.

CUI Yiyang, LI Canbing, PAN Dounan, et al. A review of unit commitment in high proportion renewable energy power systems[J]. Hunan Electric Power, 2025, 45 (1): 37- 45.

19耿健, 金玉龙, 杨宇峰, 等. 考虑调峰辅助服务的虚拟电厂日前运行优化研究[J]. 山东电力技术, 2024, 51 (12): 44- 52.

GENG Jian, JIN Yulong, YANG Yufeng, et al. Virtual power plant day-ahead operation optimization considering peak shaving auxiliary service[J]. Shandong Electric Power, 2024, 51 (12): 44- 52.

20杨珂, 王栋, 李达, 等. 虚拟电厂网络安全风险评估指标体系构建及量化计算[J]. 中国电力, 2024, 57 (8): 130- 137.

YANG Ke, WANG Dong, LI Da, et al. Network security risk assessment index system and calculation for virtual power plant[J]. Electric Power, 2024, 57 (8): 130- 137.

21何知纯, 谢敏, 黄莹, 等. 基于连续隐马尔可夫模型的风水火联合低碳检修优化[J]. 南方能源建设, 2023, 10 (4): 43- 56.

HE Zhichun, XIE Min, HUANG Ying, et al. Wind power, hydropower and thermal power combined low-carbon maintenance optimization based on continuous hidden Markov model[J]. Southern Energy Construction, 2023, 10 (4): 43- 56.

22吴垠, 牛文娟, 诸晓骏. 考虑不同场景的虚拟电厂建模与优化分析[J]. 电测与仪表, 2024, 61 (11): 40- 45.

WU Yin, NIU Wenjuan, ZHU Xiaojun. Modeling and optimization analysis of virtual power plant considering different scenarios[J]. Electrical Measurement & Instrumentation, 2024, 61 (11): 40- 45.

23李继传, 罗朝春, 王宇, 等. 适应大峰谷差的虚拟电厂市场化交易机制研究[J]. 湖南电力, 2025, 45 (2): 129- 136.

LI Jichuan, LUO Chaochun, WANG Yu, et al. Study on market trading mechanism of virtual power plant adapted to large peak-valley difference[J]. Hunan Electric Power, 2025, 45 (2): 129- 136.

24刘继春, 刘可欣, 柯贤波, 等. 新型电力系统下储能参与电能量-惯量-一次调频多市场交易方法[J]. 电网技术, 2025, 49 (3): 1018- 1031.

LIU Jichun, LIU Kexin, KE Xianbo, et al. Participation of energy storage in electric energy-inertia-primary frequency regulation multi-market trading method under a new power system[J]. Power System Technology, 2025, 49 (3): 1018- 1031.

25丁泉, 窦晓波, 温鹏, 等. 电力市场环境下配电网侧多类型主体纳什议价交易模型[J]. 电力自动化设备, 2024, 44 (12): 170- 177.

DING Quan, DOU Xiaobo, WEN Peng, et al. Nash bargaining based transaction model for multi-type entities on distribution network side under electricity market environment[J]. Electric Power Automation Equipment, 2024, 44 (12): 170- 177.

26郭祚刚, 喻磊, 胡洋, 等. 基于合作博弈的综合能源服务商现货市场风险规避策略[J]. 中国电力, 2019, 52 (11): 28- 34.

GUO Zuogang, YU Lei, HU Yang, et al. Research on competition strategy of integrated energy service provider under the pool-based market mechanism[J]. Electric Power, 2019, 52 (11): 28- 34.

27包涛, 李昊飞, 余涛, 等. 考虑市场因素的电力系统供需互动混合博弈强化学习算法[J]. 控制理论与应用, 2020, 37 (4): 907- 917.

BAO Tao, LI Haofei, YU Tao, et al. Mixed game reinforcement learning of supply-demand interaction in power system dis-patch on electricity market[J]. Control Theory & Applications, 2020, 37 (4): 907- 917.

28顼佳宇, 王晓冰, 李冰, 等. 考虑灵活性资源配置的配电网光伏承载力评估方法[J]. 电测与仪表, 2025, 62 (2): 143- 153.

XU Jiayu, WANG Xiaobing, LI Bing, et al. Distributed PV hosting capacity assessment method for distribution network considering planning of flexible resource[J]. Electrical Measurement & Instrumentation, 2025, 62 (2): 143- 153.

29易文飞, 朱卫平, 郑明忠. 计及数据中心和风电不确定性的微电网经济调度[J]. 中国电力, 2024, 57 (2): 19- 26.

YI Wenfei, ZHU Weiping, ZHENG Mingzhong. Economic dispatch of microgrid considering data center and wind power uncertainty[J]. Electric Power, 2024, 57 (2): 19- 26.

30张冬冬, 单琳珂, 刘天皓. 人工智能技术在风力与光伏发电数据挖掘及功率预测中的应用综述[J]. 综合智慧能源, 2025, 47 (3): 32- 46.

ZHANG Dongdong, SHAN Linke, LIU Tianhao. Review on the application of artificial intelligence in data mining and wind and photovoltaic power forecasting[J]. Integrated Intelligent Energy, 2025, 47 (3): 32- 46.

31王辉, 李乃慧, 梁凌, 等. 耦合系统下含光热电站的需求响应虚拟电厂优化调度[J]. 电测与仪表, 2025, 62 (12): 30- 40.

WANG Hui, LI Naihui, LIANG Ling, et al. Optimal scheduling of virtual power plant with concentrating solar power based on demand response in coupled system[J]. Electrical Measurement & Instrumentation, 2025, 62 (12): 30- 40.

32张玉敏, 尹延宾, 吉兴全, 等. 计及热网不同运行状态下灵活性供给能力的综合能源系统优化调度[J]. 中国电力, 2025, 58 (2): 88- 102.

ZHANG Yumin, YIN Yanbin, JI Xingquan, et al. Optimal dispatch of integrated electric-heat energy system considering supply flexibility of heat networks under different operation states[J]. Electric Power, 2025, 58 (2): 88- 102.

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

The Practice and Perspective of Promoting Renewable Energy Integration in China Southern Power Grid

Related Author

Feixiong HU

Baorong ZHOU

Siyu LU

Related Institution

China Southern Power Grid

Electric Power Research Institute, China Southern Power Grid

AI问答

⁰