Two-layer Robust Planning of Urban Integrated Energy Station Based on Multi-agent Safe Reinforcement Learning

LYU Kai; SHAO Zhenguo; CHEN Changming; CHEN Feixiong

doi:10.7500/AEPS20251228001

English Version Journal Login

您当前的位置：

首页 >

文章列表页 >

Two-layer Robust Planning of Urban Integrated Energy Station Based on Multi-agent Safe Reinforcement Learning

更新时间：2026-09-23

- Two-layer Robust Planning of Urban Integrated Energy Station Based on Multi-agent Safe Reinforcement Learning
- Automation of Electric Power Systems Pages: 1-18(2026)
- 作者机构：
  
  数字能源福建省高校重点实验室（福州大学），福建省福州市 350108
- 作者简介：
- 基金信息：
  
  National Natural Science Foundation of China(72501064;52377087);Fujian Provincial Natural Science Foundation of China(2025J08055)
- DOI：10.7500/AEPS20251228001
  CLC：
- Received：28 December 2025，
  
  Revised：2026-03-24，
  
  Online First：03 August 2026，
- 稿件说明：
移动端阅览
吕凯,邵振国,陈昌铭等.基于多智能体安全强化学习的城市综合能源站双层鲁棒规划[J/OL].电力系统自动化, http://dx.doi.org/10.7500/AEPS20251228001.

LYU Kai,SHAO Zhenguo,CHEN Changming,et al.Two-layer Robust Planning of Urban Integrated Energy Station Based on Multi-agent Safe Reinforcement Learning[J/OL].Automation of Electric Power Systems, http://dx.doi.org/10.7500/AEPS20251228001.
吕凯,邵振国,陈昌铭等.基于多智能体安全强化学习的城市综合能源站双层鲁棒规划[J/OL].电力系统自动化, http://dx.doi.org/10.7500/AEPS20251228001. DOI：

LYU Kai,SHAO Zhenguo,CHEN Changming,et al.Two-layer Robust Planning of Urban Integrated Energy Station Based on Multi-agent Safe Reinforcement Learning[J/OL].Automation of Electric Power Systems, http://dx.doi.org/10.7500/AEPS20251228001. DOI：

摘要

针对城市综合能源站选址定容规划中面临的负荷不确定性、地理信息复杂等挑战，提出一种基于多智能体安全强化学习的双层鲁棒规划方法。首先，构建城市综合能源站选址与定容的双层规划框架：上层基于位置集合覆盖模型，融合多能负荷时空特性与地理信息系统数据，以经济性和覆盖效率最优为目标优化城市综合能源站的选址策略；下层考虑多阶段建设时序，以全生命周期成本最小为目标优化设备的多阶段定容与运行策略。然后，提出多智能体安全近端策略优化算法，处理规划过程中的复杂高维不确定性问题。所提算法通过引入价值约束网络，将优化策略引导至安全域内，并采用增广拉格朗日法将约束优化问题转化为无约束优化问题，通过智能体自适应学习不确定环境下的鲁棒优化策略。最后，以某实际城市为例进行算例分析。仿真结果表明，所提模型与算法在经济性、计算效率及应对长期负荷增长不确定性的鲁棒性方面均优于传统方法，能为城市能源系统规划提供有效的决策支持。

Abstract

To address challenges including load uncertainty and complex geographic information faced in the site selection and capacity determination planning of urban integrated energy stations

this paper proposes a two-layer robust planning method based on multi-agent safe reinforcement learning. Firstly

a two-layer planning framework for the site selection and capacity determination of urban integrated energy stations is constructed. At the upper level

based on the location set-covering model

the spatio-temporal characteristics of multi-energy loads and geographic information system data are integrated to optimize the site selection strategy of urban integrated energy stations with the objectives of optimal economy and coverage efficiency. At the lower level

considering the multi-stage construction sequence

the multi-stage capacity determination and operation strategies of equipment are optimized to minimize the full-life-cycle cost. Secondly

a multi-agent safe proximal policy optimization algorithm is proposed to tackle complex high-dimensional uncertainty problems in the planning process. By introducing a value constraint network

the proposed algorithm guides the optimization policy into the safe feasible region. The augmented Lagrangian method is adopted to transform the constrained optimization problem into an unconstrained one

where agents adaptively learn robust optimization policies under uncertain environments. Finally

numerical cases are carried out based on a real-world city. Simulation results demonstrate that the proposed model and algorithm outperform conventional methods in terms of economic performance

computational efficiency

and robustness against long‑term load-growth uncertainty

and can provide effective decision-making support for urban energy system planning.

关键词

Keywords

references

刘硕，滕云，陈哲 . 融合减碳型多能源微网的城市能源系统环境-经济协调优化模型［J］. 电工技术学报， 2025 ， 40 （ 23 ）： 7532 - 7553 .

Liu Shuo ， Teng Yun ， Chen Zhe . An environment-economic coordination optimization model for urban energy systems integrating carbon-reducing multi-energy microgrids ［J］. Transactions of China Electrotechnical Society ， 2025 ， 40 （ 23 ）： 7532 - 7553 .

李争博，刘友波，任鹏哲，等 . 考虑地理信息聚类分区的配电网多阶段动态规划方法［J］. 电力系统自动化， 2022 ， 46 （ 14 ）： 38 - 45 .

Li Zhengbo ， Liu Youbo ， Ren Pengzhe ， et al . Multi-stage dynamic planning method for distribution network considering clustering and partitioning of geographic information ［J］. Automation of Electric Power Systems ， 2022 ， 46 （ 14 ）： 38 - 45 .

朱继忠，骆腾燕，吴皖莉，等 . 综合能源系统运行可靠性评估评述Ⅰ：模型驱动法［J］. 电工技术学报， 2022 ， 37 （ 11 ）： 2761 - 2776 .

Zhu Jizhong ， Luo Tengyan ， Wu Wanli ， et al . A review of operational reliability assessment of integrated energy systems Ⅰ： model-driven method ［J］. Transactions of China Electrotechnical Society ， 2022 ， 37 （ 11 ）： 2761 - 2776 .

王兆强，刘青荣，阮应君，等 . 区域能源规划中站网布置优化设计方法研究与应用［J］. 热能动力工程， 2019 ， 34 （ 8 ）： 25 - 30 .

Wang Zhaoqiang ， Liu Qingrong ， Ruan Yingjun ， et al . Research and application of optimal layout method for station network layout in regional energy planning ［J］. Journal of Engineering for Thermal Energy and Power ， 2019 ， 34 （ 8 ）： 25 - 30 .

Maroufmashat A ， Elkamel A ， Fowler M ， et al . Modeling and optimization of a network of energy hubs to improve economic and emission considerations ［J］. Energy ， 2015 ， 93 ： 2546 - 2558 .

黄伟，柳思岐，叶波，等 . 基于电-热-气混合潮流的园区综合能源系统站-网协同规划［J］. 电力建设， 2019 ， 40 （ 11 ）： 73 - 86 .

Huang Wei ， Liu Siqi ， Ye Bo ， et al . Station-network collaborative planning based on power-heat-gas multi-energy flow of RIES ［J］. Electric Power Construction ， 2019 ， 40 （ 11 ）： 73 - 86 .

江城，王浩，徐辰冠，等 . 考虑互联协同的分布式能源站低碳多目标选址规划研究［J］. 全球能源互联网， 2024 ， 7 （ 3 ）： 256 - 270 .

Jiang Cheng ， Wang Hao ， Xu Chenguan ， et al . Low-carbon multi-objective site selection planning for distributed energy stations considering interconnection synergy ［J］. Journal of Global Energy Interconnection ， 2024 ， 7 （ 3 ）： 256 - 270 .

徐成司，董树锋，吴金城，等 . 考虑区域综合能源系统拓扑特性的能源站和管线规划［J］. 电力系统自动化， 2020 ， 44 （ 3 ）： 74 - 82 .

Xu Chengsi ， Dong Shufeng ， Wu Jincheng ， et al . Planning of energy station and pipeline considering topological characteristics of regional integrated energy system ［J］. Automation of Electric Power Systems ， 2020 ， 44 （ 3 ）： 74 - 82 .

陈娟，黄元生，鲁斌 . 区域能源互联网“站-网”布局优化研究［J］. 中国电机工程学报， 2018 ， 38 （ 3 ）： 675 - 684 .

Chen Juan ， Huang Yuansheng ， Lu Bin . Research on “stations-pipelines” layout and optimization of regional energy Internet ［J］. Proceedings of the CSEE ， 2018 ， 38 （ 3 ）： 675 - 684 .

孟政吉，陈铄鑫，胡雪凯，等 . 多区域分布式能源站选址及供能管线布局规划研究［J］. 河北电力技术， 2022 ， 41 （ 1 ）： 23 - 27 .

Meng Zhengji ， Chen Shuoxin ， Hu Xuekai ， et al . Research on location selection and pipeline layout planning of multi regional distributed energy stations ［J］. Hebei Electric Power ， 2022 ， 41 （ 1 ）： 23 - 27 .

Chen C M ， Zhang T H ， Wang Y C ， et al . Coordinated siting and sizing for integrated energy system considering generalized energy storage ［J］. International Journal of Electrical Power & Energy Systems ， 2024 ， 155 ： 109619 .

Cunden T S M ， Doorga J ， Lollchund M R ， et al . Multi-level constraints wind farms siting for a complex terrain in a tropical region using MCDM approach coupled with GIS ［J］. Energy ， 2020 ， 211 ： 118533 .

Jayarathna L ， Kent G ， O’Hara I ， et al . A Geographical Information System based framework to identify optimal location and size of biomass energy plants using single or multiple biomass types ［J］. Applied Energy ， 2020 ， 275 ： 115398 .

Marques-Perez I ， Guaita-Pradas I ， Gallego A ， et al . Territorial planning for photovoltaic power plants using an outranking approach and GIS ［J］. Journal of Cleaner Production ， 2020 ， 257 ： 120602 .

Unternährer J ， Moret S ， Joost S ， et al . Spatial clustering for district heating integration in urban energy systems： Application to geothermal energy ［J］. Applied Energy ， 2017 ， 190 ： 749 - 763 .

Finney K N ， Sharifi V N ， Swithenbank J ， et al . Developments to an existing city-wide district energy network： Part Ⅰidentification of potential expansions using heat mapping ［J］. Energy Conversion and Management ， 2012 ， 62 ： 165 - 175 .

Molina Bacca E J ， Knight A ， Trifkovic M . Optimal land use and distributed generation technology selection via geographic-based multicriteria decision analysis and mixed-integer programming ［J］. Sustainable Cities and Society ， 2020 ， 55 ： 102055 .

窦嘉铭，王小君，刘曌，等 . 面向综合能源系统调度多场景泛化提升的扩散策略学习［J］. 电力系统自动化， 2026 ， 50 （ 3 ）： 145 - 155 .

Dou Jiaming ， Wang Xiaojun ， Liu Zhao ， et al . Diffusion-policy learning for multi-scenario generalized boosting of integrated energy system scheduling ［J］. Automation of Electric Power Systems ， 2026 ， 50 （ 3 ）： 145 - 155 .

张薇，王浚宇，杨茂，等 . 基于分布式双层强化学习的区域综合能源系统多时间尺度优化调度［J］. 电工技术学报， 2025 ， 40 （ 11 ）： 3529 - 3544 .

Zhang Wei ， Wang Junyu ， Yang Mao ， et al . The multi-time-scale optimal scheduling for regional integrated energy system based on the distributed bi-layer reinforcement learning ［J］. Transactions of China Electrotechnical Society ， 2025 ， 40 （ 11 ）： 3529 - 3544 .

https：//github.com/cityflow-project/CityFlow https://github.com/cityflow-project/CityFlow .

Tang Z ， Naphade M ， Liu M Y ， et al . CityFlow： a city-scale benchmark for multi-target multi-camera vehicle tracking and re-identification ［C］// IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）， 2019 ： 8789 - 8798 .

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Economic Dispatch of Energy Station with Building Virtual Energy Storage in Demand Response Mechanism

考虑微电网运行域的配电网分布式能源规划

计及元胞发展程度的空间负荷预测方法

Adaptive Control Strategy for Grid-following Converter Considering DC Synchronization Characteristics in Weak Grid

Practices and Recommendations of Market-based Capacity Adequacy Mechanisms for New Power Systems

Related Author

Jingyi LIU

Hong LIU

Yiran WANG

Chenxiao ZHAO

Wei SUN

Shaoyun GE

CHEN Yaosheng

XIAO Bai

Related Institution

Key Laboratory of the Ministry of Education on Smart Power Grids(Tianjin University)

Sunwoda Electronic Solution Co., Ltd.

School of Electric Power, South China University of Technology

School of Electrical Engineering, Northeast Electric Power University

Research Center of Smart Energy Technology, School of Electric Power Engineering, South China University of Technology

AI问答

⁰