Substation voltage and reactive power control strategy based on state exploration-aware deep reinforcement learning

YAN Heguizhi; YAN Wei; GAO Qian; ZHANG Haodong

doi:10.19783/j.cnki.pspc.251397

English Version Journal Login

您当前的位置：

首页 >

文章列表页 >

Substation voltage and reactive power control strategy based on state exploration-aware deep reinforcement learning

更新时间：2026-07-27

- Substation voltage and reactive power control strategy based on state exploration-aware deep reinforcement learning
- Power System Protection and Control Vol. 54, Issue 12, (2026)
- 作者机构：
  
  输变电装备技术全国重点实验室 (重庆),重庆,400044
- 作者简介：
- 基金信息：
- DOI：10.19783/j.cnki.pspc.251397
  CLC：
- Published：2026
- 稿件说明：
移动端阅览
YAN Heguizhi, YAN Wei, GAO Qian, et al. Substation voltage and reactive power control strategy based on state exploration-aware deep reinforcement learning[J]. Power System Protection and Control, 2026, 54(12).
DOI：

YAN Heguizhi, YAN Wei, GAO Qian, et al. Substation voltage and reactive power control strategy based on state exploration-aware deep reinforcement learning[J]. Power System Protection and Control, 2026, 54(12). DOI： 10.19783/j.cnki.pspc.251397.

摘要

新能源和间歇性负荷的大规模接入加剧了变电站电压无功运行波动，对变电站电压无功控制 (voltage and reactive power control

VQC) 策略提出了更高的要求。深度强化学习 (deep reinforcement learning

DRL) 为此提供了新思路，但现有 DRL 方法在训练阶段普遍存在探索偏置，限制了所学策略的在线应用性能。为此，提出一种基于状态探索感知 DRL 的变电站 VQC 策略。首先，构建状态探索感知 DRL (state exploration-aware DRL

SEA-DRL) 框架，通过无监督表示学习刻画状态空间结构，并基于状态访问频次设计动态探索激励机制，引导训练过程兼顾低频状态探索与策略优化性能，从而提升策略学习的整体效果。然后，针对变电站调节设备的离散特性，结合 SEA-DRL 框架与软行动者 - 评论家离散算法 (soft actor-critic for discrete action

SAC-D)，构建基于 SEA-SAC 的变电站 VQC 策略。最后，在某 C 市 220 kV 变电站的仿真结果表明，所提方法在复杂运行场景下具有良好的控制效果。

Abstract

The large-scale integration of renewable energy sources and intermittent loads has intensified fluctuations in substation voltage and reactive power

posing higher requirements for voltage and reactive power control (VQC). Deep reinforcement learning (DRL) offers a promising solution; however

existing DRL-based methods generally suffer from exploration bias during training

limiting their online control performance. To address this issue

this paper proposes a substation VQC strategy based on state-exploration-aware DRL (SEA-DRL). First

a SEA-DRL framework is developed

in which the intrinsic structure of the state space is captured via unsupervised representation learning. Based on state visitation frequency

a dynamic exploration incentive mechanism is then introduced to balance low-frequency state exploration and policy optimization

thereby enhancing overall learning performance. Then

considering the discrete operating characteristics of substation regulation equipment

the proposed framework is integrated with the soft actor-critic algorithm for discrete action spaces (SAC-D) to construct a SEA-SAC-based VQC strategy for substations. Finally

simulation results on a 220 kV substation in City C demonstrate that the proposed method achieves effective control performance under complex operating conditions.

关键词

Keywords

references

SUN H, GUO Q, ZHANG B, et al. An adaptive zone- division-based automatic voltage control system with applications in China[J]. IEEE Transactions on Power Systems, 2013, 28(2): 1816-1828.

SUN Q, WU Z, GU W, et al. Multi-stage co-planning model for power distribution system and hydrogen energy system under uncertainties[J]. Journal of Modern Power Systems and Clean Energy, 2023, 11(1): 80-93.

TIAN M, LI X, ZHU Z, et al. Robust voltage control for active distribution networks via safe deep reinforcement learning against state perturbations[J]. Protection and Control of Modern Power Systems, 2026, 11(1): 192-207.

WU P, CHEN C, LAI D, et al. Real-time optimal power flow method via safe deep reinforcement learning based on primal-dual and prior knowledge guidance[J]. IEEE Transactions on Power Systems, 2025, 40(1): 597-611.

MATEOS G, GIANNAKIS G B. Load curve data cleansing and imputation via sparsity and low rank[J]. IEEE Transactions on Smart Grid, 2013, 4(4): 2347-2355.

SONG K, BAEK Y S, HONG D, et al. Short-term load forecasting for the holidays using fuzzy linear regression method[J]. IEEE Transactions on Power Systems, 2005, 20(1): 96-101.

YAN R, XU Y. Multi-objective and multi-agent deep reinforcement learning for real-time decentralized volt/ var control of distribution networks considering PV inverter lifetime[J]. IEEE Transactions on Power Systems, 2025, 40(2): 1558-1569.

CHARIKAR M S. Similarity estimation techniques from rounding algorithms[C] // Proceedings of the 34th Annual ACM Symposium on Theory of Computing (STOC), May 19-21, 2002, Montreal, Canada: 380-388.

JING X, WANG R, LEI H, et al. Multi-agent discrete soft actor-critic algorithm-based multi-user collaborative anti- jamming strategy[J]. IEEE Transactions on Information Forensics and Security, 2025, 20: 5025-5038.

JOLLIFFE I T, CADIMA J. Principal component analysis: a review and recent developments[J]. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 2016, 374.

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Research and application of a new generation remote verification technology for substation relay protection

配电网中基于混合DRL的任务卸载与多资源协同调度优化方法

面向频率稳定的基于预生成经验池驱动的紧急切负荷智能在线决策

基于深度强化学习的考虑燃气动态掺氢的多能微网低碳经济调度

基于信息熵量化评价的深度强化学习储能微网柔性补偿自抗扰稳压技术

Related Author

TAN Qian

WANG Feng

WANG Liwei

FU Yuanhuan

LIU Wei

LU Ying

XU Jiayan

周雅

Related Institution

北京四方继保工程技术有限公司

广东电网有限责任公司佛山供电局

北京四方继保自动化股份有限公司

广东电网有限责任公司电力调度控制中心

华北水利水电大学

AI问答

⁰