基于状态探索感知深度强化学习的变电站电压无功控制策略

闫何贵枝; 颜伟; 高倩; 张昊栋

doi:10.19783/j.cnki.pspc.251397

欢迎来到能源电力期刊网！

English Version 期刊登录

您当前的位置：

首页 >

文章列表页 >

基于状态探索感知深度强化学习的变电站电压无功控制策略

理论分析 | 更新时间：2026-07-27

- 基于状态探索感知深度强化学习的变电站电压无功控制策略
- Substation voltage and reactive power control strategy based on state exploration-aware deep reinforcement learning
- 电力系统保护与控制 2026年54卷第12期
- 作者机构：
  
  输变电装备技术全国重点实验室 (重庆),重庆,400044
- 作者简介：
- 基金信息：
- DOI：10.19783/j.cnki.pspc.251397
  中图分类号：
- 纸质出版：2026
- 稿件说明：
移动端阅览
闫何贵枝, 颜伟, 高倩, 等. 基于状态探索感知深度强化学习的变电站电压无功控制策略[J]. 电力系统保护与控制, 2026,54(12).

YAN Heguizhi, YAN Wei, GAO Qian, et al. Substation voltage and reactive power control strategy based on state exploration-aware deep reinforcement learning[J]. Power System Protection and Control, 2026, 54(12).
闫何贵枝, 颜伟, 高倩, 等. 基于状态探索感知深度强化学习的变电站电压无功控制策略[J]. 电力系统保护与控制, 2026,54(12). DOI： 10.19783/j.cnki.pspc.251397.

YAN Heguizhi, YAN Wei, GAO Qian, et al. Substation voltage and reactive power control strategy based on state exploration-aware deep reinforcement learning[J]. Power System Protection and Control, 2026, 54(12). DOI： 10.19783/j.cnki.pspc.251397.

摘要

新能源和间歇性负荷的大规模接入加剧了变电站电压无功运行波动，对变电站电压无功控制 (voltage and reactive power control

VQC) 策略提出了更高的要求。深度强化学习 (deep reinforcement learning

DRL) 为此提供了新思路，但现有 DRL 方法在训练阶段普遍存在探索偏置，限制了所学策略的在线应用性能。为此，提出一种基于状态探索感知 DRL 的变电站 VQC 策略。首先，构建状态探索感知 DRL (state exploration-aware DRL

SEA-DRL) 框架，通过无监督表示学习刻画状态空间结构，并基于状态访问频次设计动态探索激励机制，引导训练过程兼顾低频状态探索与策略优化性能，从而提升策略学习的整体效果。然后，针对变电站调节设备的离散特性，结合 SEA-DRL 框架与软行动者 - 评论家离散算法 (soft actor-critic for discrete action

SAC-D)，构建基于 SEA-SAC 的变电站 VQC 策略。最后，在某 C 市 220 kV 变电站的仿真结果表明，所提方法在复杂运行场景下具有良好的控制效果。

Abstract

The large-scale integration of renewable energy sources and intermittent loads has intensified fluctuations in substation voltage and reactive power

posing higher requirements for voltage and reactive power control (VQC). Deep reinforcement learning (DRL) offers a promising solution; however

existing DRL-based methods generally suffer from exploration bias during training

limiting their online control performance. To address this issue

this paper proposes a substation VQC strategy based on state-exploration-aware DRL (SEA-DRL). First

a SEA-DRL framework is developed

in which the intrinsic structure of the state space is captured via unsupervised representation learning. Based on state visitation frequency

a dynamic exploration incentive mechanism is then introduced to balance low-frequency state exploration and policy optimization

thereby enhancing overall learning performance. Then

considering the discrete operating characteristics of substation regulation equipment

the proposed framework is integrated with the soft actor-critic algorithm for discrete action spaces (SAC-D) to construct a SEA-SAC-based VQC strategy for substations. Finally

simulation results on a 220 kV substation in City C demonstrate that the proposed method achieves effective control performance under complex operating conditions.

关键词

Keywords

references

SUN H, GUO Q, ZHANG B, et al. An adaptive zone- division-based automatic voltage control system with applications in China[J]. IEEE Transactions on Power Systems, 2013, 28(2): 1816-1828.

SUN Q, WU Z, GU W, et al. Multi-stage co-planning model for power distribution system and hydrogen energy system under uncertainties[J]. Journal of Modern Power Systems and Clean Energy, 2023, 11(1): 80-93.

TIAN M, LI X, ZHU Z, et al. Robust voltage control for active distribution networks via safe deep reinforcement learning against state perturbations[J]. Protection and Control of Modern Power Systems, 2026, 11(1): 192-207.

WU P, CHEN C, LAI D, et al. Real-time optimal power flow method via safe deep reinforcement learning based on primal-dual and prior knowledge guidance[J]. IEEE Transactions on Power Systems, 2025, 40(1): 597-611.

MATEOS G, GIANNAKIS G B. Load curve data cleansing and imputation via sparsity and low rank[J]. IEEE Transactions on Smart Grid, 2013, 4(4): 2347-2355.

SONG K, BAEK Y S, HONG D, et al. Short-term load forecasting for the holidays using fuzzy linear regression method[J]. IEEE Transactions on Power Systems, 2005, 20(1): 96-101.

YAN R, XU Y. Multi-objective and multi-agent deep reinforcement learning for real-time decentralized volt/ var control of distribution networks considering PV inverter lifetime[J]. IEEE Transactions on Power Systems, 2025, 40(2): 1558-1569.

CHARIKAR M S. Similarity estimation techniques from rounding algorithms[C] // Proceedings of the 34th Annual ACM Symposium on Theory of Computing (STOC), May 19-21, 2002, Montreal, Canada: 380-388.

JING X, WANG R, LEI H, et al. Multi-agent discrete soft actor-critic algorithm-based multi-user collaborative anti- jamming strategy[J]. IEEE Transactions on Information Forensics and Security, 2025, 20: 5025-5038.

JOLLIFFE I T, CADIMA J. Principal component analysis: a review and recent developments[J]. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 2016, 374.

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

变电站新一代继电保护远程验证技术研究及应用

配电网中基于混合DRL的任务卸载与多资源协同调度优化方法

面向频率稳定的基于预生成经验池驱动的紧急切负荷智能在线决策

基于深度强化学习的考虑燃气动态掺氢的多能微网低碳经济调度

基于信息熵量化评价的深度强化学习储能微网柔性补偿自抗扰稳压技术