A hierarchically correlated equilibrium Q-learning (HCEQ) algorithm for reactive power optimization that considers carbon emission on the grid-side as an optimization objective

is proposed here. Based on the multi-area decentralized collaborative framework

the controllable variables in each region are divided into several optimization layers

which is an effective method for solving the limitations posed by dimensionality. The HCEQ provides constant information on the interaction between the state-action value function matrices

as well as on the cooperative game equilibrium among agents in each region. After acquiring the optimal value function matrix in the pre-learning process

HCEQ is able to quickly achieve an optimal solution online. Simulation of the IEEE 57-bus system is performed

which demonstrates that the proposed algorithm can effectively solve multi-area decentralized collaborative reactive power optimization

with the desired global search capabilities and convergence speed.