To maximize the profits of power grid operators (GOs)

load aggregators (LAs) and electricity customers (ECs)

this paper proposes a hierarchical demand response (HDR) framework that considers competing interaction based on multi-agent deep deterministic policy gradient (MaDDPG). The ECs are divided into conventional ECs and the electric vehicles (EVs) which are managed by ECs agent (ECA) and EV agent (EVA) to exploit the flexibility of the HDR framework. Thus

the HDR is a tri-layer model determined by five types of agents engaging in competing interaction to maximize their own profits. To address the limitations of mathematical expression and participation scale in the Stackelberg game within the HDR model

a dynamic interaction mechanism is adopted. Moreover

to tackle the HDR involving various entities

the MaDDPG develops multiple agents to simulation the dynamic competing interactions between each subject as well as solve the problem of continuous action control. Furthermore

MaDDPG adopts soft target update and priority experience replay method to ensure stable and effective training

and makes the exploration strategy comprehensive by using exploration noise. Simulation studies are conducted to verify the performance of the MaDDPG with dynamic interaction mechanism in dealing with multilayer multi-agent continuous action control

compared to the double deep Q network (DDQN)

deep Q network (DQN) and dueling DQN. Additionally

comparisons among the proposed HDR with the price based DR (PBDR) and incentive based DR (IBDR) are analyzed to investigate the flexibility of the HDR.