With the liberalization of the retail market

customers can sell their demand response (DR) resources to the distribution company (Disco) through the DR aggregator (DRA). In this paper

an intelligent DR resource trading framework between Disco and DRA is proposed by exploiting the benefits of deep reinforcement learning (DRL). The hierarchical decision process of the two players is modeled as a Stackelberg game. In the game

Disco is the leader who determines the retail price while DRA is the follower who responds to it. To protect their privacy

a dueling deep Q-network (dueling DQN) is then constructed to model the bi-Ievel Stackelberg game

such that the lower-level problem doesn't need to reveal its detailed model to the upper-level. In the learning process

the uncertainties from the DRA's baseline load and wind power are considered. In order to boost the robustness against the estimation error

the baseline load is discretized into symbols before being used as the input states of the dueling DQN. And to mitigate the uncertainty of wind power

the scenario-based method is introduced when designing the reward. We demonstrate that the proposed dueling DQN-based method has good performance and is more robust against uncertainties.