The deregulation of power markets is expected to extend to heat markets in the future. Considering that heat is not suitable for long-distance transmission

the coupling operation of electricity and heat markets at the regional level is more practical. Therefore

a combinatorial double auction market (CDAM) is designed in this paper. The optimal bidding strategy based on hierarchical reinforcement learning for prosumers is proposed. First

according to the characteristics of energy coupling in regional electricity-heat market

a double auction mechanism

including bidding mechanism and clearing mechanism

is established

taking the combination of electricity and heat as a single commodity. Then

considering the complexity of the bidding environment in the regional electricity-heat market

an optimal bidding strategy based on hierarchical multi-agent reinforcement learning is proposed

which includes the upper and lower layers learning the bidding information of market public and private players respectively. Finally

taking a regional integrated energy system as an example

the simulation results show that the proposed combined double auction mechanism and optimal bidding strategy can increase the economic benefit of the regional integrated energy system by 20.1%. At the same time

compared with common “cost plus method”

“forecast clearing method” and “traditional reinforcement learning”

the proposed bidding strategy is more competitive in the market.