Abstract: The multi-energy collaborative operation of distributed energy systems is of great significance for promoting the consumption of renewable energy. However, the uncertainty of sources and loads in distributed energy systems, as well as the spatiotemporal differences in heterogeneous energy networks, pose significant challenges to the optimization problem of multienergy collaboration. A two-stage multi-energy collaborative optimization model for distributed energy systems is proposed to address this issue. A two-stage decoupling decision-making approach of long-time scale control and short-time scale control is adopted, thereby achieving sequential decision-making for composite spaces with different time response characteristics. Subsequently, in the face of high-dimensional composite search space and source-load uncertainty factors, a deep reinforcement learning model free solution is adopted, and a novel hierarchical deep reinforcement learning algorithm is proposed for solving. The effectiveness and superiority of the proposed model and solving method are verified through numerical simulations.