Abstract: For the safe and high-efficient operation of the new-type power systems, a multi-agent Actor-double-critic based real-time optimal dispatch method for source-grid-load- storage is proposed in this paper. This method aims to overcome the problems of model-driven dispatch methods, such as difficulties in solving the optimization model and the slow speed of real-time decision-making. The constrained Markov coalitional game model is established based on the Vickey-Clark-Groves mechanism and the real-time optimal dispatch model with the consideration of both controllable resources operation constraints and system safe constraints. Then, the centralized dispatch model can be transformed into a distributed optimization problem among multiple agents. Furthermore, the multi-agent Actor-double-critic algorithm is proposed, which utilizes the Self-critic and cons-critic networks to evaluate the action-value and action-cost of agents respectively, ultimately reducing the training difficulty and eliminating the influence of the sparse reward and sparse safety constraint cost. The proposed algorithm enhances the training convergence and ensures that the real-time dispatch decision can meet the system operation constraints. Finally, the simulation case verifies that the proposed method can significantly save dispatch decision-making time and guarantee the safe, reliable, and economic operation of the power system.