Abstract: The offshore wind speed has a high spatial-temporal correlation, which aggravates the power fluctuation of the whole wind farm and poses significant challenges to the power system, especially when large-scale offshore wind power is integrated. Smoothing control of large-scale offshore wind power clusters is a key solution to mitigating the above problems. However, most existing methods are inefficient and difficult to support higher frequency control and are susceptible to wind power forecast errors and the deviation of actual action from the optimal control command. Therefore, this paper proposes a new control framework for “offline-training, online-optimization and self-evolution”, and establishes a deep-reinforcement-learning-based model for the smoothing control of the active power of offshore wind power clusters. Firstly, a short-term revenue function for cluster power smoothing control is proposed to solve the optimal command based on the Markov decision process model. Secondly, a long-term revenue policy function for power policy calibration is proposed to effectively correct the control deviation according to the historical feedback data. Finally, a deep neural network model is established for the mapping between the state of the agent, the control benefit and the control decision to realize the training and solution of the agent based on the deep deterministic policy gradient algorithm. The results show that the proposed method can reduce the power fluctuation by 20% and control the power loss within 5% under the given wind condition of 7.5 m/s average wind speed.