Under the objective of achieving carbon peak and neutrality

a substantial amount of renewable energy sources have been integrated into the power system to replace traditional synchronous generators.However

their inherent uncertainty and low characteristics inertia pose challenges

including frequency fluctuations and oscillations in power systems.Benefiting from the recent advancement in power electronic technology

energy storage systems are capable of providing rapid frequency response.Among these

virtual inertia control effectively mitigates frequency deviations and rate of change of frequency.The implementation of virtual inertia control for energy storage systems usually needs to consider control cost and design optimal controller.Nevertheless

most of existing methods rely on linearized models

which fail to adequately address complex nonlinear dynamics.To this end

a Lyapunov-based reinforcement learning scheme is proposed in this paper

formulating virtual inertia control as a Markov decision process and designing the controller to ensure local stability via Lyapunov theory. The deep deterministic policy gradient algorithm optimizes controller parameters. Simulation results indicate that the proposed method significantly enhances frequency control performance.