Abstract: A large amount of renewable energy is integrated into the power grid, which reduces the inertia of the power system.The virtual inertia control（VIC） technology for the converter effectively improves the frequency stability of the power system. In this context, estimating the inertia parameter of the converter is of great importance to help understand the frequency-regulation potential of the power system. However, considering the differences in frequency response characteristics, the existing inertia estimation methods for synchronous generators are not suitable for the digital-control-based virtual inertia. To address this issue,this paper proposes a non-intrusive online estimation method of inertia parameters of the grid-forming converter based on deep reinforcement learning（DRL）. First, the frequency response principle and delay characteristic of the grid-forming converter are analyzed and tested. Then, the series-parallel identification structure is designed in the DRL framework. The proximal policy optimization（PPO） method is used to establish actor and critic networks considering frequency response delay, further estimating the inertia parameters of the grid-forming converter. Finally, numerical simulation and hardware-in-the-loop experiments are used to verify the effectiveness and accuracy of the proposed method.