Abstract: Virtual synchronous generator (VSG) provides a new solution to deal with insufficient inertia in a power-electronic-dominated power system. However, the VSG is derived from the mathematical model of synchronous generator, thus the problem of power oscillation of traditional SG also exists. Consequently, a three-order model of the synchronous generator is used to construct VSG. Based on the small-signal model of the VSG under the connection to infinite bus system, analysis by a complex torque coefficient method shows that, in certain conditions, the system may produce negative damping torque under the influences of line impedance and other factors. Meanwhile, the line resistance will introduce nonlinear effects into the negative damping torque, which has the risk of power oscillation. The additional damping torque compensator (ADTC) is designed to improve the damping characteristics of the system, in which a positive damping torque is introduced to suppress the power oscillation. Finally, the simulation is built to verify the influencing factors of negative damping torque and the feasibility and effectiveness of ADTC.