Abstract: To guide the transformer fault diagnosis based on dissolved gas analysis (DGA), it is necessary to study the thermal cracking mechanism and gas behavior of mixed oil at the atomic level. Based on ReaxFF molecular dynamics (MD), we built models of single molecule, multi-molecule, and three-element mixed insulation oil, and simulated the thermal cracking kinetics process of these models. Moreover, we studied the thermal cracking paths and gas behavior of different types of molecules in mineral oil, natural ester, and modified natural ester, as well as the gas production characteristics of mixed insulation oil. Then, the simulation results were verified by overheating fault tests. The results show that the oil molecules are transformed into chain hydrocarbons through the reactions of decycling, decarboxylation, and decarbonylation under thermal stress, and then gradually cleaved into small hydrocarbon gas molecules. The increase of molecule number in the system does not affect the thermal cracking paths, but it will accelerate the cracking process. The simulation results of thermal cracking of three-element mixed insulation oil indicate that CO₂ content in the oil is the highest when low-temperature overheating faults happen. H₂ and hydrocarbon gas contents increase when medium-temperature overheating faults happen. When high-temperature overheating fault happen, the C₂H₂ content increases, and C₂H₄, and C₂H₆ contents increase significantly. At last, the test results of gas behavior in mixed insulation oil are consistent with the simulation results. This paper provides theoretical guidance for the implementation of the three-element mixed insulation oil transformer overheating fault diagnosis based on DGA.