Abstract: The safe operation of China's new electric power system highly relies on the relay protection device as the first line of defense, and it is of great significance to comprehensively evaluate the operation level of relay protection. Because of the complicated structure of relay protection, numerous factors affecting their operation status, and the technical difficulties in determining weights of evaluation indexes, this paper proposed a comprehensive evaluation method based on fully referring to the existing evaluation indexes. This paper proposes five evaluation elements: equipment advancement, operating reliability, maintainability, assurance, and professional management normalization. The relationship between different elements is grasped based on configuration theory, and the comprehensive evaluation system of relay protection is obtained by eliminating redundant indexes by support vector machine. Based on the protection operation evaluation system's subjective and objective weighting results, a method for determining the combination weights based on the fit goodness of the sample evaluation result sequence to the normal distribution is proposed. The construction of the evaluation index system of relay protection operation and the scientific calculation of index weights provide an important reference basis for the operation and management of relay protection.