电力设备运维过程中积累了大量缺陷图像与文本数据,这些数据对指导电力设备故障诊断及维护决策具有重要意义。针对现有电力设备缺陷分类任务中数据形式单一、融合层次浅、数据质量差等问题,该文提出了一种基于改进注意力机制和对比学习的图文融合分类方法(image-text fusion classification method based on improved attention mechanism and contrastive learning,IAC-ITFusion)。首先,该方法设计了一种双循环跨模态注意力机制(dual-cycle cross-modal attention,DCCA),用于捕捉图文数据映射关系的同时整合特征信息。其次,基于对比学习的思想,提出了一种注意力引导损失函数,用于调控DCCA机制的学习方向,使其聚焦于正确的特征信息,实现图文数据特征的有效融合。最后,针对电力线、变电站设备缺陷图文融合分类任务进行实验验证,结果显示所提方法准确率分别达到98.48%和98.57%,证明了该方法在电力设备缺陷图文融合分类任务上的有效性,对于推动电力设备运维智能化发展具有重要意义。
Abstract
A large number of defect images and textual data have been accumulated during the operation and maintenance of power equipment. These data are crucial for guiding fault diagnosis and maintenance decisions for power equipment. To address issues such as the singular data modalities
low-level feature fusion
and poor data quality in existing power equipment defect classification tasks
this paper proposes an image-text fusion classification method based on an improved attention mechanism and contrastive learning (IAC-ITFusion). Firstly
the method introduces a dual-cycle cross-modal attention (DCCA) mechanism to capture the mapping relationships between image and text data while integrating feature information. Secondly
based on the concept of contrastive learning
an attention-guided loss function is proposed to regulate the learning direction of the DCCA mechanism
ensuring it to focus on the correct feature information to achieve effective fusion of image and text features. Finally
experiments were conducted on the image and text fusion classification tasks for power line and substation equipment defects. The results show that the proposed method can achieve accuracy rates of 98.48% and 98.57%
respectively
thus the method's effectiveness in image and text fusion classification tasks for power equipment defects is verified
which is significant for advancing the intelligent development of power equipment operation and maintenance.