Detection and assessment of bird nests on transmission line towers are crucial for preventing faults in transmission lines and ensuring the safe and stable operation of power systems. The stability of the bird nests on transmission lines is closely related to its structural and shape characteristics

and bird nests with asymmetric structure or abnormal shape are more prone to material shedding

thus causing transmission line faults. Current risk assessments of bird nest faults primarily focus on object detection and localization

without fully accounting for the impact of the structural differences and shape characteristics of bird nests

resulting in inaccurate assessment results. To this end

a fine-grained detection method for bird nests on transmission lines based on multimodal fusion is proposed

comprising following three modules: image feature extraction

semantic information extraction

and multimodal fusion. The image feature extraction module utilizes the K-Net image segmentation framework to accurately segment the contours of bird nests and crop the images

reducing irrelevant information interference and extracting image modality features. The semantic information extraction module focuses on extracting shape and structural semantic information closely related to the sagging trends of bird nests

and a semantic correlation model for dynamic correction is established to generate effective text modal features. The multimodal fusion module adopts the Low-rank Multimodal Fusion (LMF) strategy

in which the semantic understanding of bird nest structure and shape from the text modality are utilized to compensate for the fine details that the image modality cannot capture

in order to generate more representative fusion features for fine-grained detection of bird nests. Experimental results demonstrate that this method can be adopted to further distinguish the sagging state of the bird nests on the basis of object detection and localization

achieving a detection accuracy of 92.59%

which provides a reliable basis for bird nest failure risk assessment and prevention work.