融合第一视角轻量目标检测的配电网带电作业操作监护多模态智能体构建

刘子豪; 刘友波; 宫昊辰; 万力; 袁林; 刘俊勇

doi:10.19783/j.cnki.pspc.251328

欢迎来到能源电力期刊网！

English Version 期刊登录

您当前的位置：

首页 >

文章列表页 >

融合第一视角轻量目标检测的配电网带电作业操作监护多模态智能体构建

工程应用 | 更新时间：2026-06-18

- 融合第一视角轻量目标检测的配电网带电作业操作监护多模态智能体构建
- Construction of a multimodal intelligent agent for live-line operation monitoring in distribution networks via first-person lightweight object detection fusion
- 电力系统保护与控制 2026年54卷第11期
- 作者机构：
- 作者简介：
- 基金信息：
- DOI：10.19783/j.cnki.pspc.251328
  中图分类号：
- 纸质出版：2026
- 稿件说明：
移动端阅览
刘子豪, 刘友波, 宫昊辰, 等. 融合第一视角轻量目标检测的配电网带电作业操作监护多模态智能体构建[J]. 电力系统保护与控制, 2026,54(11).

LIU Zihao, LIU Youbo, GONG Haochen, et al. Construction of a multimodal intelligent agent for live-line operation monitoring in distribution networks via first-person lightweight object detection fusion[J]. Power System Protection and Control, 2026, 54(11).
刘子豪, 刘友波, 宫昊辰, 等. 融合第一视角轻量目标检测的配电网带电作业操作监护多模态智能体构建[J]. 电力系统保护与控制, 2026,54(11). DOI： 10.19783/j.cnki.pspc.251328.

LIU Zihao, LIU Youbo, GONG Haochen, et al. Construction of a multimodal intelligent agent for live-line operation monitoring in distribution networks via first-person lightweight object detection fusion[J]. Power System Protection and Control, 2026, 54(11). DOI： 10.19783/j.cnki.pspc.251328.

摘要

针对配电网带电作业监护面临视角局限、边缘端算力不足，且缺乏专用轻量化智能方案支撑操作精准解析的问题，提出融合第一视角轻量目标检测的多模态智能体构建方法。首先，基于 YOLO11 模型引入改进多尺度注意力、风车形状卷积及跨视图交互模块，分别优化多尺度特征提取性能、低对比度特征轻量化提取性能与视图抗干扰性能，采用最小点距交并比损失函数优化边界框回归。然后，根据目标检测信息实现第一视角信息提取并生成结构化提示词。最后，结合目标检测模型与 DeepSeek-V3 构建多模态智能体，整合视觉检测结果，量化输出风险评估与操作分析。所提方法浮点运算次数为 7.3 G，其交并比阈值小于 0.5 的目标检测多类别平均精度显著优于主流单阶段模型。输出内容在状态判断准确率、结构化程度和内容可解释度上均优于主流开源多模态模型。所提方法为多场景配电网带电作业监护提供高效、可推广的方案。

Abstract

To address the limitations in live-line operation monitoring for distribution networks

including restricted field of view

insufficient edge computing capability

and the lack of dedicated lightweight intelligent solutions for accurate operation analysis

a construction method for multimodal intelligent agent integrating first-person lightweight object detection is proposed. First

built on the YOLO11 model

an improved architecture incorporating multi-scale attention mechanisms

windmill-shaped convolution

and cross-view interaction modules is introduced to enhance multi-scale feature extraction

lightweight low-contrast feature representation

and view robustness. The model adopts minimum point distance intersection over union (MPDIoU) loss function to refine bounding box regression. Then

first-person view information is extracted from detection results to generate structured prompts. Finally

a multimodal intelligent agent is constructed by integrating the optimized detection model with DeepSeek-V3

enabling the fusion of visual outputs for quantitative risk assessment and operation analysis. Experimental results show that the proposed method reduces computational complexity to 7.3 G FLOPs

and achieves up to a 9.44% improvement in mean precision for multi-class detection with IoU thresholds below 0.5 compared to mainstream single-stage models. The generated outputs outperform leading open-source multimodal models in state judgment accuracy

structuring degree

and interpretability

providing an efficient and scalable solution for multimodal monitoring of live-line operations in distribution networks.

关键词

Keywords

references

KOMAROMI P. Live working method comparisons: Rubber glove work vs. hotstick work vs. barehand work[C] // 2017 12th International Conference on Live Maintenance (ICOLIM), 2017: 1-6.

YANG Qi, ZOU Dehua, TONG Cheng, et al. The safety analysis of live working on 500/220 kV four-parallel lines on the same tower[C] // 2016 IEEE International Conference on Power and Renewable Energy (ICPRE), October 21-23, 2016, Shanghai, China: 15-18.

LOVRENČIĆ V, RIBIČ P, OGRIZEK P, et al. Safe work in the vicinity of live parts on high voltage double-circuit overhead lines[C] // 2022 13th International Conference on Live Maintenance (ICOLIM), June 15-17, 2022, Turin, Italy: 1-6.

ZHANG Haibo, ZHANG Yixuan, ZHONG Bin, et al. A comprehensive survey of vision-based human action recognition methods[J]. Sensors, 2019, 19(5).

YETGIN Ö E, BENLIGIRAY B, GEREK Ö N. Power line recognition from aerial images with deep learning[J]. IEEE Transactions on Aerospace and Electronic Systems, 2019, 55(5): 2241-2252.

GUO Hao, XU Liang, QIAO Ming, et al. A depth completion method used in live working safety monitoring tasks[C] // 2024 9th International Symposium on Computer and Information Processing Technology (ISCIPT), May 24-26, 2024, Xi’an, China: 427-432.

HU Wei, ZHENG Tingting, DU Puliang, et al. An intelligent assisted assessment method for distribution grid engineering quality based on large language modelling and ensemble learning[J]. Protection and Control of Modern Power Systems, 2025, 10(3): 83-97.

LIU Yuxuan, JIA Meng, ZHANG Yuxuan, et al. RePower: an LLM-driven autonomous platform for power system data-guided research[J]. Patterns, 2025, 6(4).

YANG Jiannan, LIU Shengjie, WU Jiayi, et al. Pinwheel- shaped convolution and scale-based dynamic loss for infrared small target detection[J]. arxiv preprint arxiv: 2412.16986, 2024.

WANG Yan, LI Ying, WANG Gang, et al. Multi-scale attention network for single image super-resolution[C] // Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, June 17-18, 2024, Seattle, WA, USA: 5950-5960.

LI Yijun, ZOU Wenbin, WEI Qiquan, et al. Multi-level feature fusion network for lightweight stereo image super-resolution[C] // Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, June 17-18, 2024, Seattle, WA, USA: 6489-6498.

MA Simao, XU Yong. MPDIoU: a loss for efficient and accurate bounding box regression[J]. arxiv preprint arxiv:2307.07662, 2023.

LIU Aixin, FENG Bei, XUE Bing, et al. Deepseek-v3 technical report[J]. arxiv preprint arxiv:2412.19437, 2024.

LIN Bin, YE Yang, ZHU Bin, et al. Video-llava: learning united visual representation by alignment before projection[J]. arxiv preprint arxiv:2311.10122, 2023.

BAI Shuai, CHEN Keqin, LIU Xian, et al. Qwen2.5-vl technical report[J]. arxiv preprint arxiv:2502.13923, 2025.

XU Jin, GUO Zhenru, HE Junyang, et al. Qwen2.5-omni technical report[J]. arxiv preprint arxiv: 2503.20215, 2025.

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

基于BERT模型的主设备缺陷诊断方法研究

基于单相背靠背变流器的不对称串并联混合扰动注入型阻抗测量装置

考虑温度影响的电动汽车集群市场化双型需求响应运营策略

同塔不换位线路的不对称度对差动保护的影响分析

基于生成对抗网络小样本扩展与多模态特征深度融合的变压器绕组故障诊断方法