基于高效时序建模的RGB-T跟踪

倪新宇; 曾碧; 张仲轩

doi:10.12052/gdutxb.240169

欢迎来到能源电力期刊网！

English Version 期刊登录

您当前的位置：

首页 >

文章列表页 >

基于高效时序建模的RGB-T跟踪

计算机科学与技术 | 更新时间：2026-05-27

- 基于高效时序建模的RGB-T跟踪
- Efficient Temporal Modeling for RGB-T Tracking
- 广东工业大学学报 2026年43卷第2期页码：21-29
- 作者机构：
  
  广东工业大学计算机学院,广东,广州,510006
- 作者简介：
  
  倪新宇(2000–) ，女，硕士研究生，主要研究方向为计算机视觉、目标跟踪，E-mail：2112205066@mail2.gdut.edu.cn
  曾碧(1963–) ，女，教授，博士，主要研究方向为机器学习、计算机视觉，E-mail：zb9215@gdut.edu.cn
- 基金信息：
  
  国家自然科学基金资助项目(62172111);广东省自然科学基金资助项目(2021A1515012233)
- DOI：10.12052/gdutxb.240169
  中图分类号： TP391
- 纸质出版：2026
- 稿件说明：
移动端阅览
倪新宇, 曾碧, 张仲轩. 基于高效时序建模的RGB-T跟踪[J]. 广东工业大学学报, 2026, 43(2): 21-29.
倪新宇, 曾碧, 张仲轩. 基于高效时序建模的RGB-T跟踪[J]. 广东工业大学学报, 2026, 43(2): 21-29. DOI： 10.12052/gdutxb.240169.

DOI： 10.12052/gdutxb.240169

摘要

可见光−热红外(RGB-Thermal，RGB-T)跟踪方法利用可见光和热红外图像的互补性，提升了在低光照以及恶劣天气等场景下目标跟踪的准确度。然而，现有的研究大多局限于图像级外观匹配，难以应对目标形变、复杂环境干扰等挑战。对此，本文提出了一种基于高效时序建模的跟踪方法(Efficient Temporal Modeling for RGB-T Tracking

ETMTrack) 。首先，利用时序信息建模，并改进特征融合模块以处理时序信息。然后，利用轻量级适配器微调改进热红外图像特征提取模块，提高模型对不同模态信息的特征提取能力，并降低计算显存，提高训练效率。最后，提出一种动态模板更新和选择方法，充分挖掘并利用时序信息，提高模型性能。本文模型在3个公开数据集上达到了最先进的性能，在遮挡、相似外观等挑战上表现出色，验证了时序建模跟踪算法的有效性和鲁棒性。

Abstract

RGB-Thermal (RGB-T) tracking methods utilize the complementarity of visible light and thermal infrared images to improve the accuracy of target tracking in the scenarios of low light conditions and adverse weather. However

most existing studies focus only on image-level appearance matching

making them difficult to cope with challenges of target deformation and interference under complex environments. To address this problem

a tracking method based on efficient temporal modeling is proposed. Firstly

the temporal information is modeled

and the feature fusion module is improved to process temporal information. Then

a lightweight adapter is used for fine-tuning to improve the feature extraction module for thermal infrared images

enhancing the model's ability to extract features from different modal information

reducing the computational memory usage

and improving the training efficiency. Finally

a dynamic template update and selection method is proposed to fully explore and utilize temporal information

thereby improving the model's performance. ETMTrack achieves state-of-the-art performance on three public datasets

and performs excellently in dealing with challenges such as occlusion and similar appearances

demonstrating the effectiveness and robustness of the tracking algorithm based on temporal modeling.

关键词

Keywords

references

GALLEGO J, PARDAS M, LANDABASO J L. Segmentation and tracking of static and moving objects in video surveillance scenarios[C]//2008 15th IEEE International Conference on Image Processing. San Diego: IEEE, 2008: 2716-2719.

LI J, ZHAN W, HU Y, et al . Generic tracking and probabilistic prediction framework and its application in autonomous driving[J ] . IEEE Transactions on Intelligent Transportation Systems, 2019, 21(9): 3634-3649.

LIN L, FAN H, ZHANG Z, et al . SwinTrack: a simple and strong baseline for transformer tracking[J ] . Advances in Neural Information Processing Systems, 2022, 35: 16743-16754.

CHEN B, LI P, BAI L, et al . Backbone is all your need: a simplified architecture for visual object tracking[C ] //European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2022: 375-392.

SUN Y, CAO B, ZHU P, et al . Drone-based RGB-infrared cross-modality vehicle detection via uncertainty-aware learning[J ] . IEEE Transactions on Circuits and Systems for Video Technology, 2022, 32(10): 6700-6713.

FENG G, BAO L. Face recognition method in complex environment based on infrared visible fusion[J]. Journal of Guangdong University of Technology, 2024, 41(3): 62-70,109.

LI C, LIU L, LU A, et al . Challenge-aware RGBT tracking[C ] //European Conference on Computer Vision. Cham: Springer International Publishing, 2020: 222-237.

ZHANG P, ZHAO J, WANG D, et al . Visible-thermal UAV tracking: a large-scale benchmark and new baseline[C ] //Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans: IEEE, 2022: 8886-8895.

LIU L, LI C, XIAO Y, et al . RGBT tracking via challenge-based appearance disentanglement and interaction[J ] . IEEE Transactions on Image Processing, 2024, 33: 1753-1767.

WANG H, LIU X, LI Y, et al . Temporal adaptive RGBT tracking with modality prompt[C ] //Proceedings of the AAAI Conference on Artificial Intelligence. Vancouver: IEEE, 2024: 5436-5444.

WU Y, BLASCH E, CHEN G, et al . Multiple source data fusion via sparse representation for robust visual tracking[C ] //14th International Conference on Information Fusion. Chicago: IEEE, 2011: 1-8.

ZHAI S, SHAO P, LIANG X, et al . Fast RGB-T tracking via cross-modal correlation filters[J ] . Neurocomputing, 2019, 334: 172-181.

LI C, WU X, ZHAO N, et al . Fusing two-stream convolutional neural networks for RGB-T object tracking[J ] . Neurocomputing, 2018, 281: 78-85.

ZHANG H, ZHANG L, ZHUO L, et al . Object tracking in RGB-T videos using modal-aware attention network and competitive learning[J ] . Sensors, 2020, 20(2): 393.

VASWANI A, SHAZEER N, PARMAR N, et al . Attention is all you need[J ] . Advances in Neural Information Processing Systems, 2017, 30: 6000-6010.

HUI T, XUN Z, PENG F, et al . Bridging search region interaction with template for RGB-t tracking[C ] //Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Vancouver: IEEE, 2023: 13630-13639.

ZHU J, LAI S, CHEN X, et al . Visual prompt multi-modal tracking[C ] //Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Vancouver: IEEE, 2023: 9516-9526.

YAN B, PENG H, FU J, et al . Learning spatio-temporal transformer for visual tracking[C ] //Proceedings of the IEEE/CVF International Conference on Computer Vision. Montreal: IEEE, 2021: 10448-10457.

CUI Y, JIANG C, WANG L, et al . Mixformer: end-to-end tracking with iterative mixed attention[C ] //Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans: IEEE, 2022: 13608-13618.

XIE F, CHU L, LI J, et al . Videotrack: learning to track objects via video transformer[C ] //Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Vancouver: IEEE, 2023: 22826-22835.

WEI X, BAI Y, ZHENG Y, et al . Autoregressive visual tracking[C ] //Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Vancouver: IEEE, 2023: 9697-9706.

CAI Y, LIU J, TANG J, et al . Robust object modeling for visual tracking[C ] //Proceedings of the IEEE/CVF International Conference on Computer Vision. Paris: IEEE, 2023: 9589-9600.

ZHENG Y, ZHONG B, LIANG Q, et al . ODTrack: online dense temporal token learning for visual tracking[C ] //Proceedings of the AAAI Conference on Artificial Intelligence. Vancouver: IEEE, 2 024: 7588-7596.

HOU X, XING J, QIAN Y, et al . SDSTrack: self-distillation symmetric adapter learning for multi-modal visual object tracking[C ] //Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2024: 26551-26561.

HONG L, YAN S, ZHANG R, et al . OneTracker: unifying visual object tracking with foundation models and efficient tuning[C ] //Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2024: 19079-19091.

YE B, CHANG H, MA B, et al . Joint feature learning and relation modeling for tracking: a one-stream framework[C ] //European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2022: 341-357.

LAW H, DENG J. Cornernet: detecting objects as paired keypoints[C]//Proceedings of the European Conference on Computer Vision (ECCV) . Munich: Springer, 2018: 734-750.

REZATOFIGHI H, TSOI N, GWAK J Y, et al . Generalized intersection over union: a metric and a loss for bounding box regression[C ] //Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019: 658-666.

LI C, XUE W, JIA Y, et al . LasHeR: a large-scale high-diversity benchmark for RGBT tracking[J ] . IEEE Transactions on Image Processing, 2021, 31: 392-404.

LI C, ZHAO N, LU Y, et al . Weighted sparse representation regularized graph learning for RGB-T object tracking[C ] //Proceedings of the 25th ACM International Conference on Multimedia. New York: Association for Computing Machinery, 2017: 1856-1864.

LI C, LIANG X, LU Y, et al . RGB-T object tracking: benchmark and baseline[J ] . Pattern Recognition, 2019, 96: 106977.

CAO B, GUO J, ZHU P, et al . Bi-directional adapter for multimodal tracking[C ] //Proceedings of the AAAI Conference on Artificial Intelligence. Vancouver: IEEE, 2024: 927-935.

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

基于Transformer的人脸深度伪造检测技术综述

基于时序对齐的风格控制语音合成算法

基于红外可见光融合的复杂环境下人脸识别方法

多套餐共享储能模式下考虑功率互济的社区联盟优化运行策略

基于AAGV-YOLOX模型的输电线路异物检测研究