基于红外可见光融合的复杂环境下人脸识别方法

冯广; 鲍龙

doi:10.12052/gdutxb.230052

欢迎来到能源电力期刊网！

English Version 期刊登录

您当前的位置：

首页 >

文章列表页 >

基于红外可见光融合的复杂环境下人脸识别方法

计算机科学与技术 | 更新时间：2026-05-27

- 基于红外可见光融合的复杂环境下人脸识别方法
- Face Recognition Method in Complex Environment Based on Infrared Visible Fusion
- 广东工业大学学报 2024年41卷第3期页码：62-70, 109
- 作者机构：
  
  1. 广东工业大学自动化学院,广东,广州,510006
  2. 广东工业大学计算机学院,广东,广州,510006
- 作者简介：
  
  冯广(1973–) ，男，教授级高级实验师，博士，主要研究方向为网络控制、机器学习、大数据， E-mail：von@gdut.edu.cn
- 基金信息：
  
  国家自然科学基金资助项目(62237001);广东省哲学社会科学项目(GD23YJY08)
- DOI：10.12052/gdutxb.230052
  中图分类号： TP183
- 纸质出版：2024
- 稿件说明：
移动端阅览
冯广, 鲍龙. 基于红外可见光融合的复杂环境下人脸识别方法[J]. 广东工业大学学报, 2024, 41(3): 62-70, 109.

Feng Guang, Bao Long. Face Recognition Method in Complex Environment Based on Infrared Visible Fusion[J]. 2024, 41(3): 62-70, 109.
冯广, 鲍龙. 基于红外可见光融合的复杂环境下人脸识别方法[J]. 广东工业大学学报, 2024, 41(3): 62-70, 109. DOI： 10.12052/gdutxb.230052.

Feng Guang, Bao Long. Face Recognition Method in Complex Environment Based on Infrared Visible Fusion[J]. 2024, 41(3): 62-70, 109. DOI： 10.12052/gdutxb.230052.

摘要

随着深度学习方法的发展，理想环境下基于可见光的人脸识别精度和速度已经达到优秀的水平。但是在弱光等复杂环境下，由于缺少光源，可见光图像无法体现人脸细节，导致人脸识别效果下降甚至失效。为了解决这一问题，提出一种基于红外可见光融合的复杂环境下人脸识别方法。首先，针对低照度环境提出联合CNN(Convolutional Neural Network) 和Transformer的红外与可见光融合识别网络，并联CNN和视觉Transformer组成单模态特征融合模块，充分利用源图像的局部细节信息和全局上下文信息。同时，提出一种基于模态平均差异度的多模态特征融合策略，强化对源图像不同区域特征的差异化表达。其次，针对实际应用中融合识别网络模型大、速度慢的问题提出轻量化人脸识别网络MobileFaceNet-Coo和基于边云协同的自适应识别策略，通过图像质量选择识别模型，有效利用硬件资源。实验结果表明，弱光条件下，融合红外光与仅使用可见光图像相比，识别率提升了13.96个百分点。同时，将本方法应用实际项目中，结果表明：本方法在复杂环境下，能提高人脸识别的实时性和准确率。

Abstract

With the development of deep learning methods

the accuracy and speed of face recognition based on visible light in ideal environments have reached an excellent level. However

in complex environments such as low light

the lack of a light source keeps visible images from reflecting face details

resulting in reduced or even invalid face recognition. Aiming at the problems in this issue

a face recognition method in complex environments based on infrared-visible light fusion is proposed. Firstly

an infrared and visible fusion recognition network combining CNN and Transformer is introduced

specifically designed for low illumination environments. This network combines CNN and visual Transformer in parallel to form a single-mode feature fusion module

which is utilized to effectively utilize local details and global context information from the source image. Additionally

a multimodal feature fusion strategy based on the average difference of modes is proposed to enhance the distinctive expression of different regional features in the source image. Secondly

a lightweight face recognition network MobileFaceNet-Coo and an adaptive recognition strategy based on edge-cloud collaboration are proposed in order to solve the problem of large and slow fusion recognition network models in practical applications. This strategy selects the recognition model through image quality and effectively utilizes hardware resources. Experimental results demonstrate that the recognition rate of fused infrared light is 13.96 percentage point higher than that of visible light alone. Real-world project result shows that this method significantly improves real-time and accuracy of face recognition in complex environments.

关键词

Keywords

references

SINGH R, AHMED T, SINGH R, et al . Identifying tiny faces in thermal images using transfer learning [J ] . Journal of Ambient Intelligence and Humanized Computing, 2020, 11(5): 1957-1966.

马娜. 基于高光谱图像的人脸识别算法和实验研究[D]. 济南: 山东师范大学, 2017.

GUO T, ZHAG T X, GUO S J. A face recognition algorithm based on infrared image and wide spectrum fusion [J]. Journal of Wuhan Engineering University, 2022, 44(3): 320-324.

XIE Z H, LI Y, NIU J Y. Hyperspectral face recognition combining block spectral band optimization and depth feature [J]. Chinese Journal of Image Graphics, 2021, 26(12): 2870-2878.

LI H, WU X J. DenseFuse: a fusion approach to infrared and visible images [J]. IEEE Transactions on Image Processing, 2018, 28(5): 2614-2623.

MA J, YU W, LIANG P, et al . FusionGAN: a generative adversarial network for infrared and visible image fusion [J ] . Information Fusion, 2019, 48: 11-26.

PRABHAKAR K R, SRIKAR V S, BABU R V. Deepfuse: a deep unsupervised approach for exposure fusion with extreme exposure image pairs[C]//Proceedings of the IEEE International Conference on Computer Vision. Piscataway: IEEE Press, 2017: 4714-4722.

KIM M, JAIN A K, LIU X. AdaFace: quality adaptive margin for face recognition[C] //Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2022: 18750-18759.

ZHANG K, ZHANG Z, LI Z, et al . Joint face detection and alignment using multitask cascaded convolutional networks [J ] . IEEE Signal Processing Letters, 2016, 23(10): 1499-1503.

HOU Q, ZHOU D, FENG J. Coordinate attention for efficient mobile network design[C] //Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2021: 13713-13722.

HUANG G, LIU Z, VAN Der M L , et al. 2017. Densely connected convolutional networks[J]. IEEE Computer Society, 2017: 2261-2269.

ZAMIR S W, ARORA A, KHAN S , et al. Restormer: efficient transformer for high-resolution image restoration[C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway: IEEE Press, 2022: 5718-5729.

DAI C . Attentional feature fusion[C]// Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. Piscataway: IEEE Press, 2021: 3560-3569.

WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module[C]// European Conference on Computer Vision(ECCV) . Munich: Springer, 2018: 3-19.

YI D, LEI Z, LIAO S, et al . Learning face representation from scratch [J ] . Computer Science, 2014, 29(1): 51-59.

STAN L, DONG Y, ZHEN L, et al. The casia nir-vis 2.0 face database[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. Piscataway: IEEE Press, 2013: 348-353.

MOSCHOGLOU S, PAPAIOANNOU A, SAGONAS C, et al. Agedb: the first manually collected, in-the-wild age database[C]//Proceedings of the IEEE Conference on Compute Vision and Pattern Recognition Workshops. [S.l.]: IEEE, 2017: 51-59.

TANG L, YUAN J, MA J. Image fusion in the loop of high-level vision tasks: a semantic-aware real-time infrared and visible image fusion network [J]. Information Fusion, 2022, 82: 28-42.

MA J, ZHANG H, SHAO Z, et al. GANMcC: a generative adversarial network with multiclassification constraints for infrared and visible image fusion[J]. IEEE Transactions on Instrumentation and Measurement, 2020, 99: 1.

PANETTA, KAREN, QIANWEN W, et al. A comprehensive database for benchmarking imaging systems[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42(3): 509-520.

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

基于高效时序建模的RGB-T跟踪

基于Transformer的人脸深度伪造检测技术综述

基于时序对齐的风格控制语音合成算法

基于CycleGAN的非配对人脸图片光照归一化方法

多套餐共享储能模式下考虑功率互济的社区联盟优化运行策略