Face Recognition Method in Complex Environment Based on Infrared Visible Fusion

Feng Guang; Bao Long

doi:10.12052/gdutxb.230052

English Version Journal Login

您当前的位置：

首页 >

文章列表页 >

Face Recognition Method in Complex Environment Based on Infrared Visible Fusion

更新时间：2026-05-27

- Face Recognition Method in Complex Environment Based on Infrared Visible Fusion
- Vol. 41, Issue 3, Pages: 62-70, 109(2024)
- 作者机构：
  
  1. 广东工业大学自动化学院,广东,广州,510006
  2. 广东工业大学计算机学院,广东,广州,510006
- 作者简介：
- 基金信息：
- DOI：10.12052/gdutxb.230052
  CLC： TP183
- Published：2024
- 稿件说明：
移动端阅览
冯广, 鲍龙. 基于红外可见光融合的复杂环境下人脸识别方法[J]. 广东工业大学学报, 2024, 41(3): 62-70, 109.

Feng Guang, Bao Long. Face Recognition Method in Complex Environment Based on Infrared Visible Fusion[J]. 2024, 41(3): 62-70, 109.
冯广, 鲍龙. 基于红外可见光融合的复杂环境下人脸识别方法[J]. 广东工业大学学报, 2024, 41(3): 62-70, 109. DOI： 10.12052/gdutxb.230052.

Feng Guang, Bao Long. Face Recognition Method in Complex Environment Based on Infrared Visible Fusion[J]. 2024, 41(3): 62-70, 109. DOI： 10.12052/gdutxb.230052.

摘要

随着深度学习方法的发展，理想环境下基于可见光的人脸识别精度和速度已经达到优秀的水平。但是在弱光等复杂环境下，由于缺少光源，可见光图像无法体现人脸细节，导致人脸识别效果下降甚至失效。为了解决这一问题，提出一种基于红外可见光融合的复杂环境下人脸识别方法。首先，针对低照度环境提出联合CNN(Convolutional Neural Network) 和Transformer的红外与可见光融合识别网络，并联CNN和视觉Transformer组成单模态特征融合模块，充分利用源图像的局部细节信息和全局上下文信息。同时，提出一种基于模态平均差异度的多模态特征融合策略，强化对源图像不同区域特征的差异化表达。其次，针对实际应用中融合识别网络模型大、速度慢的问题提出轻量化人脸识别网络MobileFaceNet-Coo和基于边云协同的自适应识别策略，通过图像质量选择识别模型，有效利用硬件资源。实验结果表明，弱光条件下，融合红外光与仅使用可见光图像相比，识别率提升了13.96个百分点。同时，将本方法应用实际项目中，结果表明：本方法在复杂环境下，能提高人脸识别的实时性和准确率。

Abstract

With the development of deep learning methods

the accuracy and speed of face recognition based on visible light in ideal environments have reached an excellent level. However

in complex environments such as low light

the lack of a light source keeps visible images from reflecting face details

resulting in reduced or even invalid face recognition. Aiming at the problems in this issue

a face recognition method in complex environments based on infrared-visible light fusion is proposed. Firstly

an infrared and visible fusion recognition network combining CNN and Transformer is introduced

specifically designed for low illumination environments. This network combines CNN and visual Transformer in parallel to form a single-mode feature fusion module

which is utilized to effectively utilize local details and global context information from the source image. Additionally

a multimodal feature fusion strategy based on the average difference of modes is proposed to enhance the distinctive expression of different regional features in the source image. Secondly

a lightweight face recognition network MobileFaceNet-Coo and an adaptive recognition strategy based on edge-cloud collaboration are proposed in order to solve the problem of large and slow fusion recognition network models in practical applications. This strategy selects the recognition model through image quality and effectively utilizes hardware resources. Experimental results demonstrate that the recognition rate of fused infrared light is 13.96 percentage point higher than that of visible light alone. Real-world project result shows that this method significantly improves real-time and accuracy of face recognition in complex environments.

关键词

Keywords

references

SINGH R, AHMED T, SINGH R, et al . Identifying tiny faces in thermal images using transfer learning [J ] . Journal of Ambient Intelligence and Humanized Computing, 2020, 11(5): 1957-1966.

马娜. 基于高光谱图像的人脸识别算法和实验研究[D]. 济南: 山东师范大学, 2017.

GUO T, ZHAG T X, GUO S J. A face recognition algorithm based on infrared image and wide spectrum fusion [J]. Journal of Wuhan Engineering University, 2022, 44(3): 320-324.

XIE Z H, LI Y, NIU J Y. Hyperspectral face recognition combining block spectral band optimization and depth feature [J]. Chinese Journal of Image Graphics, 2021, 26(12): 2870-2878.

LI H, WU X J. DenseFuse: a fusion approach to infrared and visible images [J]. IEEE Transactions on Image Processing, 2018, 28(5): 2614-2623.

MA J, YU W, LIANG P, et al . FusionGAN: a generative adversarial network for infrared and visible image fusion [J ] . Information Fusion, 2019, 48: 11-26.

PRABHAKAR K R, SRIKAR V S, BABU R V. Deepfuse: a deep unsupervised approach for exposure fusion with extreme exposure image pairs[C]//Proceedings of the IEEE International Conference on Computer Vision. Piscataway: IEEE Press, 2017: 4714-4722.

KIM M, JAIN A K, LIU X. AdaFace: quality adaptive margin for face recognition[C] //Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2022: 18750-18759.

ZHANG K, ZHANG Z, LI Z, et al . Joint face detection and alignment using multitask cascaded convolutional networks [J ] . IEEE Signal Processing Letters, 2016, 23(10): 1499-1503.

HOU Q, ZHOU D, FENG J. Coordinate attention for efficient mobile network design[C] //Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2021: 13713-13722.

HUANG G, LIU Z, VAN Der M L , et al. 2017. Densely connected convolutional networks[J]. IEEE Computer Society, 2017: 2261-2269.

ZAMIR S W, ARORA A, KHAN S , et al. Restormer: efficient transformer for high-resolution image restoration[C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway: IEEE Press, 2022: 5718-5729.

DAI C . Attentional feature fusion[C]// Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. Piscataway: IEEE Press, 2021: 3560-3569.

WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module[C]// European Conference on Computer Vision(ECCV) . Munich: Springer, 2018: 3-19.

YI D, LEI Z, LIAO S, et al . Learning face representation from scratch [J ] . Computer Science, 2014, 29(1): 51-59.

STAN L, DONG Y, ZHEN L, et al. The casia nir-vis 2.0 face database[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. Piscataway: IEEE Press, 2013: 348-353.

MOSCHOGLOU S, PAPAIOANNOU A, SAGONAS C, et al. Agedb: the first manually collected, in-the-wild age database[C]//Proceedings of the IEEE Conference on Compute Vision and Pattern Recognition Workshops. [S.l.]: IEEE, 2017: 51-59.

TANG L, YUAN J, MA J. Image fusion in the loop of high-level vision tasks: a semantic-aware real-time infrared and visible image fusion network [J]. Information Fusion, 2022, 82: 28-42.

MA J, ZHANG H, SHAO Z, et al. GANMcC: a generative adversarial network with multiclassification constraints for infrared and visible image fusion[J]. IEEE Transactions on Instrumentation and Measurement, 2020, 99: 1.

PANETTA, KAREN, QIANWEN W, et al. A comprehensive database for benchmarking imaging systems[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42(3): 509-520.

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Efficient Temporal Modeling for RGB-T Tracking

A Survey of Deepfake Detection Techniques Based on Transformer

Temporal Alignment Style Control in Text-to-Speech Synthesis Algorithm

An Unpaired Face Illumination Normalization Method Based on CycleGAN

Research and Implementation of a Pulsed Laser Ranging Method Based on Digital Mixing

Related Author

Zhang Zhongxuan

Zeng Bi

Ni Xinyu

Liu Zhenyu

Luo Chaofan

Li Dong

Zhang Yun

Lai Zhi-mao

Related Institution

School of Information Engineering, Guangdong University of Technology

School of Immigration Administration (Guangzhou) , China People’s Police University

College of Science, Shantou University

School of Computers, Guangdong University of Technology

AI问答

⁰