Facial Expression Recognition Based on Multi-feature Fusion

Lai Dong-sheng; Feng Kai-ping; Luo Li-hong

doi:10.12052/gdutxb.210179

English Version Journal Login

您当前的位置：

首页 >

文章列表页 >

Facial Expression Recognition Based on Multi-feature Fusion

更新时间：2026-05-27

- Facial Expression Recognition Based on Multi-feature Fusion
- Vol. 40, Issue 3, Pages: 10-16(2023)
- 作者机构：
  
  1. 广东工业大学计算机学院,广东,广州,510006
  2. 广东工业大学艺术与设计学院,广东,广州,510090
- 作者简介：
- 基金信息：
- DOI：10.12052/gdutxb.210179
  CLC： TP391
- Online First：25 May 2023，
  
  Published：2023
- 稿件说明：
移动端阅览
赖东升, 冯开平, 罗立宏. 基于多特征融合的表情识别算法[J]. 广东工业大学学报, 2023, 40(3): 10-16.

Lai Dong-sheng, Feng Kai-ping, Luo Li-hong. Facial Expression Recognition Based on Multi-feature Fusion[J]. 2023, 40(3): 10-16.
赖东升, 冯开平, 罗立宏. 基于多特征融合的表情识别算法[J]. 广东工业大学学报, 2023, 40(3): 10-16. DOI： 10.12052/gdutxb.210179.

Lai Dong-sheng, Feng Kai-ping, Luo Li-hong. Facial Expression Recognition Based on Multi-feature Fusion[J]. 2023, 40(3): 10-16. DOI： 10.12052/gdutxb.210179.

摘要

针对轻量级面部表情识别算法泛化能力的不足，提出了一种结合多特征融合和注意力机制的表情识别方法。使用局部二值模式(Local Binary Pattern

LBP)算子减少面部图像中无关信息的干扰，双分支神经网络提取原始人脸图像和LBP图像的特征，融合两个网络提取的中高层特征，并通过注意力机制加强重要特征，在保持较少参数量的同时生成大量的有效特征信息提高算法的识别效果。实验结果表明，该方法在Fer2013和CK+数据集上的识别率分别为70.21%和95.59%，有效地提高了轻量级表情识别算法的性能。

Abstract

For the insufficient generalization ability of lightweight facial expression recognition algorithms

a facial expression recognition method based on multi-feature fusion and attention mechanism is proposed. LBP is used to reduce the interference of meaningless information of original face image

the dual-branch neural network extracts the features of original face image and LBP image respectively

merges the middle-level and high-level feature information extracted by these two networks and strengthens important features through attention mechanism for facial expression recognition

which can generate a large amount of discriminative features by multi-feature fusion to improve the recognition performance with a small number of parameters. The experimental results show that the recognition rate of this method has reached 70.21% on Fer2013 dataset and 95.59% on CK+ dataset respectively

which effectively improves the performance of lightweight expression recognition algorithm.

关键词

Keywords

references

SHI D, TANG H. Face recognition algorithm based on self-adaptive blocking local binary pattern [J]. Multimedia Tools and Applications, 2021, 80: 23899-23921.

YAO A, CAI D, HU P, et al. HoloNet: towards robust emotion recognition in the wild[C]// Proceedings of the 18th ACM International Conference on Multimodal Interaction. Tokyo: Association for Computing Machinary, 2016: 472-478.

DING H, ZHOU P, CHELLAPPA R. Occlusion-adaptive deep network for robust facial expression recognition[C]// IEEE International Joint Conference on Biometrics. Houston: IEEE, 2020: 1-9.

FAN Y, LAM J, LI V. Multi-ensemble convolutional neural network for facial expression recognition[C]// International Conference on Artificial Neural Networks. Rhodes: Springer International Publishing, 2018: 84-94.

WANG K, PENG X, YANG J, et al . Region attention networks for pose and occlusion robust facial expression recognition [J ] . IEEE Transactions on Image Processing, 2020, 29: 4057-4069.

WANG K, CHEN J, ZHANG X, et al. Facial expression recognition based on deep convolutional neural network[C]// IEEE 8th Annual International Conference on CYBER Technology in Automation, Control, and Intelligent Systems. Tianjin: IEEE, 2018: 629-634.

MA Y, WANG X, WEI L. Multi-level spatial and semantic enhancement network for expression recognition [J]. Applied Intelligence, 2021, 51: 1-14.

LI J, JIN K, ZHOU D, et al . Attention mechanism-based CNN for facial expression recognition [J ] . Neurocomputing, 2020, 411: 340-350.

HU J, SHEN L, SUN G. Squeeze-and-excitation networks[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 7132-7141.

CHEN S, LIU Y, GAO X, et al. Mobile Face Nets: efficient cnns for accurate real-time face verification on mobile devices[C]// Biometric Recognition. Urumqi: Springer International Publishing, 2018: 428-438.

XU L L, ZHANG S M, ZHAO J L. Expression recognition algorithm for parallel convolutional neural networks [J]. Journal of Image and Graphics, 2019, 24(2): 227-236.

MOLLAHOSSEINI A, CHAN D, MAHOOR M H. Going deeper in facial expression recognition using deep neural networks[C]// 2016 IEEE Winter Conference on Applications of Computer Vision. Lake Placid: IEEE, 2016: 1-10.

JEON J, PARK J C, JO Y J, et al. A real-time facial expression recognizer using deep neural network[C]// Proceedings of the 10th International Conference on Ubiquitous Information Management and Communication. Danang: Association for Computing Machinery, 2016: 1-4.

SANG D V, DAT N V, THUAN D P. Facial expression recognition using deep convolutional neural networks[C]// 9th International Conference on Knowledge and Systems Engineering. Hue: IEEE, 2017: 130-135.

MA N N, ZHANG X Y, ZHENG H T, et al. ShuffleNet V2: practical guidelines for efficient cnn architecture design[C]// Proceedings of the European Conference on Computer Vision. Munich: Springer International Publishing, 2018: 116-131.

HE Z C, ZHAO L Z, CHEN C. Convolution neural network with multi-resolution feature fusion for facial expression recognition [J]. Laser Optoelectronics Progress, 2018, 55(7): 370-375.

QIAN Z, MU J, ZHANG J, et al. Facial expression recognition based on sas-net attention mechanism[C]// International Conference on Computer Network, Electronic and Automation. Xian: IEEE, 2021: 159-163.

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Unsupervised GAN Multi-focus Image Fusion Based on Attention Mechanism

Point Cloud Module Based on Hidden Features

A Multi-Portrait Semantic Segmentation Method Based on Semantic Fusion Features

A Resource Allocation Optimization Method for Multi-UAV Integrated Sensing, Computing, and Communication Towards Disaster Relief

An Optimal Operation Strategy for Community Microgrid Alliances Considering Power Exchange under Multi-package Shared Energy Storage Model

Related Author

Li Weitong

Jiang Yunfeng

Tan Zhe

Huang Guoheng

Chen Jiahong

Gao Xiang

Li Guangping

Zou Shuhang

Related Institution

School of Information Engineering, Guangdong University of Technology

School of Automation, Guangdong University of Technology

School of Electronics and Information Engineering, Guangdong Polytechnic Normal University

School of Electronics and Information, Guangdong Polytechnic Normal University

School of Integrated Circuits, Guangdong University of Technology

AI问答

⁰