A Multi-Portrait Semantic Segmentation Method Based on Semantic Fusion Features

Feng Guang; Tang Chong

doi:10.12052/gdutxb.230211

English Version Journal Login

您当前的位置：

首页 >

文章列表页 >

A Multi-Portrait Semantic Segmentation Method Based on Semantic Fusion Features

更新时间：2026-05-27

- A Multi-Portrait Semantic Segmentation Method Based on Semantic Fusion Features
- Vol. 42, Issue 2, Pages: 20-28(2025)
- 作者机构：
  
  1. 广东工业大学自动化学院,广东,广州,510006
  2. 广东工业大学计算机学院,广东,广州,510006
- 作者简介：
- 基金信息：
- DOI：10.12052/gdutxb.230211
  CLC： TP391
- Published：2025
- 稿件说明：
移动端阅览
冯广, 汤翀. 基于语义融合特征的多人像语义分割方法[J]. 广东工业大学学报, 2025, 42(2): 20–28.

Feng Guang, Tang Chong. A Multi-Portrait Semantic Segmentation Method Based on Semantic Fusion Features[J]. 2025, 42(2): 20-28.
冯广, 汤翀. 基于语义融合特征的多人像语义分割方法[J]. 广东工业大学学报, 2025, 42(2): 20–28. DOI： 10.12052/gdutxb.230211

Feng Guang, Tang Chong. A Multi-Portrait Semantic Segmentation Method Based on Semantic Fusion Features[J]. 2025, 42(2): 20-28. DOI： 10.12052/gdutxb.230211.

摘要

人像语义分割是计算机视觉领域的重要研究内容之一，但现有的人像语义分割方法容易忽略多人像图像中的小尺寸人像。同时分割结果中容易出现多个人像之间相互粘连的现象。再者，图像中人像之间存在相互遮挡现象容易导致人像边缘分割精度不佳。基于以上问题，本文提出一种融合标签语义的多人像语义分割方法，对图像中的多个人像分配多个标签，并将语义标签嵌入同时作为编码器的输入，使用跨模态交叉注意力模块对语义标签和图像特征表示进行相关性建模，将语义融合的特征表示作为模型每一层编码器的输出。提出HRF attention模块，基于目标检测算法对图像生成的多个假设分别进行特征提取。将该网络在 Supervisely增强数据集上训练测试，实验结果表明该算法模型在3个评估指标PA、MIoU、Dice上分别达到95.94%、94.60%、96.02%的精度，较语义分割模型U-net、PSPNet、Deeplab v3+、PortraitNet、Swin Unet具有更高的分割精度。

Abstract

Portrait semantic segmentation is one of the important research contents in the field of computer vision

but the existing portrait semantic segmentation methods are liable to ignore the small size portraits in multi-person portrait images. At the same time

the segmentation results are prone to the phenomenon of mutual adhesion between multiple portraits. Moreover

the phenomenon of mutual occlusion between portraits in the image easily leads to poor segmentation accuracy of portrait edges. Based on the above problems

a semantic segmentation method for multiple portraits with fused label semantics is propose

where multiple labels are assigned to multiple portraits in an image

and semantic labels are embedded as inputs to the encoder at the same time

and the semantic labels and the image feature representations are correlated using the cross-modal cross-attention module

and the semantically fused feature representations are obtained as outputs of the encoder at each layer of the model. The HRF attention module is proposed to generate multiple hypotheses for image based on target detection algorithm for feature extraction separately. The network is trained and tested on Supervisely augmented dataset. The experimental results show that the algorithmic model achieves 95.94%

94.60%

and 96.02% accuracy on the three evaluation metrics of PA

MIoU

and Dice

respectively

and has higher segmentation accuracy than the semantic segmentation models U-net

PSPNet

Deeplab v3+

PortraitNet

and Swin Unet.

关键词

Keywords

references

王欣. 基于深度学习的人像分割方法研究[D]. 咸阳: 西北农业科技大学, 2022.

KUAI Y, WANG B, WU Y L, et al . Urban vegetation classification based on multi-scale feature perception network for UAV images[J ] . Journal of Geo-Information Science, 2022, 24(5): 962-980.

RONNEBERGER O, FISCHER P, BROX T. U-net: convolutional networks for biomedical image segmentation[C] //Medical Image Computing and Computer-assisted Intervention–MICCAI 2015: 18th International Conference. Munich: Springer International Publishing, 2015: 234-241.

WEI Y, XIA W, LIN M, et al . HCP: a flexible CNN framework for multi-label image classification[J ] . IEEE, 2015, 38(9): 1901-1907.

AZAD R, ASADI-AGHBOLAGHI M, FATHY M, et al . Attention DeepLabv3+: multi-level context attention mechanism for skin lesion segmentation[C ] //European Conference on Computer Vision (ECCV) . Glasgow: Springer International Publishing, 2020: 251-266.

ZHAO W P, CHEN Y, XIANG S, et al . Research on image s emantic segmentation algorithm based on improved DeepLabv3+[J ] . Journal of System Simulation, 2023, 35(11): 2333-2344.

YANG Y, WAN W, HUANG S, et al . RADCU-Net: residual attention and dual-supervision cascaded U-Net for retinal blood vessel segmentation[J ] . International Journal of Machine Learning and Cybernetics, 2023, 14: 1605-1620.

FU J, LIU J, TIAN H, et al . Dual attention network for scene segmentation[C ] //Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019: 3146-3154.

AMER A, LAMBROU T, YE X. MDA-unet: a multi-scale dilated attention U-net for medical image segmentation[J]. Applied Sciences, 2022, 12(7): 3676.

雷俊婷. 基于多尺度特征增强的多标签图像分类研究[D]. 西安: 西北大学, 2023.

ZHANG S J, PENG Z, LI H. SAU-Net: medical image seg- mentation method based on U-Net and self-attention[J]. Acta Elect Ronica Sinica, 2022, 50(10): 2433-2442.

WU W, LIU H, LI L, et al . Application of local fully convolutional neural network combined with YOLO v5 algorithm in small target detection of remote sensing image[J ] . PloS One, 2021, 16(10): e0259283.

CHEN L C, ZHU Y, PAPANDREOU G, et al . Encoder-decoder with atrous separable convolution for semantic image segmentation[C ] //Proceedings of the European Conference on Computer Vision (ECCV) . Munich: Springer International Publishing, 2018: 801-818.

ZHANG Y, TIAN Y, KONG Y, et al . Residual dense network for image super-resolution[C ] //Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Calgary: IEEE, 2018: 2472-2481.

DU M M, SIMA H F. A-LinkNet: semantic segmentation net-work based on attention and spatial information fusion[J]. Chinese Journal of Liquid Crystals and Displays, 2022, 37(9): 1199-1208.

WANG X Y, WANG C R, ZHANG J F, et al . Multimodal sentiment analysis based on cross-modal cross-attention network [J ] . Journal of Guangxi Normal University (Natural Science Edition) , 2024, 42(2) : 84-930.

SUN B, JIANG T, JIA L, et al . Multimodal sentiment analysis based on cross-modal joint-encoding[J ] . Computer Engineering and Applications, 2024, 60(18): 208-216.

CHEN Q, GE T, XU Y, et al . Semantic human matting[C ] //Proceedings of the 26th ACM International Conference on Multi-media. New York: ACM, 2018: 618-626.

HUANG Y J, SHI Z F, WANG Z Q, et al . Improved U-net based on mixed loss function for liver medical image segmentation[J ] . Laser Optoelectronics Progress, 2020, 57(22): 74-83.

ZHAO H, SHI J, QI X, et al . Pyramid scene parsing network[C ] //Proceedings of the IEEE Conference On Computer Vision and Pattern Recognition. Kyoto: IEEE, 2017: 2881-2890.

CHEN L, ZHU Y, PAPANDREOU G, et al . Encoder-decoder with atrous separable convolution for semantic image segmentation[C ] //Proceedings of the European Conference on Computer Vision (ECCV) . Munich: Springer International Publishing, 2018: 801-818.

ZHANG S H, DONG X, LI H, et al . PortraitNet: realtime portrait segmentation network for mobile device[J ] . Computers Graphics, 2019, 80: 104-113.

CAO H, WANG Y, CHEN J, et al . Swin-unet: unet-like pure transformer for medical image segmentation[C ] //European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2022: 205-218.

FENG G, PAN T F, WU W Y. An online learning behavior analysis based on bayesian network model[J]. Journal of Guangdong University of Technology, 2022, 39(3): 41-48.

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Unsupervised GAN Multi-focus Image Fusion Based on Attention Mechanism

Point Cloud Module Based on Hidden Features

Ground Deformation Monitoring and Influencing Factors Analysis in Fuzhou City Based on PSI Technology

Energy Management Strategy of Hybrid Locomotive Based on Road Condition Parameters

Robust Design for Artificial Noise-assisted UAV Terahertz Secure Communication

Related Author

Li Weitong

Jiang Yunfeng

Tan Zhe

Huang Guoheng

Chen Jiahong

Zou Shuhang

Li Guangping

Gao Xiang

Related Institution

School of Information Engineering, Guangdong University of Technology

School of Electronics and Information Engineering, Guangdong Polytechnic Normal University

School of Electronics and Information, Guangdong Polytechnic Normal University

Department of Cardiovascular Surgery, Guangdong Provincial People’s Hospital (Guangdong Academy of Medical Sciences)

School of Electronics and Information Technology, Guangdong Technical Normal University

AI问答

⁰