基于语义融合特征的多人像语义分割方法

冯广; 汤翀

doi:10.12052/gdutxb.230211

欢迎来到能源电力期刊网！

English Version 期刊登录

您当前的位置：

首页 >

文章列表页 >

基于语义融合特征的多人像语义分割方法

计算机科学与技术 | 更新时间：2026-05-27

- 基于语义融合特征的多人像语义分割方法
- A Multi-Portrait Semantic Segmentation Method Based on Semantic Fusion Features
- 广东工业大学学报 2025年42卷第2期页码：20-28
- 作者机构：
  
  1. 广东工业大学自动化学院,广东,广州,510006
  2. 广东工业大学计算机学院,广东,广州,510006
- 作者简介：
  
  冯广(1973–) ，男，教授，博士，主要研究方向为深度学习、网路控制，E-mail：von@gdut.edu.cn
  汤翀(1998–) ，男，硕士研究生，主要研究方向为图像分割，E-mail：1412562728@qq.com
- 基金信息：
  
  国家自然科学基金资助项目(62237001)
- DOI：10.12052/gdutxb.230211
  中图分类号： TP391
- 纸质出版：2025
- 稿件说明：
移动端阅览
冯广, 汤翀. 基于语义融合特征的多人像语义分割方法[J]. 广东工业大学学报, 2025, 42(2): 20–28.

Feng Guang, Tang Chong. A Multi-Portrait Semantic Segmentation Method Based on Semantic Fusion Features[J]. 2025, 42(2): 20-28.
冯广, 汤翀. 基于语义融合特征的多人像语义分割方法[J]. 广东工业大学学报, 2025, 42(2): 20–28. DOI： 10.12052/gdutxb.230211

Feng Guang, Tang Chong. A Multi-Portrait Semantic Segmentation Method Based on Semantic Fusion Features[J]. 2025, 42(2): 20-28. DOI： 10.12052/gdutxb.230211.

摘要

人像语义分割是计算机视觉领域的重要研究内容之一，但现有的人像语义分割方法容易忽略多人像图像中的小尺寸人像。同时分割结果中容易出现多个人像之间相互粘连的现象。再者，图像中人像之间存在相互遮挡现象容易导致人像边缘分割精度不佳。基于以上问题，本文提出一种融合标签语义的多人像语义分割方法，对图像中的多个人像分配多个标签，并将语义标签嵌入同时作为编码器的输入，使用跨模态交叉注意力模块对语义标签和图像特征表示进行相关性建模，将语义融合的特征表示作为模型每一层编码器的输出。提出HRF attention模块，基于目标检测算法对图像生成的多个假设分别进行特征提取。将该网络在 Supervisely增强数据集上训练测试，实验结果表明该算法模型在3个评估指标PA、MIoU、Dice上分别达到95.94%、94.60%、96.02%的精度，较语义分割模型U-net、PSPNet、Deeplab v3+、PortraitNet、Swin Unet具有更高的分割精度。

Abstract

Portrait semantic segmentation is one of the important research contents in the field of computer vision

but the existing portrait semantic segmentation methods are liable to ignore the small size portraits in multi-person portrait images. At the same time

the segmentation results are prone to the phenomenon of mutual adhesion between multiple portraits. Moreover

the phenomenon of mutual occlusion between portraits in the image easily leads to poor segmentation accuracy of portrait edges. Based on the above problems

a semantic segmentation method for multiple portraits with fused label semantics is propose

where multiple labels are assigned to multiple portraits in an image

and semantic labels are embedded as inputs to the encoder at the same time

and the semantic labels and the image feature representations are correlated using the cross-modal cross-attention module

and the semantically fused feature representations are obtained as outputs of the encoder at each layer of the model. The HRF attention module is proposed to generate multiple hypotheses for image based on target detection algorithm for feature extraction separately. The network is trained and tested on Supervisely augmented dataset. The experimental results show that the algorithmic model achieves 95.94%

94.60%

and 96.02% accuracy on the three evaluation metrics of PA

MIoU

and Dice

respectively

and has higher segmentation accuracy than the semantic segmentation models U-net

PSPNet

Deeplab v3+

PortraitNet

and Swin Unet.

关键词

Keywords

references

王欣. 基于深度学习的人像分割方法研究[D]. 咸阳: 西北农业科技大学, 2022.

KUAI Y, WANG B, WU Y L, et al . Urban vegetation classification based on multi-scale feature perception network for UAV images[J ] . Journal of Geo-Information Science, 2022, 24(5): 962-980.

RONNEBERGER O, FISCHER P, BROX T. U-net: convolutional networks for biomedical image segmentation[C] //Medical Image Computing and Computer-assisted Intervention–MICCAI 2015: 18th International Conference. Munich: Springer International Publishing, 2015: 234-241.

WEI Y, XIA W, LIN M, et al . HCP: a flexible CNN framework for multi-label image classification[J ] . IEEE, 2015, 38(9): 1901-1907.

AZAD R, ASADI-AGHBOLAGHI M, FATHY M, et al . Attention DeepLabv3+: multi-level context attention mechanism for skin lesion segmentation[C ] //European Conference on Computer Vision (ECCV) . Glasgow: Springer International Publishing, 2020: 251-266.

ZHAO W P, CHEN Y, XIANG S, et al . Research on image s emantic segmentation algorithm based on improved DeepLabv3+[J ] . Journal of System Simulation, 2023, 35(11): 2333-2344.

YANG Y, WAN W, HUANG S, et al . RADCU-Net: residual attention and dual-supervision cascaded U-Net for retinal blood vessel segmentation[J ] . International Journal of Machine Learning and Cybernetics, 2023, 14: 1605-1620.

FU J, LIU J, TIAN H, et al . Dual attention network for scene segmentation[C ] //Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019: 3146-3154.

AMER A, LAMBROU T, YE X. MDA-unet: a multi-scale dilated attention U-net for medical image segmentation[J]. Applied Sciences, 2022, 12(7): 3676.

雷俊婷. 基于多尺度特征增强的多标签图像分类研究[D]. 西安: 西北大学, 2023.

ZHANG S J, PENG Z, LI H. SAU-Net: medical image seg- mentation method based on U-Net and self-attention[J]. Acta Elect Ronica Sinica, 2022, 50(10): 2433-2442.

WU W, LIU H, LI L, et al . Application of local fully convolutional neural network combined with YOLO v5 algorithm in small target detection of remote sensing image[J ] . PloS One, 2021, 16(10): e0259283.

CHEN L C, ZHU Y, PAPANDREOU G, et al . Encoder-decoder with atrous separable convolution for semantic image segmentation[C ] //Proceedings of the European Conference on Computer Vision (ECCV) . Munich: Springer International Publishing, 2018: 801-818.

ZHANG Y, TIAN Y, KONG Y, et al . Residual dense network for image super-resolution[C ] //Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Calgary: IEEE, 2018: 2472-2481.

DU M M, SIMA H F. A-LinkNet: semantic segmentation net-work based on attention and spatial information fusion[J]. Chinese Journal of Liquid Crystals and Displays, 2022, 37(9): 1199-1208.

WANG X Y, WANG C R, ZHANG J F, et al . Multimodal sentiment analysis based on cross-modal cross-attention network [J ] . Journal of Guangxi Normal University (Natural Science Edition) , 2024, 42(2) : 84-930.

SUN B, JIANG T, JIA L, et al . Multimodal sentiment analysis based on cross-modal joint-encoding[J ] . Computer Engineering and Applications, 2024, 60(18): 208-216.

CHEN Q, GE T, XU Y, et al . Semantic human matting[C ] //Proceedings of the 26th ACM International Conference on Multi-media. New York: ACM, 2018: 618-626.

HUANG Y J, SHI Z F, WANG Z Q, et al . Improved U-net based on mixed loss function for liver medical image segmentation[J ] . Laser Optoelectronics Progress, 2020, 57(22): 74-83.

ZHAO H, SHI J, QI X, et al . Pyramid scene parsing network[C ] //Proceedings of the IEEE Conference On Computer Vision and Pattern Recognition. Kyoto: IEEE, 2017: 2881-2890.

CHEN L, ZHU Y, PAPANDREOU G, et al . Encoder-decoder with atrous separable convolution for semantic image segmentation[C ] //Proceedings of the European Conference on Computer Vision (ECCV) . Munich: Springer International Publishing, 2018: 801-818.

ZHANG S H, DONG X, LI H, et al . PortraitNet: realtime portrait segmentation network for mobile device[J ] . Computers Graphics, 2019, 80: 104-113.

CAO H, WANG Y, CHEN J, et al . Swin-unet: unet-like pure transformer for medical image segmentation[C ] //European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2022: 205-218.

FENG G, PAN T F, WU W Y. An online learning behavior analysis based on bayesian network model[J]. Journal of Guangdong University of Technology, 2022, 39(3): 41-48.

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

基于注意力机制的无监督GAN多聚焦图像融合

基于隐藏特征的点云模块

多套餐共享储能模式下考虑功率互济的社区联盟优化运行策略

基于路况参数的混合动力机车能量管理策略研究

双模态迭代交叉注意力融合集成框架