双模态迭代交叉注意力融合集成框架

蔡志宏; 曾安; 潘丹; 叶嘉宇

doi:10.12052/gdutxb.250002

欢迎来到能源电力期刊网！

English Version 期刊登录

您当前的位置：

首页 >

文章列表页 >

双模态迭代交叉注意力融合集成框架

计算机科学与技术 | 更新时间：2026-05-27

- 双模态迭代交叉注意力融合集成框架
- Bimodal Iterative Cross-attention Fusion Ensemble Framework
- 广东工业大学学报 2026年43卷第2期页码：1-11
- 作者机构：
  
  1. 广东工业大学计算机学院,广东,广州,510006
  2. 广东技术师范大学电子与信息学院,广东,广州,510665
- 作者简介：
  
  蔡志宏(1999–) ，男，硕士研究生，主要研究方向为深度学习、图像处理，E-mail：2112205101@mail2.gdut.edu.cn
  曾安(1978–) ，女，教授，博士生导师，主要研究方向为图像处理、模式识别、人工智能，E-mail：zengan@gdut.edu.cn
- 基金信息：
  
  国家自然科学基金资助项目(61976058);广州市科技计划项目(202103000034，202206010007，202002020090);广东省科技计划项目(2021A1515012300，2019A050510041，2021B0101220006)
- DOI：10.12052/gdutxb.250002
  中图分类号： TP391
- 纸质出版：2026
- 稿件说明：
移动端阅览
蔡志宏, 曾安, 潘丹, 等. 双模态迭代交叉注意力融合集成框架[J]. 广东工业大学学报, 2026, 43(2): 1-11.
蔡志宏, 曾安, 潘丹, 等. 双模态迭代交叉注意力融合集成框架[J]. 广东工业大学学报, 2026, 43(2): 1-11. DOI： 10.12052/gdutxb.250002.

DOI： 10.12052/gdutxb.250002

摘要

阿尔茨海默症(Alzheimer's Disease

AD) 作为一种进行性神经退行性疾病，其早期诊断与临床干预始终面临重大挑战。在医学影像领域中，结构性磁共振成像(Structural Magnetic Resonance Imaging

sMRI) 通过高分辨率解剖成像精准捕捉脑萎缩等结构性改变，而氟脱氧葡萄糖正电子发射断层扫描(Fluorodeoxyglucose Positron Emission Tomography

FDG-PET) 则通过监测葡萄糖代谢水平有效反映脑功能变化，二者在AD相关脑部病理改变的检测中具有重要互补价值。然而，现有的AD多模态分类模型存在特征融合效果欠佳、模态间信息交互不充分以及特征分布不一致的问题，限制了其在 AD 诊断中的应用。为了解决这些问题，提出了双模态迭代交叉注意力融合集成框架(Bimodal Iterative Cross-Attention Fusion Ensemble Framework，BICAFEF) 。该框架由基分类器和元分类器组成。基分类器通过残差网络(Residual Network

ResNet) 模块提取sMRI和FDG-PET切块的特征，并设计基于卷积操作和自适应聚合池化操作的空间特征收缩(Spatial Feature Shrinking

SFS) 模块减少模态间的冗余信息，突出关键特征。同时，构建迭代交叉注意力机制，在多轮迭代中动态捕捉和强化模态间的全局依赖关系和互补信息，解决无法充分挖掘模态间互补特性的难题，从而提升AD分类性能。最终，为了提升全脑分类精度，框架构建了元分类器对基分类器进行筛选和集成，剔除准确率低于 75% 的分类器，保留高性能分类器，进一步提高分类的鲁棒性和准确性。此外，通过可视化分析进一步验证了框架对关键脑区的关注，展现出该框架在sMRI和PET模态下对AD相关病变区域的有效识别能力。实验结果表明，该框架在ADvs.HC(正常组) 中的五折分类准确率(Accuracy

ACC) 为94.3%，敏感度(Sensitivity

SEN) 为92.6%，特异度(Specificity

SPE) 为96.3%，ROC曲线下面积(Area Under Curve

AUC) 为97.5%，马修斯相关系数(Matthews correlation coefficient

MCC) 为88.7%，优于现有先进的同类框架。

Abstract

Alzheimer’s disease (AD)

as a progressive neurodegenerative disorder

presents significant challenges in early diagnosis and clinical intervention. In medical imaging

structural magnetic resonance imaging (sMRI) captures brain atrophy and structural alterations through high-resolution anatomical imaging

while fluorodeoxyglucose positron emission tomography (FDG-PET) effectively reflects functional changes by monitoring cerebral glucose metabolism. These two modalities hold complementary value in detecting AD-related pathological brain changes. However

existing multimodal AD classification models are limited by suboptimal feature fusion

insufficient inter-modal information interaction

and feature distribution discrepancies

hindering their diagnostic utility. To address these issues

a bimodal iterative cross-attention fusion ensemble framework (BICAFEF) is proposed. This framework comprises base classifiers and a meta-classifier. The base classifiers employ ResNet modules to extract features from sMRI and FDG-PET image patches. A spatial feature shrinking (SFS) module

integrating convolutional operations and adaptive aggregation pooling

is designed to reduce inter-modal redundancy and emphasize discriminative features. Additionally

an iterative cross-attention mechanism is constructed to dynamically capture and reinforce global dependencies and complementary information across modalities through multi-round iterations

thereby resolving the challenge of insufficiently exploiting inter-modal synergies and enhancing AD classification performance. To further improve whole-brain classification accuracy

the framework incorporates a meta-classifier to screen and ensemble base classifiers by discarding those with accuracy below 75%

retaining high-performance classifiers to boost robustness and precision. Visualization analyses validate the framework’s focus on critical brain regions

demonstrating its capability to effectively identify AD-related pathological areas in sMRI and PET modalities. Experimental results show that the framework achieves a five-fold classification accuracy (ACC) of 94.3%

sensitivity (SEN) of 92.6%

specificity (SPE) of 96.3%

AUC of 97.5%

and Matthews correlation coefficient (MCC) of 88.7% in AD vs. healthy control (HC) classification

outperforming state-of-the-art multimodal frameworks.

关键词

Keywords

references

TREJO-LOPEZ J A, YACHNIS A T, PROKOP S. Neuropathology of Alzheimer's disease[J]. Neurotherapeutics, 2022, 19(1): 173-185.

REN R J, QI J L, LIN S H, et al . The China Alzheimer report 2022[J ] . General Psychiatry, 2022, 35(1): e100751.

BUCKNER R L. Memory and executive function in aging and AD: multiple factors that cause decline and reserve factors that compensate[J]. Neuron, 2004, 44(1): 195-208.

DUNNE R A, AARSLAND D, O’BRIEN J T, et al . Mild cognitive impairment: the Manchester consensus[J ] . Age and Ageing, 2021, 50(1): 72-80.

CHEN Q H, FU Q, BAI H, et al . Longformer: longitudinal transformer for Alzheimer's disease classification with structural MRIs[C ] // Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. Waikoloa: IEEE/CVF, 2024: 3575-3584.

XU P P, HUANG G H, ZHAO Q, et al . AFEM-Transformer: early diagnosis of Alzheimer's disease based on adaptive feature extraction with transformer[J ] . Journal of Guangdong University of Technology, 2025, 42(1): 51-59.

DUAN J W, LIU Y, WU H H, et al . Broad learning for early diagnosis of Alzheimer's disease using FDG-PET of the brain[J ] . Frontiers in Neuroscience, 2023, 17: 1137567.

黄殷. 基于卷积循环神经网络的AD多模态生物标记物发现[D]. 广州: 广东工业大学, 2021.

LIU X, LI W M, MIAO S, et al . HAMMF: hierarchical attention-based multi-task and multi-modal fusion model for computer-aided diagnosis of Alzheimer’s disease[J ] . Computers in Biology and Medicine, 2024, 176: 108564.

LIU F Y, YUAN S Z, LI W M, et al . Patch-based deep multi-modal learning framework for Alzheimer’s disease diagnosis using multi-view neuroimaging[J ] . Biomedical Signal Processing and Control, 2023, 80: 104400.

LI W M, GAO Q Q, DU M, et al . Multiclass diagnosis of stages of Alzheimer's disease using linear discriminant analysis scoring for multimodal data[J ] . Computers in Biology and Medicine, 2021, 134: 104478.

KHATRI U, KWON G R. An efficient combination among sMRI, CSF, cognitive Score, and APOE ε 4 biomarkers for classification of AD and MCI using extreme learning machine[J ] . Computational Intelligence and Neuroscience, 2020, 2020(1): 8015156.

WALHOVD K B, FJELL A M, BREWER J, et al . Combining MR imaging, positron-emission tomography, and CSF biomarkers in the diagnosis and prognosis of Alzheimer disease[J ] . American Journal of Neuroradiology, 2010, 31(2): 347-354.

KOHANNIM O, HUA X, HIBAR D P, et al . Boosting power for clinical trials using classifiers based on multiple biomarkers[J ] . Neurobiology of Aging, 2010, 31(8): 1429-1442.

ZHANG J, HE X H, LIU Y, et al . Multi-modal cross-attention network for Alzheimer’s disease diagnosis with multi-modality data[J ] . Computers in Biology and Medicine, 2023, 162: 107050.

GOLOVANEVSKY M, EICKHOFF C, SINGH R. Multimodal attention-based deep learning for Alzheimer’s disease diagnosis[J]. Journal of the American Medical Informatics Association, 2022, 29(12): 2014-2022.

SHARMA R, SIBILLE L, FAHMI R. Multi‐branch convolutional neural network for Alzheimer’s disease versus normal control classification using PET images[J]. Alzheimer's Dementia, 2023, 19: e061092.

EBRAHIMI A, LUO S H, CHIONG R. Introducing transfer learning to 3D ResNet-18 for Alzheimer’s disease detection on MRI images[C]//2020 35th International Conference on Image and Vision Computing New Zealand (IVCNZ) . Wellington: IEEE, 2020: 1-6.

SHEN J F, CHEN Y F, LIU Y, et al . ICAFusion: iterative cross-attention guided feature fusion for multispectral object detection[J ] . Pattern Recognition, 2024, 145: 109913.

HAN K, XIAO A, WU E H, et al . Transformer in transformer[J ] . Advances in Neural Information Processing Systems, 2021, 34: 15908-15919.

MUELLER S G, WEINER M W, THAL L J, et al . The Alzheimer's disease neuroimaging initiative[J ] . Neuroimaging Clinics, 2005, 15(4): 869-877.

FOLSTEIN M F, ROBINS L N, HELZER J E. The mini-mental state examination[J]. Archives of General Psychiatry, 1983, 40(7): 812-812.

BERG L. Clinical dementia rating[J]. The British Journal of Psychiatry, 1984, 145(3): 339-339.

SELVARAJU R R, COGSWELL M, DAS A, et al . Grad-cam: visual explanations from deep networks via gradient-based localization[C ] //Proceedings of the IEEE International Conference on Computer Vision. 2017: 618-626.

FAN L Z, LI H, ZHUO J J, et al . The human brainnetome atlas: a new brain atlas based on connectional architecture[J ] . Cerebral Cortex, 2016, 26(8): 3508-3526.

QIU Z F, YANG P, XIAO C L, et al . 3D multimodal fusion network with disease-induced joint learning for early Alzheimer’s disease diagnosis[J ] . IEEE Transactions on Medical Imaging, 2024, 43(9): 3161-3175.

TANG C S, XI M B, SUN J D, et al . MACFNet: detection of Alzheimer's disease via multiscale attention and cross-enhancement fusion network[J ] . Computer Methods and Programs in Biomedicine, 2024, 254: 108259.

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

基于隐藏特征的点云模块

基于最小二乘孪生支持向量机的不确定数据学习算法

基于正交投影的降维分类方法研究

基于分类的ESGS矛盾问题建模的研究

人工噪声辅助无人机太赫兹安全通信的鲁棒设计