Bimodal Iterative Cross-attention Fusion Ensemble Framework

Cai Zhihong; Zeng An; Pan Dan; Ye Jiayu

doi:10.12052/gdutxb.250002

English Version Journal Login

您当前的位置：

首页 >

文章列表页 >

Bimodal Iterative Cross-attention Fusion Ensemble Framework

更新时间：2026-05-27

- Bimodal Iterative Cross-attention Fusion Ensemble Framework
- Vol. 43, Issue 2, Pages: 1-11(2026)
- 作者机构：
  
  1. 广东工业大学计算机学院,广东,广州,510006
  2. 广东技术师范大学电子与信息学院,广东,广州,510665
- 作者简介：
- 基金信息：
- DOI：10.12052/gdutxb.250002
  CLC： TP391
- Published：2026
- 稿件说明：
移动端阅览
蔡志宏, 曾安, 潘丹, 等. 双模态迭代交叉注意力融合集成框架[J]. 广东工业大学学报, 2026, 43(2): 1-11.
蔡志宏, 曾安, 潘丹, 等. 双模态迭代交叉注意力融合集成框架[J]. 广东工业大学学报, 2026, 43(2): 1-11. DOI： 10.12052/gdutxb.250002.

DOI： 10.12052/gdutxb.250002

摘要

阿尔茨海默症(Alzheimer's Disease

AD) 作为一种进行性神经退行性疾病，其早期诊断与临床干预始终面临重大挑战。在医学影像领域中，结构性磁共振成像(Structural Magnetic Resonance Imaging

sMRI) 通过高分辨率解剖成像精准捕捉脑萎缩等结构性改变，而氟脱氧葡萄糖正电子发射断层扫描(Fluorodeoxyglucose Positron Emission Tomography

FDG-PET) 则通过监测葡萄糖代谢水平有效反映脑功能变化，二者在AD相关脑部病理改变的检测中具有重要互补价值。然而，现有的AD多模态分类模型存在特征融合效果欠佳、模态间信息交互不充分以及特征分布不一致的问题，限制了其在 AD 诊断中的应用。为了解决这些问题，提出了双模态迭代交叉注意力融合集成框架(Bimodal Iterative Cross-Attention Fusion Ensemble Framework，BICAFEF) 。该框架由基分类器和元分类器组成。基分类器通过残差网络(Residual Network

ResNet) 模块提取sMRI和FDG-PET切块的特征，并设计基于卷积操作和自适应聚合池化操作的空间特征收缩(Spatial Feature Shrinking

SFS) 模块减少模态间的冗余信息，突出关键特征。同时，构建迭代交叉注意力机制，在多轮迭代中动态捕捉和强化模态间的全局依赖关系和互补信息，解决无法充分挖掘模态间互补特性的难题，从而提升AD分类性能。最终，为了提升全脑分类精度，框架构建了元分类器对基分类器进行筛选和集成，剔除准确率低于 75% 的分类器，保留高性能分类器，进一步提高分类的鲁棒性和准确性。此外，通过可视化分析进一步验证了框架对关键脑区的关注，展现出该框架在sMRI和PET模态下对AD相关病变区域的有效识别能力。实验结果表明，该框架在ADvs.HC(正常组) 中的五折分类准确率(Accuracy

ACC) 为94.3%，敏感度(Sensitivity

SEN) 为92.6%，特异度(Specificity

SPE) 为96.3%，ROC曲线下面积(Area Under Curve

AUC) 为97.5%，马修斯相关系数(Matthews correlation coefficient

MCC) 为88.7%，优于现有先进的同类框架。

Abstract

Alzheimer’s disease (AD)

as a progressive neurodegenerative disorder

presents significant challenges in early diagnosis and clinical intervention. In medical imaging

structural magnetic resonance imaging (sMRI) captures brain atrophy and structural alterations through high-resolution anatomical imaging

while fluorodeoxyglucose positron emission tomography (FDG-PET) effectively reflects functional changes by monitoring cerebral glucose metabolism. These two modalities hold complementary value in detecting AD-related pathological brain changes. However

existing multimodal AD classification models are limited by suboptimal feature fusion

insufficient inter-modal information interaction

and feature distribution discrepancies

hindering their diagnostic utility. To address these issues

a bimodal iterative cross-attention fusion ensemble framework (BICAFEF) is proposed. This framework comprises base classifiers and a meta-classifier. The base classifiers employ ResNet modules to extract features from sMRI and FDG-PET image patches. A spatial feature shrinking (SFS) module

integrating convolutional operations and adaptive aggregation pooling

is designed to reduce inter-modal redundancy and emphasize discriminative features. Additionally

an iterative cross-attention mechanism is constructed to dynamically capture and reinforce global dependencies and complementary information across modalities through multi-round iterations

thereby resolving the challenge of insufficiently exploiting inter-modal synergies and enhancing AD classification performance. To further improve whole-brain classification accuracy

the framework incorporates a meta-classifier to screen and ensemble base classifiers by discarding those with accuracy below 75%

retaining high-performance classifiers to boost robustness and precision. Visualization analyses validate the framework’s focus on critical brain regions

demonstrating its capability to effectively identify AD-related pathological areas in sMRI and PET modalities. Experimental results show that the framework achieves a five-fold classification accuracy (ACC) of 94.3%

sensitivity (SEN) of 92.6%

specificity (SPE) of 96.3%

AUC of 97.5%

and Matthews correlation coefficient (MCC) of 88.7% in AD vs. healthy control (HC) classification

outperforming state-of-the-art multimodal frameworks.

关键词

Keywords

references

TREJO-LOPEZ J A, YACHNIS A T, PROKOP S. Neuropathology of Alzheimer's disease[J]. Neurotherapeutics, 2022, 19(1): 173-185.

REN R J, QI J L, LIN S H, et al . The China Alzheimer report 2022[J ] . General Psychiatry, 2022, 35(1): e100751.

BUCKNER R L. Memory and executive function in aging and AD: multiple factors that cause decline and reserve factors that compensate[J]. Neuron, 2004, 44(1): 195-208.

DUNNE R A, AARSLAND D, O’BRIEN J T, et al . Mild cognitive impairment: the Manchester consensus[J ] . Age and Ageing, 2021, 50(1): 72-80.

CHEN Q H, FU Q, BAI H, et al . Longformer: longitudinal transformer for Alzheimer's disease classification with structural MRIs[C ] // Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. Waikoloa: IEEE/CVF, 2024: 3575-3584.

XU P P, HUANG G H, ZHAO Q, et al . AFEM-Transformer: early diagnosis of Alzheimer's disease based on adaptive feature extraction with transformer[J ] . Journal of Guangdong University of Technology, 2025, 42(1): 51-59.

DUAN J W, LIU Y, WU H H, et al . Broad learning for early diagnosis of Alzheimer's disease using FDG-PET of the brain[J ] . Frontiers in Neuroscience, 2023, 17: 1137567.

黄殷. 基于卷积循环神经网络的AD多模态生物标记物发现[D]. 广州: 广东工业大学, 2021.

LIU X, LI W M, MIAO S, et al . HAMMF: hierarchical attention-based multi-task and multi-modal fusion model for computer-aided diagnosis of Alzheimer’s disease[J ] . Computers in Biology and Medicine, 2024, 176: 108564.

LIU F Y, YUAN S Z, LI W M, et al . Patch-based deep multi-modal learning framework for Alzheimer’s disease diagnosis using multi-view neuroimaging[J ] . Biomedical Signal Processing and Control, 2023, 80: 104400.

LI W M, GAO Q Q, DU M, et al . Multiclass diagnosis of stages of Alzheimer's disease using linear discriminant analysis scoring for multimodal data[J ] . Computers in Biology and Medicine, 2021, 134: 104478.

KHATRI U, KWON G R. An efficient combination among sMRI, CSF, cognitive Score, and APOE ε 4 biomarkers for classification of AD and MCI using extreme learning machine[J ] . Computational Intelligence and Neuroscience, 2020, 2020(1): 8015156.

WALHOVD K B, FJELL A M, BREWER J, et al . Combining MR imaging, positron-emission tomography, and CSF biomarkers in the diagnosis and prognosis of Alzheimer disease[J ] . American Journal of Neuroradiology, 2010, 31(2): 347-354.

KOHANNIM O, HUA X, HIBAR D P, et al . Boosting power for clinical trials using classifiers based on multiple biomarkers[J ] . Neurobiology of Aging, 2010, 31(8): 1429-1442.

ZHANG J, HE X H, LIU Y, et al . Multi-modal cross-attention network for Alzheimer’s disease diagnosis with multi-modality data[J ] . Computers in Biology and Medicine, 2023, 162: 107050.

GOLOVANEVSKY M, EICKHOFF C, SINGH R. Multimodal attention-based deep learning for Alzheimer’s disease diagnosis[J]. Journal of the American Medical Informatics Association, 2022, 29(12): 2014-2022.

SHARMA R, SIBILLE L, FAHMI R. Multi‐branch convolutional neural network for Alzheimer’s disease versus normal control classification using PET images[J]. Alzheimer's Dementia, 2023, 19: e061092.

EBRAHIMI A, LUO S H, CHIONG R. Introducing transfer learning to 3D ResNet-18 for Alzheimer’s disease detection on MRI images[C]//2020 35th International Conference on Image and Vision Computing New Zealand (IVCNZ) . Wellington: IEEE, 2020: 1-6.

SHEN J F, CHEN Y F, LIU Y, et al . ICAFusion: iterative cross-attention guided feature fusion for multispectral object detection[J ] . Pattern Recognition, 2024, 145: 109913.

HAN K, XIAO A, WU E H, et al . Transformer in transformer[J ] . Advances in Neural Information Processing Systems, 2021, 34: 15908-15919.

MUELLER S G, WEINER M W, THAL L J, et al . The Alzheimer's disease neuroimaging initiative[J ] . Neuroimaging Clinics, 2005, 15(4): 869-877.

FOLSTEIN M F, ROBINS L N, HELZER J E. The mini-mental state examination[J]. Archives of General Psychiatry, 1983, 40(7): 812-812.

BERG L. Clinical dementia rating[J]. The British Journal of Psychiatry, 1984, 145(3): 339-339.

SELVARAJU R R, COGSWELL M, DAS A, et al . Grad-cam: visual explanations from deep networks via gradient-based localization[C ] //Proceedings of the IEEE International Conference on Computer Vision. 2017: 618-626.

FAN L Z, LI H, ZHUO J J, et al . The human brainnetome atlas: a new brain atlas based on connectional architecture[J ] . Cerebral Cortex, 2016, 26(8): 3508-3526.

QIU Z F, YANG P, XIAO C L, et al . 3D multimodal fusion network with disease-induced joint learning for early Alzheimer’s disease diagnosis[J ] . IEEE Transactions on Medical Imaging, 2024, 43(9): 3161-3175.

TANG C S, XI M B, SUN J D, et al . MACFNet: detection of Alzheimer's disease via multiscale attention and cross-enhancement fusion network[J ] . Computer Methods and Programs in Biomedicine, 2024, 254: 108259.

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Point Cloud Module Based on Hidden Features

A Least Squares Twin Support Vector Machine Method with Uncertain Data

Classification Method Based on Dimension Reduction

Research on ESGS Contradiction Problem Modeling

Research on Foreign Object Detection in Power Transmission Lines Based on the AAGV-YOLOX Model

Related Author

Gao Xiang

Li Guangping

Zou Shuhang

Pan Dan

Xie Jianping

Zeng An

Liu Bo

Xiao Yan-shan

Related Institution

School of Information Engineering, Guangdong University of Technology

School of Electronics and Information Engineering, Guangdong Polytechnic Normal University

School of Automation, Guangdong University of Technology

AI问答

⁰