Alzheimer's Disease Prediction via Brain Structural-Functional Deep Fusing Network (2309.16206v2)

Abstract: Fusing structural-functional images of the brain has shown great potential to analyze the deterioration of Alzheimer's disease (AD). However, it is a big challenge to effectively fuse the correlated and complementary information from multimodal neuroimages. In this paper, a novel model termed cross-modal transformer generative adversarial network (CT-GAN) is proposed to effectively fuse the functional and structural information contained in functional magnetic resonance imaging (fMRI) and diffusion tensor imaging (DTI). The CT-GAN can learn topological features and generate multimodal connectivity from multimodal imaging data in an efficient end-to-end manner. Moreover, the swapping bi-attention mechanism is designed to gradually align common features and effectively enhance the complementary features between modalities. By analyzing the generated connectivity features, the proposed model can identify AD-related brain connections. Evaluations on the public ADNI dataset show that the proposed CT-GAN can dramatically improve prediction performance and detect AD-related brain regions effectively. The proposed model also provides new insights for detecting AD-related abnormal neural circuits.

• The paper presents CT-GAN, which fuses fMRI and DTI data to generate multimodal brain connectivity for improved AD prediction.
• It employs a swapping bi-attention mechanism and dual-channel separator to effectively align and decompose functional and structural embeddings.
• The model achieves superior classification metrics on the ADNI dataset, offering significant potential for enhanced clinical diagnosis of Alzheimer’s.

Alzheimer's Disease Prediction via Brain Structural-Functional Deep Fusing Network

The paper presents the Cross-Modal Transformer Generative Adversarial Network (CT-GAN), a novel approach for predicting Alzheimer's disease (AD) by effectively fusing functional and structural data from fMRI and DTI. CT-GAN excels in learning topological connections and generating multimodal connectivity in an efficient end-to-end manner, contributing significant advancements in AD prediction.

Methodological Advancements

Cross-Modal Transformer Generative Adversarial Network (CT-GAN)

CT-GAN employs a comprehensive framework designed to transform fMRI and DTI data into multimodal connectivity matrices, facilitating AD analysis.

Figure 1: The framework of the proposed CT-GAN, including four parts: the cross-modal transformer generator, the dual-channel separator, the SFC discriminator, and the predictor. S represents the structural embedding, and F represents the functional embedding. The framework aims to generate multimodal connectivity from DTI and fMRI.

1. Cross-Modal Transformer Generator: Incorporates a swapping bi-attention mechanism that aligns functional and structural embeddings (derived from medical imaging) to enhance the complementarities between modalities.
2. Dual-Channel Separator: Functions to decompose the generated multimodal connectivity into functional and structural connectivity, preserving the high-dimensional topological properties.

   Figure 2: The network architecture of the dual-channel separator. Given the multimodal connectivity, it outputs the structural connectivity and functional connectivity.

3. Structural-Functional Consistency Discriminator: With two sub-discriminators, it validates the consistency of the generated connectivities against empirical data, enhancing the model's resilience.

   Figure 3: The network architecture of one sub-discriminator in the SFC discriminator.

Swapping Bi-Attention Mechanism

The swapping bi-attention mechanism capitalizes on transformers' strength in attending to non-linear relationships between modalities, allowing the exchange of structural and functional tokens which aids in improving alignment and model performance.

Loss Functions

Three core loss functions ensure the model performs optimally:

• Adversarial Loss: Encourages the outputs of CT-GAN to mimic empirical SC/FC distributions.
• Classification Loss: Used to fine-tune the generator by minimizing prediction errors.
• Pair-Wise Connectivity Reconstruction Loss: Further refines modeling by ensuring topological similarities with empirical data.

Experimental Results

Extensive evaluation was conducted on the ADNI dataset incorporating various stages of cognitive decline: NC, EMCI, LMCI, and AD.

Figure 4: Examples of two multimodal connectivity matrices at different stages of cognitive disease (a) NC; (b) AD.

• Superior Classification Performance: CT-GAN demonstrated superior accuracy (ACC), sensitivity (SEN), specificity (SPE), and AUC in several classification tasks when compared to existing multimodal fusion models.
• Identifying AD-Related Changes: The generated multimodal connectivity provided insights into changes in brain connectivity patterns across different stages. Identified changes in ROIs demonstrated clinical relevance, overlapping with existing literature on AD.

  Figure 5: The ten most important brain regions between NC and EMCI groups.

Additional analyses explored connectivity alterations during disease progression, identifying both increased and decreased network connectivities critical to understanding AD pathology.

Connectivity Analysis

Figure 6: The altered connectivity between two MC groups. The first and third columns are the decreased connectivity matrices, with the threshold values set at 50\% and 75\% respectively. The second and fourth columns are the increased connectivity matrices with the threshold values at 50\% and 75\% respectively.

Extensive evaluation of connectivity alterations revealed distinct patterns correlating with disease stages, highlighting critical brain regions and their changing connectivities, providing predictive markers useful in clinical settings.

Conclusion

CT-GAN effectively generates and utilizes multimodal connectivity for improved AD prediction, demonstrating substantial advances over traditional fusion methods. As a robust model, CT-GAN not only improves diagnostic precision but offers novel insights into the neural alterations associated with Alzheimer's disease. Future work involves extending this framework to explore other neurodegenerative conditions.

The research underscores the potential of deep learning models in enhancing diagnostic tools and providing a deeper understanding of complex brain disorders, supporting ongoing endeavors in neuroscience and clinical applications.