ACVF
Multimodal fusion of brain MRI and clinical language
ICASSP 2027 · Submitted
Connect brain imaging, anatomy and clinical language through lightweight alignment with frozen backbones.
- My role
- First author
- Methods & tools
- PyTorch · 3D MRI · BioClinicalBERT · SigLIP
01
Problem & challenge
MRI representations lack explicit anatomical and clinical context. Fusion must also exclude task-specific label-related fields.
02
Key contributions
My contribution
Independently designed and implemented fusion of MRI, AAL regional volumes and BioClinicalBERT text, using subject-level SigLIP alignment with frozen backbones and cross-backbone ablations.
Method
Encode MRI, clinical text and AAL regional volumes while keeping the MRI and BioClinicalBERT backbones frozen.
Project features into a shared 256-dimensional space and align the three modalities with a subject-level SigLIP objective.
Exclude task-specific label-related fields and evaluate across backbones and ablations on 1,077 ADNI subjects.
03
Experimental results
For AD classification excluding cognitive-score inputs, average AUC improves by 0.144 across three backbones. BrainIAC accuracy rises from 64.6% to 84.3% and AUC from 0.732 to 0.922. AnatCL brain-age MAE decreases from 4.924 to 4.373 years.
Research in progress; code and manuscript are not public.