Loading…
Domain-aware multi-task pretrained 3D Swin Transformer for T1-weighted brain MRI, transferring to Alzheimer's, Parkinson's and brain age tasks.
No providers recorded yet. Browse all providers
Rotate a photograph and ask a network which way is up, and you have posed a real question. Rotate a skull-stripped T1-weighted brain volume already registered to a standard template and you have posed a much thinner one: the pretext tasks that made self-supervised learning work on natural images carry less signal once the domain has stripped out the nuisance variation they were invented to exploit. DAMT (Domain Aware Multi-Task pretraining) responds with three pretext tasks supervised by neuroanatomy rather than by image manipulation — for each input patch, predict its 120-region Desikan parcellation, the cortical thickness and curvature of the regions it contains, and radiomics texture features of the white matter, gray matter and cerebrospinal fluid inside it.
Those three objectives train jointly with four conventional ones — patch-location classification, 3D rotation classification, SimMIM-style masked image modeling, and contrastive coding over paired global and local views — so a single 3D Swin Transformer encoder learns anatomy, morphology, texture and generic spatial context in one pass. The work is from Jonghun Kim, Mansu Kim and Hyunjin Park at Sungkyunkwan University, the Center for Neuroscience Imaging Research at the Institute for Basic Science, and the Gwangju Institute of Science and Technology, published at ACCV 2024.
DAMT sits in the same family as other volumetric brain-MRI backbones such as BrainSegFounder, BrainIAC and 3D-Neuro-SimCLR, which pretrain on unlabeled scans with contrastive or masked objectives. What distinguishes DAMT is that its parcellation, morphometry and tissue-radiomics targets are cheap to derive from any T1 volume with standard processing, and the ablation shows they add accuracy on top of masked image modeling rather than duplicating it.
The backbone is a 3D Swin Transformer implemented in PyTorch and MONAI on top of the Swin UNETR encoder. Pretraining ran for 300 epochs with AdamW at an initial learning rate of 0.0005, linear warmup and cosine annealing, on four A100 80GB GPUs, over 13,687 T1-weighted scans pooled from ADNI, HCP, IXI, ABIDE, DOD-ADNI, ICBM and A4. The seven losses are equally weighted except the parcellation Dice term, which carries a weight of 0.2.
Downstream evaluation used five-fold cross-validation on cohorts held out from pretraining. For Alzheimer's disease versus cognitively normal, DAMT reaches 0.9462 accuracy and 0.9623 AUC on ADNI, against 0.9227 and 0.9204 for the same Swin Transformer trained from scratch, and generalizes to unseen cohorts at 0.9372 accuracy on AIBL and 0.8809 on OASIS. On the harder adjacent contrasts it reaches 0.7721 for AD versus MCI and 0.7037 for MCI versus CN. Parkinson's disease versus control on PPMI reaches 0.7586 accuracy and 0.6782 AUC, and age prediction on ADNI gives an MAE of 3.9138 years with an R² of 0.7886. Against 3D extensions of established self-supervised frameworks on ADNI AD/CN, DAMT's 0.9462 compares with 0.9309 for SimMIM, 0.9267 for DINO, 0.9246 for MoCo v2 and 0.9215 for BYOL.
The intended use is as a starting checkpoint for structural-MRI studies where labeled scans are scarce: neurodegenerative disease classification, staging along the CN–MCI–AD spectrum, brain-age estimation and other regression targets derived from T1 anatomy. Because the pretraining corpus spans seven public databases and generalization held on AIBL and OASIS, the encoder is a reasonable initialization for smaller single-site cohorts. GradCAM++ and M3d-CAM maps on the AD task localize to the hippocampus, corpus callosum, precuneus, prefrontal cortex and temporal lobe, giving a clinical audience something to interrogate.
DAMT demonstrates that domain knowledge which is cheap to compute — atlas parcellations, cortical morphometry, tissue radiomics — is worth folding into self-supervised pretraining for medical volumes rather than importing 2D pretext tasks wholesale, and its ablation separating the domain-aware, self-supervised and contrastive groups is the part most useful to other groups building brain-imaging backbones. The limits are practical: the training code is released under Apache 2.0, but no pretrained weights have been published, so using DAMT means re-running pretraining, and each of the seven source databases requires its own data-access approval. Scope is narrower than a general medical-imaging foundation model — pretraining is on T1-weighted structural brain MRI only, not multi-sequence or whole-body imaging.
Much of this page is generated or calculated automatically. Flag anything that looks off and we will re-run it.