Subject-Based Transfer Learning in Longitudinal Multiple Sclerosis Lesion Segmentation

Abstract

Background and Purpose

Accurate and consistent lesion segmentation from magnetic resonance imaging is required for longitudinal multiple sclerosis (MS) data analysis. In this work, we propose two new transfer learning-based pipelines to improve segmentation performance for subjects in longitudinal MS datasets.

Method

In general, transfer learning is used to improve deep learning model performance for the unseen dataset by fine-tuning a pretrained model with a limited number of labeled scans from the unseen dataset. The proposed methodologies fine-tune the deep learning model for each subject using the first scan and improve segmentation performance for later scans for the same subject. We also investigated the statistical benefits of the proposed methodology by modeling lesion volume over time between progressors according to confirmed disability progression and nonprogressors for a large in-house dataset (937 MS patients, 3210 scans) using a linear mixed effect (LME) model.

Results

The results show statistically significant improvement for the proposed methodology compared with the traditional transfer learning method using Dice (improvement: 2%), sensitivity (6%), and average volumetric difference (16%), as well as visual analysis for public and in-house datasets. The LME result showed that the proposed subject-wise transfer learning method had increased statistical power for the measurement of longitudinal lesion volume.

Conclusion

The proposed method improved lesion segmentation performance and can reduce manual effort to correct the automatic segmentations for final data analysis in longitudinal studies.