Machine learning-based combination of the central vein sign, cortical lesions and paramagnetic rim lesions: a web-based tool for the diagnosis of multiple sclerosis

Abstract Multiple sclerosis diagnostic criteria lack optimal specificity, leading to potential misdiagnosis. Advanced magnetic resonance imaging (MRI) biomarkers like the central vein sign, cortical lesions and paramagnetic rim lesions are highly specific to multiple sclerosis and could potentially improve diagnostic accuracy. In this study, we applied machine learning techniques to a retrospective, multicentric dataset of 322 multiple sclerosis/multiple sclerosis-mimic (204/118) and 84 prodromal multiple sclerosis/non-multiple sclerosis (43/41) adult patients, incorporating the central vein sign, cortical lesions and paramagnetic rim lesions. We compared (5 × 2 cross-validation combined F-test) the diagnostic performance of 71 machine learning models, each corresponding to a distinct combination of full-count or simplified biomarker inputs, against the baseline dissemination in space McDonald criteria. The aim was to evaluate the multiple sclerosis diagnostic power of combining these biomarkers in an MRI-only diagnostic framework. 51 of the 71 models significantly outperformed the dissemination in space criterion (P < 0.05), with balanced accuracy improvements up to 13.0% (confidence interval: [+10.5; +17.0]). The best overall model (random forest, using full-count assessments) achieved 95.7% (confidence interval: [93.2; 99.7]) balanced accuracy; the best simplified model (logistic regression, using only simplified assessments) reached 94.7% with no significant difference with the former (P = 0.29). Notably, 12/51 high-performing models used only simplified assessments. To further investigate the models’ generalizability, external validation on two out-of-distribution test sets using bootstrapping (1000 resamples) confirmed these results and highlighted a more robust generalization for the best model using solely simplified biomarkers. On the first external test set (n = 37, Verona), the simplified model achieved 97.2% balanced accuracy, while the full-count model reached 93.3% (versus 83.3% for baseline). On the second test set (n = 84, prodromal cases), the simplified model achieved 92.6% (versus 60.1% for baseline) showing competitive performance against the full-count model (93.9%). Both models improved all key performance metrics—balanced accuracy, sensitivity, specificity, precision and F1 score—over the baseline on both test sets (all P < 0.0001). Within a non-invasive MRI-only diagnostic framework, these results show that the incorporation of advanced imaging biomarkers into the multiple sclerosis-MRI diagnostic criteria significantly enhances the diagnostic accuracy—a statement holding true even when using simplified central vein sign, cortical lesions and paramagnetic rim lesions assessments. The study also provides a publicly available online diagnostic tool, facilitating further interaction, validation and clinical support (https://www.msdiagnostictool.org).