Infratentorial lesions in multiple sclerosis patients: intra- and inter-rater variability in comparison to a fully automated segmentation using 3D convolutional neural networks

Objective. Automated quantification of infratentorial multiple sclerosis lesions on magnetic resonance imaging is clinically relevant but challenging. To overcome some of these problems, we propose a fully automated lesion segmentation algorithm using 3D convolutional neural networks (CNNs). Methods. The CNN was trained on a FLAIR image alone or on FLAIR and T1-weighted images from 1809 patients acquired on 156 different scanners. An additional training using an extra class for infratentorial lesions was implemented. Three experienced raters manually annotated three datasets from 123 MS patients from different scanners. Results. The inter-rater sensitivity (SEN) was 80% for supratentorial lesions but only 62% for infratentorial lesions. There was no statistically significant difference between the inter-rater SEN and the SEN of the CNN with respect to the raters. For supratentorial lesions, the CNN featured an intra-rater intra-scanner SEN of 0.97 (R1 = 0.90, R2 = 0.84) and for infratentorial lesion a SEN of 0.93 (R1 = 0.61, R2 = 0.73). Conclusion. The performance of the CNN improved significantly for infratentorial lesions when specifically trained on infratentorial lesions using a T1 image as an additional input and matches the detection performance of experienced raters. Furthermore, for infratentorial lesions the CNN was more robust against repeated scans than experienced raters.