Automated Adequacy Assessment of Cervical Cytology Samples Using Deep Learning

Cervical cancer has been among the most common causes of cancer death in women. Screening tests such as liquid-based cytology (LBC) were responsible for a substantial decrease in mortality rates. Still, visual examination of cervical cells on microscopic slides is a time-consuming, ambiguous and challenging task, aggravated by inadequate sample quality (e.g. low cellularity or the presence of obscuring factors like blood or inflammation). While most works in the literature are focused on the automated detection of cervical lesions to support diagnosis, to the best of our knowledge, none of them address the automated assessment of sample adequacy, as established by The Bethesda System (TBS) guidelines. This work proposes a new methodology for automated adequacy assessment of cervical cytology samples. Since the most common reason for rejecting samples is the low count of the squamous nucleus, our approach relies on a deep learning object detection model for the detection and counting of different types of nuclei present in LBC samples. A dataset of 41 samples with a total of 42387 nuclei manually annotated by experienced specialists was used, and the best solution proposed achieved promising results for the automated detection of squamous nuclei (AP of 82.4%, Accuracy of 79.8%, Recall of 73.8% and F1 score of 81.5%). Additionally, by merging the developed automated cell counting approach with the adequacy criteria stated by the TBS guidelines, we validated our approach by correctly classifying an entire subset of 12 samples as adequate or inadequate.