Investigation of Out-of-Distribution Detection Using Contrastive Learning

Out-of-distribution (OOD) detectors are required to identify the presence of test data points that bear little association to any of the train dataclasses or are semantically dissimilar to the training data. They are crucial for safety-critical applications like medical services or self-driving cars, where errors lead to fatalities. State-of-the-art detectors capture high-level representations of the in-distribution data using deep learning, then define a detection scheme based on the learned features. We employ Contrastive Learning as the representation learning framework and investigate its application for OOD detection. Contrastive Learning (CL) is a self-supervised method that is competitive with fully-supervised methods on classification tasks. The best CL methods solve the instance-discrimination task that involves training representations of image view pairs (generated by data augmentations) to be close to one another and far from other image representations in a latent space. We utilize a contrastive training framework called Prototypical Contrastive Learning (PCL), which improves on the instance discrimination objective, by incorporating semantic information into the learned embeddings using prototypes. Using CIFAR10 and CIFAR100 as in-distribution data sets, we detect near and far OOD embeddings under a distance-metric-based protocol with metrics: the Mahalanobis, and the Cosine Similarity. The analysis proceeds under a semi-supervised setting by fine-tuning the contrastively trained Res-Net50 encoder on 10% of the in-distribution labeled data set. The fine-tuned model is distilled into a network with a smaller-sized ResNet18 backbone. We analyze the test Softmax confidence scores of both classification models to identify OOD data points. The proposed OOD detection methods achieve comparable to baselines and recommendations are made for deployment in real-world settings.