Self-Supervised Contrastive Learning for Consistent Few-Shot Image Representations

The central challenge in few-shot learning involves (1) acquiring object proposals through the support representation, (2) ensuring consistent representations for images in both support and query sets, and (3) achieving effective metric learning for images spanning support and query sets. Existing literature addresses these challnges by leveraging extensive amounts of annotated support samples to capture the object proposal and employing the resulting representation as prior information for the query decoding task. However, this approach necessitates object annotation and relying solely on the support prior might not guarantee feature representation consistency among support and query samples. This paper introduces a novel approach that simultaneously tackles these challenges and fulfills the demands of support annotation through the application of self-supervised contrastive learning. Specifically, we argue that a pre-trained encoder module can generate semantically meaningful representations for the target class in both support and query sets. Hence, by explicitly modeling the mutual interaction between these representations, we capture object proposals, providing valuable prior information for query decoding. Moreover, to improve feature consistency among support and query representations, we propose to establish a prototype space by tokenizing the support and query representations and aim to match correlated tokens in this space through contrastive learning. This approach utilizes the encoder, decoder, and cross-attention to model support, query (image) representations, and metric learning for few-shot dense prediction tasks. By integrating our self-supervision strategy into the standard few-shot learning pipeline, we achieve competitive baselines without any labels, surpassing state-of-the-art methods.