Improved identification of check-worthiness in social media data through multimodal analyses

Combatting the spread of non-intentional and intentional false information on social media is challenging due to the vast amount of data that is shared each day. In order to still be able to retrieve credible information, assessing the check-worthiness of social media content can help to identify content that requires manual review. In this paper, we present a novel approach for detecting the check-worthiness in tweets. By incorporating the analysis of image content that is frequently shared along with social media posts, the proposed method, which consists of an analysis of the content, caption, and text obtained from optical character recognition, can outperform the current state-of-the-art recognition techniques with an F1 score of 0.7658 on the CheckThat! Lab 2023 benchmark dataset. Further experiments show, that by leveraging from multimodal information where applicable, the detection rate can be further improved.