CC BY 4.0Deußer, TobiasTobiasDeußerPielka, MarenMarenPielkaPucknat, LisaLisaPucknatJacob, BasilBasilJacobDilmaghani, TimTimDilmaghaniNourimand, MahdisMahdisNourimandKliem, BerndBerndKliemLoitz, RüdigerRüdigerLoitzBauckhage, ChristianChristianBauckhageSifa, RafetRafetSifa2023-03-102023-03-102023-01-23https://publica.fraunhofer.de/handle/publica/437547https://doi.org/10.24406/publica-102910.7557/18.679910.24406/publica-1029Finding and amending contradictions in a financial report is crucial for the publishing company and its financial auditors. To automate this process, we introduce a novel approach that incorporates informed pre-training into its transformer-based architecture to infuse this model with additional Part-Of-Speech knowledge. Furthermore, we fine-tune the model on the public Stanford Natural Language Inference Corpus and our proprietary financial contradiction dataset. It achieves an exceptional contradiction detection F1 score of 89.55% on our real-world financial contradiction dataset, beating our several baselines by a considerable margin. During the model selection process we also test various financial-document-specific transformer models and find that they underperform the more general embedding approaches.encontradiction detectionnatural language processingtext miningfinancial reportsdeep learningDDC::000 Informatik, Informationswissenschaft, allgemeine Werkeconference paper