SKIE-SRL: Structured Key Information Extraction from Business Documents Using Statistical Relational Learning

Seamless automation of business processes requires the automatic analysis of complex business documents like invoices or contracts. Extracting key information in these semi-structured documents poses a significant challenge. While multimodal language models have demonstrated state-of-the-art results in this field, their application to document types with high complexity is still challenging. They neglect the underlying document structure and existing dependencies between information types. On complex document types commonly encountered in industry, this results in preventable errors such as missing predictions for mandatory elements or wrong extractions of interdependent elements. In this paper, we present SKIE-SRL, a Statistical Relational Learning model for Structured Key Information Extraction. This hybrid approach unifies symbolic reasoning with multimodal language models. It extends the capabilities of pretrained language models with expert-provided knowledge about the document structure and application domain. We evaluate our model on a multilingual invoice data set from industry and compare it to three state-of-the-art language models as well as a stacked ensemble. Our approach outperforms all benchmarks by 4% F1-score.