Table Structure Recognition via Encoder/Decoder Vision Transformers

Table structure recognition (TSR), the task of inferring the layout of tables, including the row, column, and cell structure, is a surprisingly complex task. With the growing amount and importance of digital documents, it has become an increasingly relevant problem, which nonetheless has not yet been solved adequately and still presents a very active area of research. In recent years, a growing number of deep-learning-based approaches to table parsing have been proposed.
This paper presents a novel deep-learning-based table structure recognition method that can predict row, column, and cell bounds for table images with a high degree of accuracy. To achieve this goal, a multi-stage pipeline incorporating a Vision-Transformer-based Autoencoder model was devised. This model was trained to predict cell regions for table images, from which accurate cell bounds can be inferred, including spanning cells which cover multiple rows or columns. The goal was to obtain a model that generalizes well and can return accurate predictions on various tables of differing complexity, even if they contain little initial structural information.
An additional modification to the model architecture presented in the Masked Autoencoder (MAE) approach was also evaluated.