Towards Informed Pre-Training for Critical Error Detection in English-German

This paper presents two data augmentation methods for pre-training, to find critical errors in machine translations. This includes an alignment approach used in traditional machine translation and an imitation method, mimicking the structure of the data. Both methods are adapted to a binary classification. Our approach achieves competitive results on the WMT'21 critical error detection (CED) dataset while only using 0.06% of datapoints in comparison to the first placement.