Domain Adaptation for Semantic Segmentation Using Convolutional Neural Networks

Semantic segmentation is an important analysis task for the investigation of aerial imagery. Recently, the arise of convolutional neural networks has increased the performance of computer vision methods considerably. But the success of deep learning applications mostly relies on the availability of sufficiently large training datasets. However, the manual annotation of images is time consuming and needs human effort. To reduce the necessary amount of training data it is possible to fine-tune a model which is pre-trained on a different larger dataset. But usually orthophotos are affected by weather and sensor dependent light conditions. Additionally, such images are composed of imbalanced classes which leads to poor pixel-wise classification results for sparsely represented labels. In this paper we propose a convolutional neural network based domain adaptation method for semantic segmentation. The encoder-decoder structure uses adaptation modules and an alternately training procedure to adapt the network to the target domain. We employ the large ISPRS Potsdam dataset as source domain to train a base model and adapt it using very few samples. We compared our method to the common fine-tuning approach and evaluated the results for a decreasing number of training samples. We observed an improvement of the average overall prediction accuracy but especially for the sparsely represented vehicle class.