A Shape-Model based Loss for End-to-End Training of Neural Networks

Traditional models for performing segmentation tasks on medical image datasets mostly rely on Convolutional Neural Networks (CNN). These models are trained with pixel-wise loss functions that aim for maximizing pixel accuracy of predicted segmentation maps. However, these losses do not have any knowledge about the global contour or unique shape characteristics. Despite the existence of special shape models that can capture global shape properties of certain objects, they are only used as pre- or post-processing steps but never included in CNN models themselves. At this point, our work comes into action. This thesis presents a novel method for segmenting medical images by incorporating an Active Shape Model (ASM) into the loss function of CNNs in order to add shape-knowledge to the training process. Therefore, a shape-constraining pipeline is introduced that takes initial estimates and generates shapes similar to ones from the training set. This pipeline is integrated into the loss function as an additional differentiable loss term that allows to jointly train shape-aware neural network models in an end-to-end fashion. Multiple versions of shape-loss functions are explored on a variety of datasets, and their strengths and weaknesses are discussed. Alongside this, a fully functional C++ implementation for training shape-loss CNN models is provided. The decision for C++ was made because of an already available ASM implementation in C++ and the lack of any usable Python version. In this work, TensorFlow's C++ API is utilized, which is, in contrast to the widespread Python API, rarely applied and lacks proper documentation or any further resources. Hence, great efforts went into elaborating its correct usage for implementing custom losses as well as building and training custom networks. Additionally, a detailed guide on how to use this API to build TensorFlow models is provided.