Labeling system for labeling a medical image

The invention relates to labeling of medical images using a modular model. The modules based on which the model can be generated include a) a shared first module (10) for different types of labeling (40, 41, 42), which is configured to have as input a medical image and to have as output a medical image representation (13), b) a labeling-type-specific first module (11, 12), which is specific for a type of labeling and is configured to have as input the representation provided by the shared first module and as output a labeling-type-specific representation (14, 15), and c) second modules (20, 21, 22) of which each is specific for a respective type of labeling and is configured to have as input a medical image representation and as output a labeling of the respective type of labeling. This modular structure allows for an efficient multi-task training and hence for more accurate labelings.