Evaluating Segmentation of Human Body Parts Across Datasets

Human-centric visual analysis is crucial for numerous applications across different fields and has significantly progressed recently due to deep learning methods and the availability of large-scale annotated datasets. Among the relevant tasks, segmenting human body parts, also referred to as human parsing, is recognized as particularly challenging due to ambiguities resulting from occlusion and the non-rigid nature of the human body. We contribute to the field in multiple ways: First, we address the issue of dataset heterogeneity by converting three existing datasets (two based on rendering and one using real-world photographs) to be suitable for human parsing, each with different labeling schemes. By extracting two levels of label granularity - namely, 8-class and 14-class - we enable cross-dataset training and evaluation, promoting compatibility and facilitating comparative analysis. Next, we evaluate the generalization performance of state-of-the-art (SOTA) segmentation models in both in-distribution and out-of-distribution scenarios. By utilizing a cutting-edge segmentation model, we demonstrate the achievable performance on unseen data, paving the way for robust and reliable body part segmentation in real-world applications. Additionally, we present qualitative results on previously unseen, in-the-wild images. Lastly, we contribute to the research community by publishing the datasets used in this work, which include updated labels reflecting the two levels of granularity. Our work provides a useful benchmark for future algorithms aimed at solving the problem of human parsing.