Your Input Matters - Comparing Real-Valued PolSAR Data Representations for CNN-Based Segmentation

Inspired by the success of Convolutional Neural Network (CNN)-based deep learning methods for optical image segmentation, there is a growing interest in applying these methods to Polarimetric Synthetic Aperture Radar (PolSAR) data. However, effectively utilizing well-established real-valued CNNs for PolSAR image segmentation requires converting complex-valued data into real-valued representations. This paper presents a systematic comparison of 14 different real-valued representations used as CNN input in the literature. These representations encompass various approaches, including the use of coherency matrix elements, hand-crafted feature vectors, polarimetric features based on target decomposition, and combinations of these methods. The goal is to assess the impact of the choice of PolSAR data representation on segmentation performance and identify the most suitable representation. Four test configurations are employed to achieve this, involving different CNN architectures (U-Net with ResNet-18 or EfficientNet backbone) and PolSAR data acquired in different frequency bands (S- and L-band). The results emphasize the importance of selecting an appropriate real-valued representation for CNN-based PolSAR image segmentation. This study’s findings reveal that combining multiple polarimetric features can potentially enhance segmentation performance but does not consistently improve the results. Therefore, when employing this approach, careful feature selection becomes crucial. In contrast, using coherency matrix elements with amplitude and phase representation consistently achieves high segmentation performance across different test configurations. This representation emerges as one of the most suitable approaches for CNN-based PolSAR image segmentation. Notably, it outperforms the commonly used alternative approach of splitting the coherency matrix elements into real and imaginary parts.