Probabilistic Segmentation Using PDQ-inspired Loss Functions

Obtaining spatial uncertainty estimation for object detection task is essential for safety critical applications such as autonomous driving. Probability-based Detection Quality (PDQ) is a metric introduced to evaluate probabilistic object detectors.This work facilitates PDQ as a training loss instead of a metric to directly produce pixel wise probabilities, with the goal of emulating the pixel wise probabilities usually computed from probabilistic bounding boxes during the PDQ procedure. Additionally, weaknesses of the PDQ formulation - specifically an implicit, hidden thresholding operation - in this context are shown and an explicitely thresholded version, tPDQ, is developed. Three increasingly complex models, including a MaskR-CNN, are trained using PDQ, tPDQ and BCE as losses. It is shown that in regard to PDQ and tPDQ these pixel wise probabilities are on par or better than probabilities produced by sampling from the detection heads of the same models, while still being computed magnitudes faster. Models trained on PDQ or tPDQ exhibit a significantly different behaviour than segmentation models trained on BCE, assigning more probability mass to ambiguous or uncertain image areas, while producing less defined and visually appealing segmentations as a tradeoff. PDQ as an evaluation metric is shown to be unreliable and potentially misleading when applied to segmentations instead of sampled detections. The proposed simplefix, tPDQ, solves these problems.