Temporal Multimodal Probabilistic Transformers for Safety Monitoring in Autonomous Driving Systems

Ensuring reliable safety monitoring in autonomous driving systems (ADS) under uncertainty is essential for deployment in real-world scenarios. We propose the Temporal Multimodal Probabilistic Transformer (TMPT), a novel deep learning framework that integrates uncertainty quantification (UQ) into lane-keeping safety monitoring. TMPT forecasts lane deviation metrics along with calibrated aleatoric and epistemic uncertainties by processing sequences of multimodal sensor and control data. Our framework combines Transformer-based temporal fusion with deep ensembles and post-hoc calibration to improve predictive accuracy and uncertainty estimation. We evaluate 24 model variants in the CARLA simulator, analyzing the impact of architecture, calibration, and ensembling on both prediction and uncertainty. Calibrated models achieve near-perfect uncertainty reliability (ENCE < 0.03), while uncalibrated models show sharper predictions but overconfident errors. Ensemble methods further improve robustness but incur significant computational cost. Our findings show that aligning model selection with application context—balancing precision, calibration, and efficiency—is critical for safe and practical ADS deployment.