Options
July 13, 2023
Master Thesis
Title
A Comprehensive Approach to Road User Trajectory Tracking and Prediction with Infrared Camera and Radar Data Fusion
Abstract
Intelligent traffic systems require comprehensive environmental perception sensors to provide valuable data for smart city applications and can further make assisted or autonomous driving more efficient, and safer. For this purpose, sensors that are more robust to severe weather conditions are required. Automotive Radar and Infrared thermal camera sensors are exclusive elements used for Autonomous Driving and Smart Infrastructure systems and can work in any challenging environment. These sensors are providing very crucial information for safe and effective traffic flows. The fusion of radar and infrared cameras can greatly enrich the completeness of environmental perception. Moreover, with the use of such modalities, one can avoid privacy issues. For optimizing the traffic flows, not only the detection and tracking of the road users needed but additional information about them such as forecasting of future positions (future trajectories) are also beneficiary. The smart infrastructure consisting of radar and infrared camera sensors cannot only give good environmental perception (e.g. Object Detection) but can be used to get more reliable information by tracking and predicting the future path of the detected objects. For autonomous driving, V2X communication, and traffic management, predicting future trajectories are essential since road users’ intentions can be seriously misinterpreted when they engage with one another. In this thesis, the development of a comprehensive pipeline for the detection of road user and their trajectory tracking with long-term trajectory prediction has been proposed. The pipeline consists of three major tasks, which are traffic participant detection, tracking task, and trajectory forecasting task. Detection and tracking play crucial roles in achieving accurate trajectory forecasting. To ensure a safe and reliable recognition of people and traffic participants, as well as to establish a tracking and prediction system that meets the required criteria, this thesis incorporates two methods containing 2D object detections and 3D object detections, followed by 2D object tracking and 3D object tracking, respectively. To accomplish the 3D detection method, dataset creation has been conducted by collecting real traffic data from the traffic junction. To fuse the data from infrared cameras and radar sensors, this thesis proposes a cluster-based approach to get the object proposals from the raw radar point cloud data and implements a fusion by association (Late fusion) approach to fusing with image detection. For the trajectory prediction task classical (Kalman Filter) as well as data-driven (GAN-based network, Transformer based network) approaches have been implemented and compared. The experimental results show that the 2d detection method gives better accuracy with 92% mAP, while the 3d detection approach gives more robustness for the accurate radar data association (by utilizing the 3D-IoU). In terms of trajectory prediction, the transformer-based network outperforms the GAN (Generative Adversarial Network) model and Kalman filter-based approaches by delivering more accurate and meaningful trajectory predictions.
Thesis Note
Ingolstadt, TH, Master Thesis, 2023
Author(s)