Hier finden Sie wissenschaftliche Publikationen aus den Fraunhofer-Instituten.

Real-time fusion of visible and thermal infrared images in surveillance applications on SoC hardware

: Moren, Konrad; Perschke, Thomas


Bouma, H. ; Society of Photo-Optical Instrumentation Engineers -SPIE-, Bellingham/Wash.:
Counterterrorism, Crime Fighting, Forensics, and Surveillance Technologies II : 10-11 September 2018, Berlin, Germany
Bellingham, WA: SPIE, 2018 (Proceedings of SPIE 10802)
ISBN: 978-1-5106-2188-6
ISBN: 978-1-5106-2187-9
Paper 108020Q, 9 pp.
Conference "Counterterrorism, Crime Fighting, Forensics, and Surveillance Technologies" <2, 2018, Berlin>
Conference Paper
Fraunhofer IOSB ()
image fusion; multicore CPU; SoC

Fusion of thermal infrared and visual images is an important technique for real-time surveillance applications. Since image fusion is used in many real-time night vision applications such as target detection, recognition and tracking, it is important to understand the processing requirements and provide computationally efficient methods. In this paper, we present a real-time image fusion system designed for night vision supervisory and monitoring purposes. The system is equipped with two image sensors: TV and IR (thermal infrared). We implement a processing pipeline on the NVIDIA Tegra TX2. The TX2 platform is equipped with a many-core NVIDIA GPU and multi-core ARM CPU. Additionally, we present the system architecture as well as the design process of the efficient, real-time multi-spectral signal-processing algorithm. The algorithm is based on the second-generation wavelets also called lifting scheme. We show also a novel parallelization approach to perform the calculations in place, so no auxiliary memory is needed. This enables a fast parallel and pipelined processing flow. We achieve a considerable speedup compared to an optimized CPU implementation. The experimental results show that the fusion system can realize real-time image fusion processing for dual channels images at the rate of 30 frames per second for the Full-HD images.