Hier finden Sie wissenschaftliche Publikationen aus den Fraunhofer-Instituten.

Guided optimization framework for the fusion of time-of-flight with stereo depth

: Zakeri, Faezeh Sadat; Sjöström, Mårten; Keinert, Joachim

Fulltext urn:nbn:de:0011-n-6058921 (5.7 MByte PDF)
MD5 Fingerprint: dfc77d592857d0965c12659289c92d04
Created on: 27.10.2020

Journal of electronic imaging 29 (2020), No.5, Art. 053016, 15 pp.
ISSN: 1017-9909
European Commission EC
H2020; 676401; ETN-FPI
Journal Article, Electronic Publication
Fraunhofer IIS ()
multimodal sensor fusion; time-of-flight; active and passive sensing; optimization; stereo depth estimation

The fusion of depth acquired actively with the depth estimated passively proved its significance as an improvement strategy for gaining depth. This combination allows us to benefit from two sources of modalities such that they complement each other. To fuse two sensor data into a more accurate depth map, we must consider the limitations of active sensing such as low lateral resolution while combining it with a passive depth map. We present an approach for the fusion of active time-of-flight depth and passive stereo depth in an accurate way. We propose a multimodal sensor fusion strategy that is based on a weighted energy optimization problem. The weights are generated as a result of combining the edge information from a texture map and active and passive depth maps. The objective evaluatio n of our fusion algorithm shows an improved accuracy of the generated depth map in comparison with the depth map of every single modality and with the results of other fusion methods. Additionally, a visual comparison of our result shows a better recovery on the edges considering the wrong depth values estimated in passive stereo. Moreover, the left and right consistency check on the result illustrates the ability of our approach to consistently fuse sensors.