Fast and robust generation of semantic urban terrain models from UAV video streams

We present an algorithm for extracting Level of Detail 2 (LOD2) building models from video streams captured by Unmaned Aerial Vehicles (UAVs). Typically, such imagery is of limited radiometric quality but the surface is captured with large redundancy. The first contribution of this paper is a novel algorithm exploiting this redundancy for precise depth computation. This is realized by fusing consistent depth estimations across single stereo models and generating a 2.5D elevation map from the resulting point clouds. Disparity maps are derived by a coarse-to-fine Semi-Global-Matching (SGM) method performing well on noisy imagery. The second contribution concerns a challenging step of the context-based urban terrain modeling: Dominant planes extraction for building reconstruction. Because of noisy data and complicated roof structures, both dominant plane parameters and initial values for support sets of planes are obtained by the J-Linkage algorithm. An improved pointto-plane labeling is presented to encourage the assignment of proximate points to the same plane. This is accomplished by non-local, Markov Random Field (MRF) - based optimization and segmentation of color information. The potential and the limitations of the proposed methods are shown using an UAV video sequence of limited radiometric quality.