Dense point cloud generation of urban scenes from nadir RGB images in a remote sensing system
A near real-time airborne 3D scanning system has been successfully implemented at Fraunhofer IOSB. This remote sensing system consists of a small aircraft and a ground control station. The aircraft is equipped with the following components: a Digital Acquisition System (DAQ), Inertial Navigation and Global Positioning Systems (INS/GPS), an Airborne Laser Scanner (ALS), and four industrial cameras. Two of these cameras (RGB and near-infrared ones) are nadir oriented, while the other two RGB cameras have an oblique orientation. The acquired LiDAR point clouds, images, and corresponding metadata are sent from the aircraft to the ground control station for further post-processing procedures, such as radiometric correction, boresight correction, and point cloud generation from images. In this paper, the procedure regarding point cloud generation of urban scenes, with images from the nadir RGB camera, is described in detail. To produce dense point clouds three main steps are necessary: generation of disparity maps, creation of depth maps, and calculation of world coordinates (X, Y, and Z). To create disparity maps, two adjacent images (stereopair) were rectified. Afterwards, the PatchMatch Stereo (PMS) algorithm for 3D reconstruction was executed, since it is easy to implement and provides good results according to the Middlebury Computer Vision dataset. Some steps were parallelized to optimize execution speed. Since depth is inversely proportional to disparity, depth maps were calculated from disparity maps. The height of scene elements Z was obtained by subtracting their depth to the camera height. To calculate the remaining world coordinates X and Y, the back-projection equation and the camera intrinsic and extrinsic parameters were used. To validate the PMS algorithm, its resulting point cloud was compared with a LiDAR point cloud and a PhotoScan point cloud. The root mean square errors of both comparisons showed similar values.