Orthophoto improvement using urban-SnowflakeNet

Abstract

With the increasing use of drones for capturing images in urban areas, correcting for distortion and sawtooth effects on orthophotos generated with these images has become a critical issue. This is particularly challenging due to the larger displacements generated by high objects and lower flight altitude of drones compared to crewed aircraft. In addition, image-based point cloud generation methods often fail to produce complete point clouds due to occluded areas and radiometric changes between overlapping images, especially near the borders of high objects. To address these issues, a novel method is proposed in this article for improving the generated point clouds with image-based methods using a deep learning network, called urban-SnowflakeNet, which comprises the following steps: 1) preparing and normalizing the roof's point cloud; 2) completing the point clouds of the building using the proposed deep learning network; 3) restoring the completed point clouds of the buildings to the real coordinates and combining them with the background point cloud; and, 4) correcting the DSM and generating the final true orthophotos. On two different image datasets, our method reduced distortions at the building's edges by 40% on average when compared to the most recent orthophoto enhancement method. However, by maintaining this success on more datasets, the approach has the potential to improve the accuracy and completeness of point clouds in urban regions, as well as other applications such as 3D model improvement, which require further testing in future works.