HeightFormer: Single-Imagery Height Estimation Transformer With Bilateral Feature Pyramid Fusion

Abstract

Despite their ill-posedness and inherent ambiguity, recent deep learning approaches have demonstrated promising capability to estimate plausible height information from single spaceborne and airborne imagery. However, accurately predicting the height and preserving the rich geometric detailing of aerial images with limited resolution and complex structural variations remains a challenge. To address these issues, we introduce a novel transformer-based architecture for single-imagery height estimation (SIHE) dubbed as HeightFormer. Specifically, the building-block multiscale vision transformer (MViT) constitutes the encoder and decoder of HeightFormer to facilitate the capturing of long-range dependencies across a feature pyramid. Furthermore, we propose the bilateral feature pyramid fusion scheme, which consists of step-by-step and one-stop decoder feature map augmentation, to enhance global and local information reconstruction. The stepwise fusion module (SFM) iteratively fuses encoder and decoder features, while the multiscale fusion module (MFM) combines the final decoder feature with multiscale encoder features. In the end, the Heightbins module is designed to generate the attention map and the adaptive bin width. Then, the bin centers at each pixel are linearly combined as the final estimated height. Extensive experiments validate the effectiveness of HeightFormer on the Vaihingen dataset, the Potsdam dataset, and the DFC2019 dataset. Compared with the state-of-the-art, our method improves accuracy metrics and provides the ability to preserve structure and details. Building height estimation, transformer, attention, progressive refinement.