Dynamic inference for on-orbit scene classification with the scale boosting model

Abstract

Existing scene classification methods allocate the same computational resources, i.e., all model parameters in the neural network, to each remote sensing image whenever from any geographic scene. However, this might be redundant for images of certain scenes that are easy to discriminate, e.g., homogeneous scenes. This observation motivates us to propose an efficient method for on-orbit scene classification, namely, the Scale Boosting Model (SBM). Specifically, during the training process, the SBM is built as a set of different scale learners in a scale-increasing manner, each of which is used to learn and classify image features at a specific scale. During inference, the scale learners in the SBM will be selectively run in a scale-increasing manner and automatically decide when to exit early or expand the computation according to the scene complexity. In addition, by replacing the backbone of the scale learner, the SBM could provide a deployment possibility for computationally limited models for on-orbit processing, thereby reducing their computational requirements. Extensive experiments on UC Merced Land Use, NWPU-RESISC45 and RSD46-WHU datasets show that the SBM achieved a more effective classification performance more efficiently.