Learning to segment from misaligned and partial labels

Abstract

To extract information at scale, researchers are increasingly applying semantic segmentation techniques to remotely-sensed imagery. While fully-supervised learning enables accurate pixelwise segmentation, compiling the exhaustive datasets required is often prohibitively expensive, and open-source datasets that do exists are frequently inexact and non-exhaustive. In this paper, we present a novel and generalizable two-stage framework that enables improved pixelwise image segmentation given misaligned and missing annotations. First, we introduce the Alignment Correction Network to rectify incorrectly registered open source labels. Next, we demonstrate a segmentation model - the Pointer Segmentation Network - that uses corrected labels to predict infrastructure footprints despite missing annotations. We demonstrate the transferability of our method to lower quality data sources by applying the Alignment Correction Network to correct OpenStreetMaps building footprints, and we show the accuracy of the Pointer Segmentation Network in predicting cropland boundaries in California. Overall, our methodology is robust for multiple applications with varied amounts of training data present, thus offering a method to extract reliable information from noisy, partial data.