Scaling Crowd+AI Sidewalk Accessibility Assessments: Initial Experiments Examining Label Quality and Cross-city Training on Performance

Abstract

Increasingly, crowds plus machine learning techniques are being used to semi-automatically analyze the accessibility of built environments; however, open questions remain about how to effectively combine the two. We present two experiments examining the effect of crowdsourced data in automatically classifying sidewalk accessibility features in streetscape images. In Experiment 1, we investigate the effect of validated data—which has been voted correct by the crowd but is more expensive to collect—compared with a larger but noisier aggregate dataset. In Experiment 2, we examine whether crowdsourced labeled data gathered in one city can be used as effective training data for another. Together, these experiments contribute to the growing literature in Crowd+AI approaches for semi-automatic sidewalk assessment and help identify pertinent challenges.