CollectiAR: Computer Vision-Based Word Hunt for Children with Dyslexia

Abstract

Children with dyslexia face extra challenges in reading and writing words. They need more learning exercises than children with typical development to acquire vocabulary, which is often repetitive and daunting. Research has shown that combining visuospatial information in practices helped children with dyslexia memorize words, especially the real-world physical context. Nevertheless, the existing word recognition and spelling training games for children with dyslexia were not able to leverage children’s immediate vicinity. Therefore, we designed an augmented reality mobile game, CollectiAR, that uses computer vision to identify objects in the player’s immediate vicinity and direct the player to learn words for these objects. Our formative study with two elementary school teachers and a first-grade pupil found that CollectiAR has the potential to be an integral part of teachers’ instructional design and an engaging way for pupils to practice vocabulary exercises. Our teacher participants suggested that CollectiAR provide interfaces for teachers to participate in the game content design and computer vision model correction.