Real-time incentivizing survey completion with game-based rewards in experience sampling research may increase data quantity, but reduces data quality

Abstract

Experience sampling methodology (ESM) requires participants to repeatedly rate various psychological states in everyday life. This can be burdensome, leading to frequently skipped or missed assessments that undermine the ecological validity of collected data. Financially compensating participants may counter suboptimal response rates, but monetary rewards are not always feasible or ethical, and are unfitting to counter non-response in the moment. Here, we investigated the potential of a gamified momentary reward strategy to encourage survey completion. In a between-person experiment, we randomly assigned 193 participants to take part in a standard (i.e., no momentary incentives) or gamified ESM protocol. Upon completing a survey in the gamified condition, participants immediately received some virtual coins to purchase various in-app rewards (fun facts or personalized graphs). Gamifying momentary survey completion resulted in higher response rates and shorter latencies, but only for participants who frequently bought rewards in the coin store. Paradoxically, momentary gamification also caused participants’ responses to be slightly more unreliable. Completion times did not differ. The reward schemes of future ESM studies should focus on both response quantity and quality. To further explore the potential of momentary incentive strategies, we provide open access to customizable gamification building blocks within our ESM platform, m-Path.