A systematic review of mobile device use in the primary school classroom and impact on pupil literacy and numeracy attainment: A systematic review

Background

Investment in mobile devices to support primary or elementary education is increasing and must be informed by robust evidence to demonstrate impact. This systematic review of randomised controlled trials sought to identify the overall impact of mobile devices to support literacy and numeracy outcomes in mainstream primary classrooms.

Objectives

The aim of this systematic review was to understand how mobile devices are used in primary/elementary education around the world, and in particular, determine how activities undertaken using mobile devices in the primary classroom might impact literacy and numeracy attainment for the pupils involved. Within this context, mobile devices are defined as tablets (including iPads and other branded devices), smartphones (usually those with a touchscreen interface and internet connectivity) and handheld games consoles (again usually with touchscreen and internet-enabled). The interventions of interest were those aimed at improving literacy and/or numeracy for children aged 4–12 within the primary/elementary school (or equivalent) classroom.

Specifically, the review aimed to answer the following research questions:

• -

  What is the effect of mobile device integration in the primary school classroom on children's literacy and numeracy outcomes?

• -

  Are there specific devices which are more effective in supporting literacy and numeracy? (Tablets, smartphones, or handheld games consoles)

• -

  Are there specific classroom integration activities which moderate effectiveness in supporting literacy and numeracy?

• -

  Are there specific groups of children for whom mobile devices are more effective in supporting literacy and numeracy? (Across age group and gender).

• -

  Do the benefits of mobile devices for learning last for any time beyond the study?

• -

  What is the quality of available evidence on the use of mobile devices in primary/elementary education, and where is further research needed in this regard?

An Expert Advisory Group supported the review process at key stages to ensure relevance to current practice.

Search Methods

The search strategy was designed to retrieve both published and unpublished literature, and incorporated relevant journal and other databases with a focus on education and social sciences. Robust electronic database searches were undertaken (12 databases, including APA PsychInfo, Web of Science, ERIC, British Education Index and others, and relevant government and other websites), as well as a hand-search of relevant journals and conference proceedings. Contact was also made with prominent authors in the field to identify any ongoing or unpublished research. All searches and author contact took place between October and November 2020. The review team acknowledges that new studies will likely have emerged since and are not captured at this time. A further update to the review in the future is important and would build on the evidence reflected here.

Selection Criteria

The review included children within mainstream primary/elementary/kindergarten education settings in any country (aged 4–12), and interventions or activities initiated within the primary school classroom (or global equivalent) that used mobile devices (including tablets, smartphones, or hand-held gaming devices) to intentionally support literacy or numeracy learning. In terms of study design, only Randomised Controlled Trials were included in the review.

Data Collection and Analysis

A total of 668 references were identified through a robust search strategy including published and unpublished literature. Following duplicate screening, 18 relevant studies, including 11,126 participants, 14 unique interventions, and 46 relevant outcome measures were synthesised using Robust Variance Estimation and a random effects meta-analysis model. Risk of Bias assessment was undertaken by three reviewers using the ROB2 tool to assess the quality of studies, with 13 studies rated as having some concerns, and 5 as having high risk of bias. Qualitative data was also extracted and analysed in relation to the types of interventions included to allow a comparison of the key elements of each.

Main Results

A positive, statistically significant combined effect was found (Cohen's d = 0.24, CI 0.0707 to 0.409, p < 0.01), demonstrating that in the studies and interventions included, children undertaking maths or literacy interventions using mobile devices achieved higher numeracy or literacy outcomes than those using an alternative device (e.g., a laptop or desktop computer) or no device (class activities as usual). However these results should be interpreted with caution given the risk of bias assessment noted above (5 studies rated high risk of bias and 13 rated as having some concerns). As the interventions and classroom circumstances differed quite widely, further research is needed to understand any potential impact more fully.

Sensitivity analysis aimed to identify moderating factors including age or gender, screen size, frequency/dosage of intervention exposure, and programme implementation features/activities (based on Puentedura's [2009] SAMR model of technology integration). There were too few studies identified to support quantitative analysis of sufficient power to draw robust conclusions on moderating factors, and insufficient data to determine impact beyond immediate post-test period. Sensitivty analysis was also undertaken to exclude the five studies identified as having a high risk of bias, to identify any impact they may have on overall findings.

Authors' Conclusions

Overall, this review demonstrates that for the specific interventions and studies included, mobile device use in the classroom led to a significant, positive effect on literacy and numeracy outcomes for the children involved, bringing positive implications for their continued use in primary education. However given the concerns on risk of bias assessment reported above, the differing circumstances, interventions and treatment conditions and intensities, the findings must be interpreted with caution. The review also supports the need for further robust research to better understand what works, under what circumstances, and for whom, in the use of mobile devices to support learning.