Statistical Heterogeneity in Oral Health Meta-Analyses

Abstract

Providing the summary effect size and its uncertainty, a prediction interval, and a measure of statistical heterogeneity constitute good reporting practices in meta-analyses. Popular statistical heterogeneity measures comprise the τ2 and I2 statistics. However, researchers often rely unduly on the I2 statistic, using naive categorizations to gauge the extent of heterogeneity, leading to misuses of the meta-analysis models, deficiencies in reporting, and misleading conclusions. The present study aimed to provide empirical evidence on the reporting and interpretation of statistical heterogeneity in systematic reviews of oral health published between 2021 and 2023 in 21 leading specialty and general dental journals. Systematic reviews with at least 1 meta-analysis on binary or continuous outcomes with the most studies were identified. Characteristics were extracted at the systematic review and meta-analysis levels. In total, 313 systematic reviews with meta-analyses were analyzed. Within this cohort of meta-analyses, the random-effects model (89%, n = 278) was frequently applied. Almost all meta-analyses (98%, n = 308) reported the I2 value, and 51% ( n = 160) reported the τ2 value. For this sample, the median I2 was 76% (range: 0%–100%), and the median τ2 was 0.29 (range: 0–2,632), with 13% ( n = 20/160) of these meta-analyses reporting zero τ2. Most of the meta-analyses (96%, n = 299) based the heterogeneity interpretation on I2 and only 21 (7%) on τ2. Although 49% ( n = 152) of the meta-analyses chose the meta-analysis model a priori, only 41% ( n = 63/152) justified this choice. Furthermore, 42% ( n = 131) of the 313 meta-analyses chose the meta-analysis model based on the I2. Within oral health meta-analyses, there is evidence of overreliance on I2 when reporting and interpreting statistical heterogeneity and selecting the meta-analysis model. The inappropriate use of I2 in meta-analysis model selection and interpretation of statistical heterogeneity may have implications for the quality of conclusions delivered to the end users of systematic reviews.