Understanding variability: the role of meta-analysis of variance.

Abstract

Meta-analyses traditionally compare the difference in means between groups for one or more outcomes of interest. However, they do not compare the spread of data (variability), which could mean that important effects and/or subgroups are missed. To address this, methods to compare variability meta-analytically have recently been developed, making it timely to review them and consider their strengths, weaknesses, and implementation. Using published data from trials in major depression, we demonstrate how the spread of data can impact both overall effect size and the frequency of extreme observations within studies, with potentially important implications for conclusions of meta-analyses, such as the clinical significance of findings. We then describe two methods for assessing group differences in variability meta-analytically: the variance ratio (VR) and coefficient of variation ratio (CVR). We consider the reporting and interpretation of these measures and how they differ from the assessment of heterogeneity between studies. We propose general benchmarks as a guideline for interpreting VR and CVR effects as small, medium, or large. Finally, we discuss some important limitations and practical considerations of VR and CVR and consider the value of integrating variability measures into meta-analyses.