Meta-Analysis 2020: A Dire Alert and a Fix

Abstract

It is hard to believe that mainstream meta-analysis, whose primary objective is to provide a summary estimate of effect size from a set of completed studies, has a major flaw. Yet because the mainstream treats weights and/or sample sizes as constants, rather than unequivocally seriously random variables, this is exactly the situation. Further, the mainstream random effects model does not permit association between weights and effect size, which if false, can lead to major bias. For the following scenarios, we provide a fix, relying on ratio estimation from cluster sampling, to produce simple and valid asymptotic methods for the following scenarios: Estimation of means or proportions, differences of means or proportion from randomized trials, estimation of relative risk from randomized trials, and repeated measures Bland-Altman studies aimed at replacing invasive by non-invasive measures. One horror story for mainstream methods saw a highly significant result in a major study become insignificant when we kept the study point estimates the same but universally cut the study standard errors by 30%. With over 1400 meta-analyses papers published per month in 2019, it is essential to use this paper as a springboard to mitigate this situation.