Are units of analysis properly considered in orthodontic meta-analyses?

Background: Assessment of unit of analysis (UoA) in meta-analysis with cluster, split-mouth, repeated measures designs, and multiple intervention groups is a fundamental step in the analysis. The objective of this study was to evaluate the UoAs in orthodontic meta-analysis and determine the error of the analysis.

Methods: An electronic search was conducted in the Cochrane Library and PubMed to identify orthodontic systematic reviews (SRs) with meta-analyses published in Cochrane and in the highest impact orthodontic journals between 1 January 2013 and 31 December 2022. SRs with meta-analysis assessing at least one of the following UoAs; cluster trials, crossover trials, multiple observations, or multiple intervention groups were included. Screening and data extraction were undertaken by two investigators independently. Descriptive statistics for the study characteristics were provided. The associations between avoiding the unit analysis error (yes, no) and the study characteristics were examined using Fisher's exact test and chi-square test. Logistic regression was undertaken for the significant predictors.

Results: Eighty SRs were deemed eligible for inclusion. Only 30 per cent of the included SRs avoided UoA errors. Compared to the split-mouth design, repeated measures designs had higher odds of avoiding UoA error (odds ratio: 9.6, 95% confidence interval: 2.8-32.3, P < 0.001). In contrast, fewer odds of avoiding the UoA error were found in the cluster design (OR: 0.2, 95% CI: 0.4-1.3, P = 0.09). Though multiple intervention groups have higher odds of avoiding UoA error than split-mouth studies, this was not statistically significant (OR: 2.1, 95% CI: 0.5-8, P = 0.28). None of the SRs characteristics have influenced the appropriate handling of the unit analysis except the type of the journal; the odds of avoiding the UoA error were higher in Cochrane reviews than the non-Cochrane reviews (OR: 3.3, 95% CI: 1.2-8.7, P = 0.02), and the number of authors (P < 0.05).

Conclusions: UoA errors are common in orthodontic meta-analyses and were only partially avoided in split-mouth design, repeated measures design, and multiple intervention groups.