Accuracy and distribution of baseline categorical variables and p-values in spine randomized controlled trials.

Abstract

Levayer and colleagues assessed integrity issues in randomized controlled trials (RCTs) in four spine journals using baseline p-values from categorical variables, concluding that there was no evidence of 'systemic fraudulent behaviour'. We used their published dataset to assess the accuracy of reported p-values and whether observed and expected distributions of frequency counts and p-values were consistent. In 51 out of 929 (5.5%) baseline variables, the sum of frequencies did not agree with the reported number of participants. For one-third of reported p-values (172 out of 522), we could not calculate a matching p-value using a range of statistical tests. Sparse data were common: for 22% (74 out of 332) of variables in which the reported p-value matched the p-value calculated from a chi-square test, the expected cells were smaller than recommended for the use of chi-square tests. There were 20-25% more two-arm trials with differences in frequency counts of 1 or 2 between-groups than expected. There were small differences between observed and expected distributions of baseline p-values, but these depended on analysis methods. In summary, incorrectly reported p-values and incorrect statistical test usage were common, and there were differences between observed and expected distributions of baseline p-values and frequency counts, raising questions about the integrity of some RCTs in these journals.