Baseline-dependent improvement in CF studies, plausibility of bias

Background:

It has been commonly reported that therapeutic treatments in cystic fibrosis (CF) have ceiling effects, such that their efficacy is diminished for persons with high pre-treatment health (Montgomery et al., 2012 and Newsome et al., 2019). Floor effects have also been reported where decline is of lower magnitude in those with below-average pre-treatment health (Harun et al., 2016; Konstan et al., 2012 and Szczesniak et al., 2017). When measurement error is present, the statistical literature has warned of exaggerated or spurious associations between pre-treatment measures and subsequent change (Chambless and Davis, 2003 and Yanez et al., 1998). Measurement error, equivalently described as day-to-day variation, has been described to occur in CF outcome measurements such as forced expiratory volume in 1 s taken by spirometry (FEV${}_1$pp) (Magaret et al., 2024; Stanojevic et al., 2020 and Thornton et al., 2023).

Methods:

We conducted a simulation study to assess the potential for spurious floor or ceiling effects in studies of CF therapeutics. We considered uncontrolled or single-arm studies, and evaluated estimated association between pre-treatment FEV${}_1$pp and treatment-induced change: post-versus pre-treatment.

Results:

When day-to-day variation was present in FEV${}_1$pp, at levels equivalent to those reported in large studies measuring spirometry both at home and in clinic, naive analytic approaches found spurious associations of change with baseline (Paynter et al., 2022 and Saiman et al., 2003). Type I error ranged from 31.9% to 98.3% for day-to-day variation as high as 3% to 15% relative to biological variation. Incorporating known day-to-day variation, the regression calibration approach corrected bias and controlled type I error (Chambless and Davis, 2003).

Conclusion:

Exaggerated ceiling effects are possible. Further studies could provide meaningful confirmation of ceiling effects in CF, perhaps reducing day-to-day variation by incorporating multiple pre- and post-treatment measurements.