Choice Consistency in Discrete Choice Experiments: Does Numeracy Skill Matter?

Abstract

Objectives

This study investigated the relationship between numeracy skills (NS) and choice consistency in discrete choice experiments (DCEs).

Methods

A DCE was conducted to explore patients’ preferences for kidney transplantation in Italy. Patients completed the DCE and answered 3-item numeracy questions. A heteroskedastic multinomial logit model was used to investigate the effect of numeracy on choice consistency.

Results

Higher NS were associated with greater choice consistency, increasing the scale to 1.63 (P < .001), 1.39 (P < .001), and 1.18 (P < .001) for patients answering 3 of 3, 2 of 3, and 1 of 3 questions correctly, respectively, compared with those with no correct answers. This corresponded to 63%, 39%, and 18% more consistent choices, respectively. Accounting for choice consistency resulted in varying willingness-to-wait (WTW) estimates for kidney transplant attributes. Patients with the lowest numeracy (0/3) were willing to wait approximately 42 months [95% CI: 29.37, 54.68] for standard infectious risk, compared with 33 months [95% CI: 28.48, 38.09] for 1 of 3, 28 months [95% CI: 25.13, 30.32] for 2 of 3, and 24 months [95% CI: 20.51, 27.25] for 3 of 3 correct answers. However, WTW differences for an additional year of graft survival and neoplastic risk were not statistically significant across numeracy levels. Supplementary analyses of 2 additional DCEs on COVID-19 vaccinations and rheumatoid arthritis, conducted online, supported these findings: higher NS were associated with more consistent choices across different disease contexts and survey formats.

Conclusions

The findings suggested that combining patients with varying NS could bias WTW estimates, highlighting the need to consider numeracy in DCE data analysis and interpretation.