Unexpectedness in medical research

Abstract

Whether research funding is targetable is one of the central unresolved questions of science policy. A particular question is how often research aimed at understanding one disease or problem spills over to others. This has been a perennial topic of debate at the world's largest single funding body of biomedical research, the U.S. National Institutes of Health (NIH). Critics of the agency's priority-setting process have repeatedly called for better alignment between funding and disease burden, and patient advocates for specific diseases for more funding for their causes. In response, opponents of planning have argued that research in one area frequently leads to advances in others. In this study, we provide new evidence to inform these debates by examining the extent to which research funding (grants) in one scientific or disease area leads to research findings (publications) in another. We used the NIH's Research, Condition, and Disease Categorization (RCDC) to identify categories for NIH grants awarded between 2008 and 2016. We applied machine-learning to map text to these categories and use this model to categorize publications resulting from these grants. We categorized over 1.2 million publications, resulting from over 90,000 grants. We found that 70 % of the publications have at least one RCDC category not in its grant, which we termed ‘unexpected’ categories. On average, 40 % of categories assigned to a publication were unexpected. After adjusting for similarity across some of the RCDC categories by empirically clustering the categories, we found 58 % of the publications had at least one unexpected category and, on average, 33 % of publication categories were unexpected. Our results suggest that disease-orientation and clinical research were less likely to be associated with spillovers. Grants resulting from targeted requests for applications were more likely to result in publications with unexpected categories, though the magnitude of the differences was relatively small.