Unraveling topic switching and innovation in science

Abstract

The selection of research topics shapes both individual scientific trajectories and the broader evolution of knowledge. Despite its critical role, a systematic investigation into the dynamics of topic switching among scientists and its relationship with scientific innovation remains limited. Drawing on a comprehensive dataset encompassing the career trajectories of 1.4 million scientists and 27.6 million publications from 1950 to 2020, I use a field-free and finely-grained framework to quantify shifts in research direction by measuring the knowledge distance between a paper's references and those of prior works. To account for systemic biases, I construct a null model that captures expected patterns of topic selection. My analysis reveals three key findings: (1) Scientists exhibit lower-than-expected levels of topic switching, with a decline before 2000 followed by a rising trend thereafter; (2) Early-career researchers, female scientists, and non-elite scientists demonstrate higher levels of topic switching compared to their counterparts; and (3) Increased topic switching correlates with greater research novelty, interdisciplinarity, and disruptive potential. These findings provide valuable insights into the mechanisms underlying scientific exploration and their implications for innovation, with broad relevance for research policy and talent development.