Lindsey Schader, Weishan Song, Russell Kempker, David Benkeser
{"title":"不要让你的分析变成种子:随机种子对基于机器学习的因果推理的影响。","authors":"Lindsey Schader, Weishan Song, Russell Kempker, David Benkeser","doi":"10.1097/EDE.0000000000001782","DOIUrl":null,"url":null,"abstract":"<p><p>Machine learning techniques for causal effect estimation can enhance the reliability of epidemiologic analyses, reducing their dependence on correct model specifications. However, the stochastic nature of many machine learning algorithms implies that the results derived from such approaches may be influenced by the random seed that is set before model fitting. In this work, we highlight the substantial influence of random seeds on a popular approach for machine learning-based causal effect estimation, namely doubly robust estimators. We illustrate that varying seeds can yield divergent scientific interpretations of doubly robust estimates produced from the same dataset. We propose techniques for stabilizing results across random seeds and, through an extensive simulation study, demonstrate that these techniques effectively neutralize seed-related variability without compromising the statistical efficiency of the estimators. Based on these findings, we offer practical guidelines to minimize the influence of random seeds in real-world applications, and we encourage researchers to explore the variability due to random seeds when implementing any method that involves random steps.</p>","PeriodicalId":11779,"journal":{"name":"Epidemiology","volume":" ","pages":"764-778"},"PeriodicalIF":4.7000,"publicationDate":"2024-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Don't Let Your Analysis Go to Seed: On the Impact of Random Seed on Machine Learning-based Causal Inference.\",\"authors\":\"Lindsey Schader, Weishan Song, Russell Kempker, David Benkeser\",\"doi\":\"10.1097/EDE.0000000000001782\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<p><p>Machine learning techniques for causal effect estimation can enhance the reliability of epidemiologic analyses, reducing their dependence on correct model specifications. However, the stochastic nature of many machine learning algorithms implies that the results derived from such approaches may be influenced by the random seed that is set before model fitting. In this work, we highlight the substantial influence of random seeds on a popular approach for machine learning-based causal effect estimation, namely doubly robust estimators. We illustrate that varying seeds can yield divergent scientific interpretations of doubly robust estimates produced from the same dataset. We propose techniques for stabilizing results across random seeds and, through an extensive simulation study, demonstrate that these techniques effectively neutralize seed-related variability without compromising the statistical efficiency of the estimators. Based on these findings, we offer practical guidelines to minimize the influence of random seeds in real-world applications, and we encourage researchers to explore the variability due to random seeds when implementing any method that involves random steps.</p>\",\"PeriodicalId\":11779,\"journal\":{\"name\":\"Epidemiology\",\"volume\":\" \",\"pages\":\"764-778\"},\"PeriodicalIF\":4.7000,\"publicationDate\":\"2024-11-01\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Epidemiology\",\"FirstCategoryId\":\"3\",\"ListUrlMain\":\"https://doi.org/10.1097/EDE.0000000000001782\",\"RegionNum\":2,\"RegionCategory\":\"医学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"2024/8/16 0:00:00\",\"PubModel\":\"Epub\",\"JCR\":\"Q1\",\"JCRName\":\"PUBLIC, ENVIRONMENTAL & OCCUPATIONAL HEALTH\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Epidemiology","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.1097/EDE.0000000000001782","RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"2024/8/16 0:00:00","PubModel":"Epub","JCR":"Q1","JCRName":"PUBLIC, ENVIRONMENTAL & OCCUPATIONAL HEALTH","Score":null,"Total":0}
Don't Let Your Analysis Go to Seed: On the Impact of Random Seed on Machine Learning-based Causal Inference.
Machine learning techniques for causal effect estimation can enhance the reliability of epidemiologic analyses, reducing their dependence on correct model specifications. However, the stochastic nature of many machine learning algorithms implies that the results derived from such approaches may be influenced by the random seed that is set before model fitting. In this work, we highlight the substantial influence of random seeds on a popular approach for machine learning-based causal effect estimation, namely doubly robust estimators. We illustrate that varying seeds can yield divergent scientific interpretations of doubly robust estimates produced from the same dataset. We propose techniques for stabilizing results across random seeds and, through an extensive simulation study, demonstrate that these techniques effectively neutralize seed-related variability without compromising the statistical efficiency of the estimators. Based on these findings, we offer practical guidelines to minimize the influence of random seeds in real-world applications, and we encourage researchers to explore the variability due to random seeds when implementing any method that involves random steps.
期刊介绍:
Epidemiology publishes original research from all fields of epidemiology. The journal also welcomes review articles and meta-analyses, novel hypotheses, descriptions and applications of new methods, and discussions of research theory or public health policy. We give special consideration to papers from developing countries.