School Mask Mandates and COVID-19: The Challenge of Using Difference-in-Differences Analysis of Observational Data to Estimate the Effectiveness of a Public Health Intervention.
{"title":"School Mask Mandates and COVID-19: The Challenge of Using Difference-in-Differences Analysis of Observational Data to Estimate the Effectiveness of a Public Health Intervention.","authors":"Ambarish Chandra, Tracy Beth Høeg, Shamez Ladhani, Vinay Prasad, Ram Duriseti","doi":"10.7326/M23-2907","DOIUrl":null,"url":null,"abstract":"<p><strong>Background: </strong>There are considerable challenges when using difference-in-differences (DiD) analysis of ecological data to estimate the effectiveness of public health interventions in rapidly changing situations.</p><p><strong>Objective: </strong>To discuss the shortcomings of DiD methodology for the estimation of the effects of public health interventions using ecological data.</p><p><strong>Design: </strong>As an example, the authors consider an analysis that used DiD methodology and reported a causal reduction in COVID-19 cases due to the maintenance of school mask mandates. They did alternate analyses using various control groups to assess the robustness of the prior analysis.</p><p><strong>Setting: </strong>School districts in the greater Boston area and Massachusetts during the 2021-to-2022 academic year.</p><p><strong>Participants: </strong>Students and school staff.</p><p><strong>Measurements: </strong>Changes in COVID-19 case rates in districts that did and did not lift mask mandates.</p><p><strong>Results: </strong>Important potential confounders rendered DiD methodology inappropriate for causal inference, including prior immunity, temporal variation in rates of infection, and changes in testing practices. The racial composition and income of intervention and control groups also differed substantially. Compared with maintaining the mask requirement, dropping the requirement was associated with anywhere from an increase of 5.64 cases (95% CI, 3.00 to 8.29 cases) per 1000 persons to a decrease of 2.74 cases (CI, 0.63 to 4.85 cases) per 1000 persons, depending on choice of control group and whether students or staff were examined.</p><p><strong>Limitation: </strong>Ecological data were used; detailed data on all potential confounders were unavailable.</p><p><strong>Conclusion: </strong>Alternate analyses yielded estimates consistent with a wide range of both negative and positive associations in COVID-19 case rates after removal of mask mandates. The findings highlight the challenges of using DiD analysis of ecological data to estimate the effectiveness of interventions in divergent intervention and control groups during rapidly changing circumstances.</p><p><strong>Primary funding source: </strong>None.</p>","PeriodicalId":7932,"journal":{"name":"Annals of Internal Medicine","volume":" ","pages":"1566-1572"},"PeriodicalIF":19.6000,"publicationDate":"2024-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Annals of Internal Medicine","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.7326/M23-2907","RegionNum":1,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"2024/10/8 0:00:00","PubModel":"Epub","JCR":"Q1","JCRName":"MEDICINE, GENERAL & INTERNAL","Score":null,"Total":0}
引用次数: 0
Abstract
Background: There are considerable challenges when using difference-in-differences (DiD) analysis of ecological data to estimate the effectiveness of public health interventions in rapidly changing situations.
Objective: To discuss the shortcomings of DiD methodology for the estimation of the effects of public health interventions using ecological data.
Design: As an example, the authors consider an analysis that used DiD methodology and reported a causal reduction in COVID-19 cases due to the maintenance of school mask mandates. They did alternate analyses using various control groups to assess the robustness of the prior analysis.
Setting: School districts in the greater Boston area and Massachusetts during the 2021-to-2022 academic year.
Participants: Students and school staff.
Measurements: Changes in COVID-19 case rates in districts that did and did not lift mask mandates.
Results: Important potential confounders rendered DiD methodology inappropriate for causal inference, including prior immunity, temporal variation in rates of infection, and changes in testing practices. The racial composition and income of intervention and control groups also differed substantially. Compared with maintaining the mask requirement, dropping the requirement was associated with anywhere from an increase of 5.64 cases (95% CI, 3.00 to 8.29 cases) per 1000 persons to a decrease of 2.74 cases (CI, 0.63 to 4.85 cases) per 1000 persons, depending on choice of control group and whether students or staff were examined.
Limitation: Ecological data were used; detailed data on all potential confounders were unavailable.
Conclusion: Alternate analyses yielded estimates consistent with a wide range of both negative and positive associations in COVID-19 case rates after removal of mask mandates. The findings highlight the challenges of using DiD analysis of ecological data to estimate the effectiveness of interventions in divergent intervention and control groups during rapidly changing circumstances.
学校面具规定与 COVID-19：利用观察数据的差异分析来估算公共卫生干预措施的效果所面临的挑战》（The Challenge of Using Difference-in-Differences Analysis of Observational Data to Estimate the Effectiveness of a Public Health Intervention）。
背景：在瞬息万变的形势下，利用生态数据的差异分析（DiD）来估计公共卫生干预措施的效果存在相当大的挑战：讨论使用生态数据估算公共卫生干预措施效果的 DiD 方法的不足之处：设计：作为一个例子，作者考虑了一项使用 DiD 方法进行的分析，该分析报告了由于维持学校口罩规定而导致 COVID-19 病例减少的因果关系。他们使用不同的对照组进行了交替分析，以评估先前分析的稳健性：2021至2022学年期间大波士顿地区和马萨诸塞州的学区：学生和学校教职员工：结果：重要的潜在混杂因素使得COVID-19病例率在取消和未取消口罩规定的学区发生了变化：重要的潜在混杂因素导致DiD方法不适合因果推断，这些因素包括先前的免疫力、感染率的时间变化以及检测方法的变化。干预组和对照组的种族构成和收入也有很大差异。与维持口罩要求相比，取消口罩要求与每 1000 人中增加 5.64 例（95% CI，3.00 至 8.29 例）到减少 2.74 例（CI，0.63 至 4.85 例）不等，这取决于对照组的选择以及检查的是学生还是教职员工：局限性：使用的是生态学数据；无法获得所有潜在混杂因素的详细数据：替代分析得出的估计值与取消口罩规定后 COVID-19 病例率的广泛负相关和正相关一致。研究结果凸显了在瞬息万变的环境中，使用生态数据的 DiD 分析来估算不同干预组和对照组的干预效果所面临的挑战：无。
期刊介绍：
Established in 1927 by the American College of Physicians (ACP), Annals of Internal Medicine is the premier internal medicine journal. Annals of Internal Medicine’s mission is to promote excellence in medicine, enable physicians and other health care professionals to be well informed members of the medical community and society, advance standards in the conduct and reporting of medical research, and contribute to improving the health of people worldwide. To achieve this mission, the journal publishes a wide variety of original research, review articles, practice guidelines, and commentary relevant to clinical practice, health care delivery, public health, health care policy, medical education, ethics, and research methodology. In addition, the journal publishes personal narratives that convey the feeling and the art of medicine.
京公网安备 11010802042870号
京ICP备2023020795号-1
分享
求助内容：
应助结果提醒方式：
扫码关注我们
