Lessons Learned on Observed-to-Expected Analysis Using Spontaneous Reports During Mass Vaccination

Abstract

During the COVID-19 vaccination campaign, observed-to-expected analysis was used by the European Medicines Agency to contextualise data from spontaneous reports to generate real-time evidence on emerging safety concerns that may impact the benefit-risk profile of COVID-19 vaccines. Observed-to-expected analysis compares the number of cases spontaneously reported for an event of interest after vaccination (‘observed’) to the ‘expected’ number of cases anticipated to occur in the same number of individuals had they not been vaccinated. Observed-to-expected analysis is a robust methodology that relies on several assumptions that have been described in regulatory guidelines and scientific literature. The use of observed-to-expected analysis to support the safety monitoring of COVID-19 vaccines has provided valuable insights and lessons on its design and interpretability, which could prove to be beneficial in future analyses. When undertaking an observed-to-expected analysis within the context of safety monitoring, several aspects need attention. In particular, we emphasise the importance of stratified and harmonised data collection both for vaccine exposure and spontaneous reporting data, the need for alignment between coding dictionaries and the crucial role of accurate background incidence rates for adverse events of special interest. While these considerations and recommendations were determined in the context of the COVID-19 mass vaccination setting, they are generalisable in principle.