Stephanie Coffey, Olga Maslovskaya, Cameron McPhee
The use of mixed-mode surveys has grown rapidly in recent years, due to both technological advances and the COVID-19 pandemic. The increased mixing of modes (and the adoption of newer digital modes like web and text messaging) necessitates an evaluation of the impact of these newer designs on survey errors and costs, as well as new techniques for disaggregating and adjusting for nonresponse and measurement errors. This special issue highlights recent innovations, applications, and evaluations of mixed-mode survey designs and identifies areas where additional research is required.
{"title":"Recent Innovations and Advances in Mixed-Mode Surveys","authors":"Stephanie Coffey, Olga Maslovskaya, Cameron McPhee","doi":"10.1093/jssam/smae025","DOIUrl":"https://doi.org/10.1093/jssam/smae025","url":null,"abstract":"\u0000 The use of mixed-mode surveys has grown rapidly in recent years, due to both technological advances and the COVID-19 pandemic. The increased mixing of modes (and the adoption of newer digital modes like web and text messaging) necessitates an evaluation of the impact of these newer designs on survey errors and costs, as well as new techniques for disaggregating and adjusting for nonresponse and measurement errors. This special issue highlights recent innovations, applications, and evaluations of mixed-mode survey designs and identifies areas where additional research is required.","PeriodicalId":17146,"journal":{"name":"Journal of Survey Statistics and Methodology","volume":null,"pages":null},"PeriodicalIF":2.1,"publicationDate":"2024-06-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141365619","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Anagha Tolpadi, L. Parast, Marc N Elliott, Ann Haas, Melissa A Bradley, Joshua Wolf, Joan M Teno, Maria DeYoreo, Lauren Fuentes, Rebecca Anhang Price
Patient experience surveys are vital to evaluating healthcare provider performance. However, declining response rates over time and questions about whether responses reflect the perspectives of all patients under care have raised concerns. One proposed approach to address these concerns is web-based survey administration, a mode that has not been studied in the hospice setting. We tested a sequential web–mail mode for administering a care experience survey in this unique setting, where family caregivers respond after the patient dies. Sampled caregivers of 15,515 patients who died March–August 2021 while receiving care from 56 hospices across the US were randomized to one of four survey modes: mail-only, telephone-only, mail–telephone (mail with telephone follow-up), or web–mail (email invitation to a web survey with mail follow-up). Email addresses were available for 31.3 percent of sampled eligible caregivers. Relative to mail-only (estimated response rate = 35.1 percent), response rates were significantly higher for web–mail (39.7 percent) and mail–telephone (45.3 percent) and significantly lower for telephone-only (31.5 percent). The web–mail response rate was similar to the mail-only response rate among caregivers without email addresses (35.2 versus 34.3 percent), but substantially higher among caregivers with email addresses (49.6 versus 36.7 percent). Web–mail and mail-only respondents reported similar experiences for 26 of 27 evaluative items. Among eligible sampled caregivers, several patient/caregiver characteristics differed by caregivers’ email address availability, but web–mail and mail-only respondents did not differ on any characteristic. A web–mail mode is feasible for surveying bereaved caregivers about care experiences, producing substantially higher response rates than single-mode approaches, with increasing benefits for hospices with higher proportions of caregivers with email addresses. Findings may be applicable to surveys of other sensitive topics and to populations that prefer asynchronous survey modes.
{"title":"Effects of a Web–Mail Mode on Response Rates and Responses to a Care Experience Survey: Results of a Randomized Experiment","authors":"Anagha Tolpadi, L. Parast, Marc N Elliott, Ann Haas, Melissa A Bradley, Joshua Wolf, Joan M Teno, Maria DeYoreo, Lauren Fuentes, Rebecca Anhang Price","doi":"10.1093/jssam/smae013","DOIUrl":"https://doi.org/10.1093/jssam/smae013","url":null,"abstract":"\u0000 Patient experience surveys are vital to evaluating healthcare provider performance. However, declining response rates over time and questions about whether responses reflect the perspectives of all patients under care have raised concerns. One proposed approach to address these concerns is web-based survey administration, a mode that has not been studied in the hospice setting. We tested a sequential web–mail mode for administering a care experience survey in this unique setting, where family caregivers respond after the patient dies. Sampled caregivers of 15,515 patients who died March–August 2021 while receiving care from 56 hospices across the US were randomized to one of four survey modes: mail-only, telephone-only, mail–telephone (mail with telephone follow-up), or web–mail (email invitation to a web survey with mail follow-up). Email addresses were available for 31.3 percent of sampled eligible caregivers.\u0000 Relative to mail-only (estimated response rate = 35.1 percent), response rates were significantly higher for web–mail (39.7 percent) and mail–telephone (45.3 percent) and significantly lower for telephone-only (31.5 percent). The web–mail response rate was similar to the mail-only response rate among caregivers without email addresses (35.2 versus 34.3 percent), but substantially higher among caregivers with email addresses (49.6 versus 36.7 percent). Web–mail and mail-only respondents reported similar experiences for 26 of 27 evaluative items. Among eligible sampled caregivers, several patient/caregiver characteristics differed by caregivers’ email address availability, but web–mail and mail-only respondents did not differ on any characteristic. A web–mail mode is feasible for surveying bereaved caregivers about care experiences, producing substantially higher response rates than single-mode approaches, with increasing benefits for hospices with higher proportions of caregivers with email addresses. Findings may be applicable to surveys of other sensitive topics and to populations that prefer asynchronous survey modes.","PeriodicalId":17146,"journal":{"name":"Journal of Survey Statistics and Methodology","volume":null,"pages":null},"PeriodicalIF":2.1,"publicationDate":"2024-04-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140666578","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
The integration of multiple survey, administrative, and third-party data offers the opportunity to innovate and improve survey estimation via statistical modeling. With decreasing response rates and increasing interest for more timely and geographically detailed estimates, imputation methodology that combines multiple data sources to adjust for low unit response and allow for more detailed publication levels, including geographic estimates, is both timely and necessary. Motivated by the Advance Monthly Retail Trade Survey (MARTS) and Monthly Retail Trade Survey (MRTS), we propose Bayesian hierarchical multiple imputation-dependent data models with the goals of automating imputation for the MARTS by using historic MRTS data and providing geographically granular (state-level) estimates for the MRTS via mass imputation using third-party data and spatial dependence. As a natural byproduct of this approach, measures of uncertainty are provided. This article illustrates the advantages of applying established Bayesian hierarchical modeling techniques with multiple source data to address practical problems in official statistics and is, therefore, of independent interest. The motivating empirical studies are unified by their hierarchical modeling framework, which ultimately results in a more principled approach for estimation for the MARTS and a more geographically granular data product for the MRTS.
{"title":"Bayesian Multisource Hierarchical Models with Applications to the Monthly Retail Trade Survey","authors":"Stephen J Kaputa, Darcy Steeg Morris, S. Holan","doi":"10.1093/jssam/smae019","DOIUrl":"https://doi.org/10.1093/jssam/smae019","url":null,"abstract":"\u0000 The integration of multiple survey, administrative, and third-party data offers the opportunity to innovate and improve survey estimation via statistical modeling. With decreasing response rates and increasing interest for more timely and geographically detailed estimates, imputation methodology that combines multiple data sources to adjust for low unit response and allow for more detailed publication levels, including geographic estimates, is both timely and necessary. Motivated by the Advance Monthly Retail Trade Survey (MARTS) and Monthly Retail Trade Survey (MRTS), we propose Bayesian hierarchical multiple imputation-dependent data models with the goals of automating imputation for the MARTS by using historic MRTS data and providing geographically granular (state-level) estimates for the MRTS via mass imputation using third-party data and spatial dependence. As a natural byproduct of this approach, measures of uncertainty are provided. This article illustrates the advantages of applying established Bayesian hierarchical modeling techniques with multiple source data to address practical problems in official statistics and is, therefore, of independent interest. The motivating empirical studies are unified by their hierarchical modeling framework, which ultimately results in a more principled approach for estimation for the MARTS and a more geographically granular data product for the MRTS.","PeriodicalId":17146,"journal":{"name":"Journal of Survey Statistics and Methodology","volume":null,"pages":null},"PeriodicalIF":2.1,"publicationDate":"2024-04-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140694201","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Eliud Kibuchi, Patrick Sturgis, Gabriele B. Durrant, Olga Maslovskaya
Effective evaluation of data quality between data collected in different modes is complicated by the confounding of selection and measurement effects. This study evaluates the utility of propensity score matching (PSM) as a method that has been proposed as a means of removing selection effects across surveys conducted in different modes. Our results show large differences in estimates for the same variables between parallel face-to-face and online surveys, even after matching on standard demographic variables. Moreover, discrepancies in estimates are still present after matching between surveys conducted in the same (online) mode, where differences in measurement properties can be ruled out a priori. Our findings suggest that PSM has substantial limitations as a method for separating measurement and selection differences across modes and should be used only with caution.
{"title":"The efficacy of propensity score matching for separating selection and measurement effects across different survey modes","authors":"Eliud Kibuchi, Patrick Sturgis, Gabriele B. Durrant, Olga Maslovskaya","doi":"10.1093/jssam/smae017","DOIUrl":"https://doi.org/10.1093/jssam/smae017","url":null,"abstract":"\u0000 Effective evaluation of data quality between data collected in different modes is complicated by the confounding of selection and measurement effects. This study evaluates the utility of propensity score matching (PSM) as a method that has been proposed as a means of removing selection effects across surveys conducted in different modes. Our results show large differences in estimates for the same variables between parallel face-to-face and online surveys, even after matching on standard demographic variables. Moreover, discrepancies in estimates are still present after matching between surveys conducted in the same (online) mode, where differences in measurement properties can be ruled out a priori. Our findings suggest that PSM has substantial limitations as a method for separating measurement and selection differences across modes and should be used only with caution.","PeriodicalId":17146,"journal":{"name":"Journal of Survey Statistics and Methodology","volume":null,"pages":null},"PeriodicalIF":2.1,"publicationDate":"2024-04-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140692979","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Due to rising costs and declining response rates, surveys are increasingly moving from face-to-face interviewing to a self-administered mixed-mode design. Mixed-mode surveys can be conducted using a concurrent or a sequential design. A sequential design in which the web mode is offered first is a common strategy for mixed-mode surveys as it reduces survey costs. However, when deciding which mode choice sequence to use, sample balance should also be considered. One approach to achieving a balanced sample might be to tailor the sequence of the choice of modes, or the mode choice sequence. For this purpose, we use an indicator that assigns the sampled persons to the different mode choice sequences to minimize the variability of response probabilities. In this study, we compare the sample composition achieved with a concurrent and a sequential design. Additionally, we investigate whether indicator-based tailoring of the two mode choice sequences can improve sample composition. We implemented a randomized experiment in the 2021 German General Social Survey (ALLBUS), which surveyed the general population aged 18 and older in private households (N = 5,342) using a mixed-mode design (web and mail). In a first step, respondents were randomly assigned to a concurrent or a sequential design. We find that the two mode choice sequences lead to a similar sample composition. Next, we identify age as the best available single indicator of the variables known before the survey to tailor the mode choice sequence. Our analyses show that a tailored approach based on age improves the sample composition slightly.
{"title":"Sequential and Concurrent Mixed-Mode Designs: A Tailored Approach","authors":"Alexandra Asimov, Michael Blohm","doi":"10.1093/jssam/smae016","DOIUrl":"https://doi.org/10.1093/jssam/smae016","url":null,"abstract":"\u0000 Due to rising costs and declining response rates, surveys are increasingly moving from face-to-face interviewing to a self-administered mixed-mode design. Mixed-mode surveys can be conducted using a concurrent or a sequential design. A sequential design in which the web mode is offered first is a common strategy for mixed-mode surveys as it reduces survey costs. However, when deciding which mode choice sequence to use, sample balance should also be considered. One approach to achieving a balanced sample might be to tailor the sequence of the choice of modes, or the mode choice sequence. For this purpose, we use an indicator that assigns the sampled persons to the different mode choice sequences to minimize the variability of response probabilities. In this study, we compare the sample composition achieved with a concurrent and a sequential design. Additionally, we investigate whether indicator-based tailoring of the two mode choice sequences can improve sample composition. We implemented a randomized experiment in the 2021 German General Social Survey (ALLBUS), which surveyed the general population aged 18 and older in private households (N = 5,342) using a mixed-mode design (web and mail). In a first step, respondents were randomly assigned to a concurrent or a sequential design. We find that the two mode choice sequences lead to a similar sample composition. Next, we identify age as the best available single indicator of the variables known before the survey to tailor the mode choice sequence. Our analyses show that a tailored approach based on age improves the sample composition slightly.","PeriodicalId":17146,"journal":{"name":"Journal of Survey Statistics and Methodology","volume":null,"pages":null},"PeriodicalIF":2.1,"publicationDate":"2024-04-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140728165","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Yang Ha Cho, María Guadarrama-Sanz, Isabel Molina, A. Eideh, Emily Berg
Two challenges in small area estimation occur when (i) the sample selection mechanism depends on the outcome variable and (ii) the parameter of interest is a nonlinear function of the response variable in the assumed model. If, given the values of the model covariates, the sample selection mechanism depends on the model response variable, the design is said to be informative for the model. Pfeffermann and Sverchkov (2007) develop a small area estimation procedure for informative sampling, focusing on the prediction of small area means. Molina and Rao (2010) develop a small area estimation procedure for general parameters that are nonlinear functions of the model response variable. The method of Molina and Rao assumes noninformative sampling. We combine these two approaches to develop a procedure for the estimation of general parameters in small areas under informative sampling. We introduce a parametric bootstrap MSE estimator that is appropriate for an informative sample design. We evaluate the validity of the proposed procedures through extensive simulation studies and illustrate the procedures utilizing Mexico’s income data.
{"title":"Optimal Predictors of General Small Area Parameters Under an Informative Sample Design Using Parametric Sample Distribution Models","authors":"Yang Ha Cho, María Guadarrama-Sanz, Isabel Molina, A. Eideh, Emily Berg","doi":"10.1093/jssam/smae007","DOIUrl":"https://doi.org/10.1093/jssam/smae007","url":null,"abstract":"\u0000 Two challenges in small area estimation occur when (i) the sample selection mechanism depends on the outcome variable and (ii) the parameter of interest is a nonlinear function of the response variable in the assumed model. If, given the values of the model covariates, the sample selection mechanism depends on the model response variable, the design is said to be informative for the model. Pfeffermann and Sverchkov (2007) develop a small area estimation procedure for informative sampling, focusing on the prediction of small area means. Molina and Rao (2010) develop a small area estimation procedure for general parameters that are nonlinear functions of the model response variable. The method of Molina and Rao assumes noninformative sampling. We combine these two approaches to develop a procedure for the estimation of general parameters in small areas under informative sampling. We introduce a parametric bootstrap MSE estimator that is appropriate for an informative sample design. We evaluate the validity of the proposed procedures through extensive simulation studies and illustrate the procedures utilizing Mexico’s income data.","PeriodicalId":17146,"journal":{"name":"Journal of Survey Statistics and Methodology","volume":null,"pages":null},"PeriodicalIF":2.1,"publicationDate":"2024-03-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140371923","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Barry Schouten, Thomas Klausch, B. Buelens, Jan van den Brakel
Reinterview designs are a potential tool to estimate and adjust for mode measurement effects, that is, relative differences in mode-specific measurement error bias. In 2011, a reinterview design was successfully applied to the Dutch Crime Victimization Survey, which led to a redesign of the survey. Reinterview designs may, however, be very costly, especially when face to face is included as a survey mode. The crucial question is whether benefits outweigh costs, that is, whether the potential increase in the accuracy of survey statistics is worth the investment. The answer to this question depends heavily on the purpose of the reinterview, that is, assessment versus adjustment, the size of the measurement effects, and the relative cost of the modes. Reinterview designs also make a number of assumptions that will not hold for every setting. In this article, we perform a cost–benefit analysis for two surveys, the Dutch Health Survey and the Dutch Labour Force Survey, and discuss the utility and validity of reinterviews. We conclude that a reinterview may not be useful due to relatively small measurement differences for the Labour Force Survey, whereas it may be useful for the Health Survey.
{"title":"A Cost–Benefit Analysis of Reinterview Designs for Estimating and Adjusting Mode Measurement Effects: A Case Study for the Dutch Health Survey and Labour Force Survey","authors":"Barry Schouten, Thomas Klausch, B. Buelens, Jan van den Brakel","doi":"10.1093/jssam/smae011","DOIUrl":"https://doi.org/10.1093/jssam/smae011","url":null,"abstract":"\u0000 Reinterview designs are a potential tool to estimate and adjust for mode measurement effects, that is, relative differences in mode-specific measurement error bias. In 2011, a reinterview design was successfully applied to the Dutch Crime Victimization Survey, which led to a redesign of the survey. Reinterview designs may, however, be very costly, especially when face to face is included as a survey mode. The crucial question is whether benefits outweigh costs, that is, whether the potential increase in the accuracy of survey statistics is worth the investment. The answer to this question depends heavily on the purpose of the reinterview, that is, assessment versus adjustment, the size of the measurement effects, and the relative cost of the modes. Reinterview designs also make a number of assumptions that will not hold for every setting. In this article, we perform a cost–benefit analysis for two surveys, the Dutch Health Survey and the Dutch Labour Force Survey, and discuss the utility and validity of reinterviews. We conclude that a reinterview may not be useful due to relatively small measurement differences for the Labour Force Survey, whereas it may be useful for the Health Survey.","PeriodicalId":17146,"journal":{"name":"Journal of Survey Statistics and Methodology","volume":null,"pages":null},"PeriodicalIF":2.1,"publicationDate":"2024-03-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140373045","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
When deciding which modes to offer, researchers consider cost, known respondent contact information, and potential mode effects. For a short survey on employment, we evaluate the effect of adding one of two new electronic data collection modes to a mailed questionnaire. We sent a survey to principals who previously responded to the National Center for Education Statistics’ (NCES) National Teacher and Principal Survey (NTPS) asking about their current job status. This questionnaire, known as the Principal Follow-up Survey (PFS), has typically been administered as a short paper form that is mailed to NTPS respondents. In 2022, the PFS introduced two new modes of completion, and principals were randomly assigned to receive: (i) a paper form only; (ii) a paper form, as well as emails with a direct link to complete a web survey; or (iii) a paper form, as well as invitations by text message to complete an automated two-way short message service text survey by responding to texted “yes/no” questions. This article compares overall response rates and time-to-response by mode to determine respondent preferences for completing short surveys. Adding either electronic mode significantly increased response rates and decreased the number of days in which completed surveys were received, compared to offering only a paper questionnaire. Although email and text messages are both forms of electronic communication that may be accessible on a smartphone, the added text message survey resulted in higher response rates than the added web survey. This suggests that respondents interact differently with emails and text messages they receive and that offering an option to complete a survey by text message can increase the speed and efficiency of data collection for short surveys.
{"title":"Supplementing a Paper Questionnaire with Web and Two-way Short Message Service (SMS) Surveys","authors":"Maura Spiegelman, Allison Zotti, Julia Merlin","doi":"10.1093/jssam/smae006","DOIUrl":"https://doi.org/10.1093/jssam/smae006","url":null,"abstract":"\u0000 When deciding which modes to offer, researchers consider cost, known respondent contact information, and potential mode effects. For a short survey on employment, we evaluate the effect of adding one of two new electronic data collection modes to a mailed questionnaire. We sent a survey to principals who previously responded to the National Center for Education Statistics’ (NCES) National Teacher and Principal Survey (NTPS) asking about their current job status. This questionnaire, known as the Principal Follow-up Survey (PFS), has typically been administered as a short paper form that is mailed to NTPS respondents. In 2022, the PFS introduced two new modes of completion, and principals were randomly assigned to receive: (i) a paper form only; (ii) a paper form, as well as emails with a direct link to complete a web survey; or (iii) a paper form, as well as invitations by text message to complete an automated two-way short message service text survey by responding to texted “yes/no” questions. This article compares overall response rates and time-to-response by mode to determine respondent preferences for completing short surveys. Adding either electronic mode significantly increased response rates and decreased the number of days in which completed surveys were received, compared to offering only a paper questionnaire. Although email and text messages are both forms of electronic communication that may be accessible on a smartphone, the added text message survey resulted in higher response rates than the added web survey. This suggests that respondents interact differently with emails and text messages they receive and that offering an option to complete a survey by text message can increase the speed and efficiency of data collection for short surveys.","PeriodicalId":17146,"journal":{"name":"Journal of Survey Statistics and Methodology","volume":null,"pages":null},"PeriodicalIF":2.1,"publicationDate":"2024-03-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140385243","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
This article is concerned with the transition of a longitudinal survey from a single-mode design to a web-first mixed-mode design and the role that text messages to sample members can play in smoothing that transition. We present the results of an experiment that investigates the effects of augmenting the contact strategy of letters and emails with text messages, inviting the sample members to complete a web questionnaire and reminding them of the invite. The experiment was conducted in a subsample of Understanding Society, a household panel survey in the United Kingdom, in the wave that transitioned from a CAPI-only design to a sequential design combining web and CATI. In the experiment, a quarter of the sample received letters and emails, while the rest received between one and three text messages with a personalized link to the questionnaire. We examine the effect of the text messages on response rates, both at the web phase of a sequential design and at the end of the fieldwork after a CATI follow-up phase, and explore various mechanisms that might drive the increase in response rates. We also look at the effects on the device used to complete the survey and field efforts needed at the CATI stage. The findings indicate that text messages did not help to significantly increase response rates overall, although some subgroups benefited from them, such as panel members who had not provided an email or postal address before. Likewise, the text messages increased web completion among younger panel members and those with an irregular response pattern. We only found a slight and nonsignificant effect on smartphone use and no effect on the web household response rate, a proxy for fieldwork efforts.
{"title":"Text Messages to Facilitate the Transition to Web-First Sequential Mixed-Mode Designs in Longitudinal Surveys","authors":"Pablo Cabrera-Álvarez, Peter Lynn","doi":"10.1093/jssam/smae003","DOIUrl":"https://doi.org/10.1093/jssam/smae003","url":null,"abstract":"\u0000 This article is concerned with the transition of a longitudinal survey from a single-mode design to a web-first mixed-mode design and the role that text messages to sample members can play in smoothing that transition. We present the results of an experiment that investigates the effects of augmenting the contact strategy of letters and emails with text messages, inviting the sample members to complete a web questionnaire and reminding them of the invite. The experiment was conducted in a subsample of Understanding Society, a household panel survey in the United Kingdom, in the wave that transitioned from a CAPI-only design to a sequential design combining web and CATI. In the experiment, a quarter of the sample received letters and emails, while the rest received between one and three text messages with a personalized link to the questionnaire. We examine the effect of the text messages on response rates, both at the web phase of a sequential design and at the end of the fieldwork after a CATI follow-up phase, and explore various mechanisms that might drive the increase in response rates. We also look at the effects on the device used to complete the survey and field efforts needed at the CATI stage. The findings indicate that text messages did not help to significantly increase response rates overall, although some subgroups benefited from them, such as panel members who had not provided an email or postal address before. Likewise, the text messages increased web completion among younger panel members and those with an irregular response pattern. We only found a slight and nonsignificant effect on smartphone use and no effect on the web household response rate, a proxy for fieldwork efforts.","PeriodicalId":17146,"journal":{"name":"Journal of Survey Statistics and Methodology","volume":null,"pages":null},"PeriodicalIF":2.1,"publicationDate":"2024-03-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140228984","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
In this article, we propose methods to minimize bias due to unit nonresponse. We consider a two-phase sampling design where the second phase is a probability subsample of nonrespondents from the first phase. In this context, we propose three weighting procedures to estimate the total when not all units in the subsample respond. The weighting is based on the response homogeneity group (RHG) model. Given the RHG model, theoretical results on bias and variance estimation are obtained for all estimators. In a simulation study, we evaluate the empirical properties of the three estimators as well as of estimators based on two commonly used procedures to handle unit nonresponse in single-phase sampling design. These two procedures include: (i) nonresponse calibration weighting, also known as the one-step approach, and (ii) nonresponse probability weighting followed by calibration, also known as the two-step approach. Our results indicate that when there is significant deviation from the assumed RHG model, the nonresponse follow-up estimators perform better in terms of bias and coverage.
在本文中，我们提出了一些方法来尽量减少因单位无响应而造成的偏差。我们考虑了两阶段抽样设计，其中第二阶段是第一阶段未应答者的概率子样本。在这种情况下，我们提出了三种加权程序，以便在子样本中并非所有单位都作出回应时估计总数。加权是基于响应同质组（RHG）模型。根据 RHG 模型，我们得到了所有估计器的偏差和方差估计的理论结果。在模拟研究中，我们评估了这三种估计器的经验特性，以及基于两种常用程序的估计器的经验特性，这两种程序用于处理单阶段抽样设计中的单位非响应。这两种程序包括(i) 非响应校准加权，也称为一步法，以及 (ii) 非响应概率加权后再校准，也称为两步法。我们的研究结果表明，当假定的 RHG 模型出现重大偏差时，非响应跟踪估计器在偏差和覆盖率方面表现较好。
{"title":"Estimation of a Population Total Under Nonresponse Using Follow-up","authors":"Marius Stefan, M. Hidiroglou","doi":"10.1093/jssam/smae002","DOIUrl":"https://doi.org/10.1093/jssam/smae002","url":null,"abstract":"\u0000 In this article, we propose methods to minimize bias due to unit nonresponse. We consider a two-phase sampling design where the second phase is a probability subsample of nonrespondents from the first phase. In this context, we propose three weighting procedures to estimate the total when not all units in the subsample respond. The weighting is based on the response homogeneity group (RHG) model. Given the RHG model, theoretical results on bias and variance estimation are obtained for all estimators. In a simulation study, we evaluate the empirical properties of the three estimators as well as of estimators based on two commonly used procedures to handle unit nonresponse in single-phase sampling design. These two procedures include: (i) nonresponse calibration weighting, also known as the one-step approach, and (ii) nonresponse probability weighting followed by calibration, also known as the two-step approach. Our results indicate that when there is significant deviation from the assumed RHG model, the nonresponse follow-up estimators perform better in terms of bias and coverage.","PeriodicalId":17146,"journal":{"name":"Journal of Survey Statistics and Methodology","volume":null,"pages":null},"PeriodicalIF":2.1,"publicationDate":"2024-03-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140254031","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：