For regression models where data are obtained from sampling surveies, the statistical analysis is often based on approaches that are either non-robust or inefficient. The handling of survey data requires more appropriate techniques, as the classical methods usually result in biased and inefficient estimates of the underlying model parameters. This article is concerned with the development of a new approach of obtaining robust and efficient estimates of regression model parameters when dealing with survey sampling data. Asymptotic properties of such estimators are established under mild regularity conditions. To demonstrate the performance of the proposed method, Monte Carlo simulation experiments are carried out and show that the estimators obtained from the proposed methodology are robust and more efficient than many of those obtained from existing approaches, mainly if the survey data tend to result in residuals with heavy-tailed or skewed distributions and/or when there are few gross outliers. Finally, the proposed approach is illustrated with a real data example.
{"title":"Rank-Based Inference for Survey Sampling Data","authors":"A. Adekpedjou, H. Bindele","doi":"10.1093/jssam/smab019","DOIUrl":"https://doi.org/10.1093/jssam/smab019","url":null,"abstract":"\u0000 For regression models where data are obtained from sampling surveies, the statistical analysis is often based on approaches that are either non-robust or inefficient. The handling of survey data requires more appropriate techniques, as the classical methods usually result in biased and inefficient estimates of the underlying model parameters. This article is concerned with the development of a new approach of obtaining robust and efficient estimates of regression model parameters when dealing with survey sampling data. Asymptotic properties of such estimators are established under mild regularity conditions. To demonstrate the performance of the proposed method, Monte Carlo simulation experiments are carried out and show that the estimators obtained from the proposed methodology are robust and more efficient than many of those obtained from existing approaches, mainly if the survey data tend to result in residuals with heavy-tailed or skewed distributions and/or when there are few gross outliers. Finally, the proposed approach is illustrated with a real data example.","PeriodicalId":17146,"journal":{"name":"Journal of Survey Statistics and Methodology","volume":" ","pages":""},"PeriodicalIF":2.1,"publicationDate":"2021-08-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"44965476","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Text answers to open-ended questions are typically manually coded into one of several codes. Usually, a random subset of text answers is double-coded to assess intercoder reliability, but most of the data remain single-coded. Any disagreement between the two coders points to an error by one of the coders. When the budget allows double coding additional text answers, we propose employing statistical learning models to predict which single-coded answers have a high risk of a coding error. Specifically, we train a model on the double-coded random subset and predict the probability that the single-coded codes are correct. Then, text answers with the highest risk are double-coded to verify. In experiments with three data sets, we found that this method identifies two to three times as many coding errors in the additional text answers as compared to random guessing, on average. We conclude that this method is preferred if the budget permits additional double-coding. When there are a lot of intercoder disagreements, the benefit can be substantial.
{"title":"A Model-Assisted Approach for Finding Coding Errors in Manual Coding of Open-Ended Questions","authors":"Zhoushanyue He, Matthias Schonlau","doi":"10.1093/jssam/smab022","DOIUrl":"https://doi.org/10.1093/jssam/smab022","url":null,"abstract":"\u0000 Text answers to open-ended questions are typically manually coded into one of several codes. Usually, a random subset of text answers is double-coded to assess intercoder reliability, but most of the data remain single-coded. Any disagreement between the two coders points to an error by one of the coders. When the budget allows double coding additional text answers, we propose employing statistical learning models to predict which single-coded answers have a high risk of a coding error. Specifically, we train a model on the double-coded random subset and predict the probability that the single-coded codes are correct. Then, text answers with the highest risk are double-coded to verify. In experiments with three data sets, we found that this method identifies two to three times as many coding errors in the additional text answers as compared to random guessing, on average. We conclude that this method is preferred if the budget permits additional double-coding. When there are a lot of intercoder disagreements, the benefit can be substantial.","PeriodicalId":17146,"journal":{"name":"Journal of Survey Statistics and Methodology","volume":" ","pages":""},"PeriodicalIF":2.1,"publicationDate":"2021-08-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"45517153","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Enrijeta Shino, Michael D. Martinez, Michael Binder
With the increasing usage of dual-mode data collection, researchers of public opinion have shown considerable interest in understanding response differences across different interview modes. Are mode effects an outcome of representation or measurement differences across modes? We conducted a dual-mode survey (web and telephone) using Florida’s voter file as the sampling frame, randomly assigning registered voters into one mode versus the other. Having a priori information about the respondents allows us to gauge whether and how sample composition differences may be driven by mode effects, and whether mode affects estimated models of political behavior. Survey mode effects are still significant for issue voting even when sampling design is similar for both modes.
{"title":"DETERMINED BY MODE? REPRESENTATION AND MEASUREMENT EFFECTS IN A DUAL-MODE STATEWIDE SURVEY","authors":"Enrijeta Shino, Michael D. Martinez, Michael Binder","doi":"10.1093/jssam/smab012","DOIUrl":"https://doi.org/10.1093/jssam/smab012","url":null,"abstract":"\u0000 With the increasing usage of dual-mode data collection, researchers of public opinion have shown considerable interest in understanding response differences across different interview modes. Are mode effects an outcome of representation or measurement differences across modes? We conducted a dual-mode survey (web and telephone) using Florida’s voter file as the sampling frame, randomly assigning registered voters into one mode versus the other. Having a priori information about the respondents allows us to gauge whether and how sample composition differences may be driven by mode effects, and whether mode affects estimated models of political behavior. Survey mode effects are still significant for issue voting even when sampling design is similar for both modes.","PeriodicalId":17146,"journal":{"name":"Journal of Survey Statistics and Methodology","volume":" ","pages":""},"PeriodicalIF":2.1,"publicationDate":"2021-06-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"45317436","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Response burden has been a concern in survey research for some time. One area of concern is the negative impact that response burden can have on response rates. In an effort to mitigate negative impacts on response rates, survey research organizations try to minimize the burden respondents are exposed to and maximize the likelihood of response. Many organizations also try to be mindful of the role burden may play in respondents’ likelihood to participate in future surveys by implementing rest periods or survey holidays. Recently, new evidence from a study of cross-sectional household surveys provided an interesting lens to examine burden. The evidence demonstrated that those sampled in two independent surveys are more likely to respond to the second survey if the first survey was more difficult to complete, and that this effect was not significantly influenced by the rest period in between the two surveys. These findings are compelling, and since the mechanisms influencing response in household and establishment surveys differ in important ways, a similar examination in an establishment survey context is warranted. To accomplish this, data are used from the National Agricultural Statistics Service. Overall, our research finds that prior survey features such as questionnaire complexity (or burden), prior response disposition and rest period are significantly associated with response to subsequent surveys. We also find that sample units first receiving a more complex questionnaire have significantly higher probabilities of response to a subsequent survey than do those receiving a simpler questionnaire first. The findings in this paper have implications for nonresponse adjustments and identification of subgroups for adaptive design data collection.
{"title":"QUESTIONNAIRE COMPLEXITY, REST PERIOD, AND RESPONSE LIKELIHOOD IN ESTABLISHMENT SURVEYS","authors":"J. Rodhouse, T. Wilson, Heather E Ridolfo","doi":"10.1093/JSSAM/SMAB017","DOIUrl":"https://doi.org/10.1093/JSSAM/SMAB017","url":null,"abstract":"\u0000 Response burden has been a concern in survey research for some time. One area of concern is the negative impact that response burden can have on response rates. In an effort to mitigate negative impacts on response rates, survey research organizations try to minimize the burden respondents are exposed to and maximize the likelihood of response. Many organizations also try to be mindful of the role burden may play in respondents’ likelihood to participate in future surveys by implementing rest periods or survey holidays. Recently, new evidence from a study of cross-sectional household surveys provided an interesting lens to examine burden. The evidence demonstrated that those sampled in two independent surveys are more likely to respond to the second survey if the first survey was more difficult to complete, and that this effect was not significantly influenced by the rest period in between the two surveys. These findings are compelling, and since the mechanisms influencing response in household and establishment surveys differ in important ways, a similar examination in an establishment survey context is warranted. To accomplish this, data are used from the National Agricultural Statistics Service. Overall, our research finds that prior survey features such as questionnaire complexity (or burden), prior response disposition and rest period are significantly associated with response to subsequent surveys. We also find that sample units first receiving a more complex questionnaire have significantly higher probabilities of response to a subsequent survey than do those receiving a simpler questionnaire first. The findings in this paper have implications for nonresponse adjustments and identification of subgroups for adaptive design data collection.","PeriodicalId":17146,"journal":{"name":"Journal of Survey Statistics and Methodology","volume":" ","pages":""},"PeriodicalIF":2.1,"publicationDate":"2021-06-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"41839493","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Katharina Meitinger, A. Toroslu, Klara Raiber, Michael Braun
Web probing is a valuable tool to assess the validity and comparability of survey items. It uses different probe types—such as category-selection probes and specific probes—to inquire about different aspects of an item. Previous web probing studies often asked one probe type per item, but research situations exist where it might be preferable to test potentially problematic items with multiple probes. However, the response behavior might be affected by two factors: question order and the visual presentation of probes on one screen versus multiple screens as well as their interaction. In this study, we report evidence from a web experiment that was conducted with 532 respondents from Germany in September 2013. Experimental groups varied by screen number (1 versus 2) and probe order (category-selection probe first versus specific probe first). We assessed the impact of these manipulations on several indicators of response quality, probe answer content, and the respondents’ motivation with logistic regressions and two-way ANOVAs. We reveal that multiple mechanisms push response behavior in this context: perceived response burden, the focus of attention, the need for justification, and verbal context effects. We find that response behavior in the condition with two screens and category-selection probe first outperforms all other experimental conditions. We recommend this implementation in all but one scenario: if the goal is to test an item that includes a key term with a potentially too large lexical scope, we recommend starting with a specific probe but on the same screen as the category-selection probe.
{"title":"PERCEIVED BURDEN, FOCUS OF ATTENTION, AND THE URGE TO JUSTIFY: THE IMPACT OF THE NUMBER OF SCREENS AND PROBE ORDER ON THE RESPONSE BEHAVIOR OF PROBING QUESTIONS","authors":"Katharina Meitinger, A. Toroslu, Klara Raiber, Michael Braun","doi":"10.1093/JSSAM/SMAA043","DOIUrl":"https://doi.org/10.1093/JSSAM/SMAA043","url":null,"abstract":"\u0000 Web probing is a valuable tool to assess the validity and comparability of survey items. It uses different probe types—such as category-selection probes and specific probes—to inquire about different aspects of an item. Previous web probing studies often asked one probe type per item, but research situations exist where it might be preferable to test potentially problematic items with multiple probes. However, the response behavior might be affected by two factors: question order and the visual presentation of probes on one screen versus multiple screens as well as their interaction. In this study, we report evidence from a web experiment that was conducted with 532 respondents from Germany in September 2013. Experimental groups varied by screen number (1 versus 2) and probe order (category-selection probe first versus specific probe first). We assessed the impact of these manipulations on several indicators of response quality, probe answer content, and the respondents’ motivation with logistic regressions and two-way ANOVAs. We reveal that multiple mechanisms push response behavior in this context: perceived response burden, the focus of attention, the need for justification, and verbal context effects. We find that response behavior in the condition with two screens and category-selection probe first outperforms all other experimental conditions. We recommend this implementation in all but one scenario: if the goal is to test an item that includes a key term with a potentially too large lexical scope, we recommend starting with a specific probe but on the same screen as the category-selection probe.","PeriodicalId":17146,"journal":{"name":"Journal of Survey Statistics and Methodology","volume":" ","pages":""},"PeriodicalIF":2.1,"publicationDate":"2021-06-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"45424242","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Respondents who break off from a web survey prior to completing it are a prevalent problem in data collection. To prevent breakoff bias, it is crucial to keep as many diverse respondents in a web survey as possible. As a first step of preventing breakoffs, this study aims to understand breakoff and the associated response behavior. We analyze data from an annual online survey using dynamic survival models and ROC analyses. We find that breakoff risks between respondents using mobile devices versus PCs do not differ at the beginning of the questionnaire, but the risk for mobile device users increases as the survey progresses. Very fast respondents as well as respondents with changing response times both have a higher risk of quitting the questionnaire, compared to respondents with slower and steady response times. We conclude with a discussion of the implications of these findings for future practice and research in web survey methodology.
{"title":"A DYNAMIC SURVIVAL MODELING APPROACH TO THE PREDICTION OF WEB SURVEY BREAKOFF","authors":"Felicitas Mittereder, B. West","doi":"10.1093/JSSAM/SMAB015","DOIUrl":"https://doi.org/10.1093/JSSAM/SMAB015","url":null,"abstract":"\u0000 Respondents who break off from a web survey prior to completing it are a prevalent problem in data collection. To prevent breakoff bias, it is crucial to keep as many diverse respondents in a web survey as possible. As a first step of preventing breakoffs, this study aims to understand breakoff and the associated response behavior. We analyze data from an annual online survey using dynamic survival models and ROC analyses. We find that breakoff risks between respondents using mobile devices versus PCs do not differ at the beginning of the questionnaire, but the risk for mobile device users increases as the survey progresses. Very fast respondents as well as respondents with changing response times both have a higher risk of quitting the questionnaire, compared to respondents with slower and steady response times. We conclude with a discussion of the implications of these findings for future practice and research in web survey methodology.","PeriodicalId":17146,"journal":{"name":"Journal of Survey Statistics and Methodology","volume":" ","pages":""},"PeriodicalIF":2.1,"publicationDate":"2021-06-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"46746139","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Jill A Dever, Ashley Amaya, Anup Srivastav, Peng-Jun Lu, Jessica Roycroft, Marshica Stanley, M Christopher Stringer, Michael G Bostwick, Stacie M Greby, Tammy A Santibanez, Walter W Williams
Researchers strive to design and implement high-quality surveys to maximize the utility of the data collected. The definitions of quality and usefulness, however, vary from survey to survey and depend on the analytic needs. Survey teams must evaluate the trade-offs of various decisions, such as when results are needed and their required level of precision, in addition to practical constraints like budget, before finalizing the design. Characteristics within the concept of fit for purpose (FfP) can provide the framework for considering the trade-offs. Furthermore, this tool can enable an evaluation of quality for the resulting estimates. Implementation of a FfP framework in this context, however, is not straightforward. In this article, we provide the reader with a glimpse of a FfP framework in action for obtaining estimates on early season influenza vaccination coverage estimates and on knowledge, attitudes, behaviors, and barriers related to influenza and influenza prevention among civilian noninstitutionalized adults aged 18 years and older in the United States. The result is the National Internet Flu Survey (NIFS), an annual, two-week internet survey sponsored by the US Centers for Disease Control and Prevention. In addition to critical design decisions, we use the established NIFS FfP framework to discuss the quality of the NIFS in meeting the intended objectives. We highlight aspects that work well and other survey traits requiring further evaluation. Differences found in comparing the NIFS to the National Flu Survey, the National Health Interview Survey, and Behavioral Risk Factor Surveillance System are discussed via their respective FfP characteristics. The findings presented here highlight the importance of the FfP framework for designing surveys, defining data quality, and providing a set a metrics used to advertise the intended use of the survey data and results.
{"title":"FIT FOR PURPOSE IN ACTION: DESIGN, IMPLEMENTATION, AND EVALUATION OF THE NATIONAL INTERNET FLU SURVEY.","authors":"Jill A Dever, Ashley Amaya, Anup Srivastav, Peng-Jun Lu, Jessica Roycroft, Marshica Stanley, M Christopher Stringer, Michael G Bostwick, Stacie M Greby, Tammy A Santibanez, Walter W Williams","doi":"10.1093/jssam/smz050","DOIUrl":"10.1093/jssam/smz050","url":null,"abstract":"<p><p>Researchers strive to design and implement high-quality surveys to maximize the utility of the data collected. The definitions of quality and usefulness, however, vary from survey to survey and depend on the analytic needs. Survey teams must evaluate the trade-offs of various decisions, such as when results are needed and their required level of precision, in addition to practical constraints like budget, before finalizing the design. Characteristics within the concept of fit for purpose (FfP) can provide the framework for considering the trade-offs. Furthermore, this tool can enable an evaluation of quality for the resulting estimates. Implementation of a FfP framework in this context, however, is not straightforward. In this article, we provide the reader with a glimpse of a FfP framework in action for obtaining estimates on early season influenza vaccination coverage estimates and on knowledge, attitudes, behaviors, and barriers related to influenza and influenza prevention among civilian noninstitutionalized adults aged 18 years and older in the United States. The result is the National Internet Flu Survey (NIFS), an annual, two-week internet survey sponsored by the US Centers for Disease Control and Prevention. In addition to critical design decisions, we use the established NIFS FfP framework to discuss the quality of the NIFS in meeting the intended objectives. We highlight aspects that work well and other survey traits requiring further evaluation. Differences found in comparing the NIFS to the National Flu Survey, the National Health Interview Survey, and Behavioral Risk Factor Surveillance System are discussed via their respective FfP characteristics. The findings presented here highlight the importance of the FfP framework for designing surveys, defining data quality, and providing a set a metrics used to advertise the intended use of the survey data and results.</p>","PeriodicalId":17146,"journal":{"name":"Journal of Survey Statistics and Methodology","volume":" ","pages":"449-476"},"PeriodicalIF":1.6,"publicationDate":"2021-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9434706/pdf/nihms-1800712.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"40349807","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
J. Wesołowski, R. Wieczorkowski, Wojciech W'ojciak
We derive a formula for the optimal sample allocation in a general stratified scheme under upper bounds on the sample strata sizes. Such a general scheme includes SRSWOR within strata as a special case. The solution is given in terms of V allocation with V being the set of take-all strata. We use V allocation to give a formal proof of optimality of the popular recursive Neyman algorithm, rNa. We also propose a quick proof of optimality of the algorithm of Stenger and Gabler, SGa, as well as of our proposed modification, coma. Finally, we compare running times of rNa, SGa, and coma. Ready-to-use R-implementations of these algorithms are available on CRAN repository at https://cran.r-project.org/web/packages/stratallo.
{"title":"Optimality of the Recursive Neyman Allocation","authors":"J. Wesołowski, R. Wieczorkowski, Wojciech W'ojciak","doi":"10.1093/jssam/smab018","DOIUrl":"https://doi.org/10.1093/jssam/smab018","url":null,"abstract":"\u0000 We derive a formula for the optimal sample allocation in a general stratified scheme under upper bounds on the sample strata sizes. Such a general scheme includes SRSWOR within strata as a special case. The solution is given in terms of V allocation with V being the set of take-all strata. We use V allocation to give a formal proof of optimality of the popular recursive Neyman algorithm, rNa. We also propose a quick proof of optimality of the algorithm of Stenger and Gabler, SGa, as well as of our proposed modification, coma. Finally, we compare running times of rNa, SGa, and coma. Ready-to-use R-implementations of these algorithms are available on CRAN repository at https://cran.r-project.org/web/packages/stratallo.","PeriodicalId":17146,"journal":{"name":"Journal of Survey Statistics and Methodology","volume":" ","pages":""},"PeriodicalIF":2.1,"publicationDate":"2021-05-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"43743867","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Analyzing trends in public opinion is important for monitoring social change and for testing theories aimed at explaining this change. With growing availability of multi-wave surveys, social scientists are increasingly turning to latent trend models applied to survey data for examining changes in social and political attitudes. With the aim of facilitating this research, our study compares different approaches to modeling latent trends of aggregate public opinion: splines, Gaussian processes, and discrete autoregressive models. We examine the ability of these models to recover latent trends with simulated data that vary with regard to the frequency and magnitude of changes in the true trend, model complexity and data availability. Overall, we find that all three latent trend models perform well in all scenarios, even the most difficult ones with frequent and weak changes of the latent trend and sparse data. The two main performance differences we find include the relatively higher squared errors of autoregressive models compared to the other models, and the under-coverage of posterior intervals in high-frequency low-amplitude trends with splines. For all models and across all scenarios performance improves with increased data availability, which emphasizes the need of supplying sufficient data for accurate estimation of latent trends.
{"title":"Modeling public opinion over time: A simulation study of latent trend models","authors":"M. Kołczyńska, P. Bürkner","doi":"10.31235/osf.io/gauvx","DOIUrl":"https://doi.org/10.31235/osf.io/gauvx","url":null,"abstract":"Analyzing trends in public opinion is important for monitoring social change and for testing theories aimed at explaining this change. With growing availability of multi-wave surveys, social scientists are increasingly turning to latent trend models applied to survey data for examining changes in social and political attitudes. With the aim of facilitating this research, our study compares different approaches to modeling latent trends of aggregate public opinion: splines, Gaussian processes, and discrete autoregressive models. We examine the ability of these models to recover latent trends with simulated data that vary with regard to the frequency and magnitude of changes in the true trend, model complexity and data availability. Overall, we find that all three latent trend models perform well in all scenarios, even the most difficult ones with frequent and weak changes of the latent trend and sparse data. The two main performance differences we find include the relatively higher squared errors of autoregressive models compared to the other models, and the under-coverage of posterior intervals in high-frequency low-amplitude trends with splines. For all models and across all scenarios performance improves with increased data availability, which emphasizes the need of supplying sufficient data for accurate estimation of latent trends.","PeriodicalId":17146,"journal":{"name":"Journal of Survey Statistics and Methodology","volume":" ","pages":""},"PeriodicalIF":2.1,"publicationDate":"2021-05-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"49120772","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Survey researchers are often confronted with the question of how long to set the length of the field period. Longer fielding time might lead to greater participation yet requires survey managers to devote more of their time to data collection efforts. With the aim of facilitating the decision about the length of the field period, we investigated whether a longer fielding time reduces the risk of nonresponse bias to judge whether field periods can be ended earlier without endangering the performance of the survey. By using data from six waves of a probability-based mixed-mode (online and mail) panel of the German population, we analyzed whether the risk of nonresponse bias decreases over the field period by investigating how day-by-day coefficients of variation develop during the field period. We then determined the optimal cut-off points for each mode after which data collection can be terminated without increasing the risk of nonresponse bias and found that the optimal cut-off points differ by mode. Our study complements prior research by shifting the perspective in the investigation of the risk of nonresponse bias to panel data as well as to mixed-mode surveys, in particular. Our proposed method of using coefficients of variation to assess whether the risk of nonresponse bias decreases significantly with each additional day of fieldwork can aid survey practitioners in finding the optimal field period for their mixed-mode surveys.
{"title":"Risk of Nonresponse Bias and the Length of the Field Period in a Mixed-Mode General Population Panel","authors":"Bella Struminskaya, Tobias Gummer","doi":"10.1093/JSSAM/SMAB011","DOIUrl":"https://doi.org/10.1093/JSSAM/SMAB011","url":null,"abstract":"\u0000 Survey researchers are often confronted with the question of how long to set the length of the field period. Longer fielding time might lead to greater participation yet requires survey managers to devote more of their time to data collection efforts. With the aim of facilitating the decision about the length of the field period, we investigated whether a longer fielding time reduces the risk of nonresponse bias to judge whether field periods can be ended earlier without endangering the performance of the survey. By using data from six waves of a probability-based mixed-mode (online and mail) panel of the German population, we analyzed whether the risk of nonresponse bias decreases over the field period by investigating how day-by-day coefficients of variation develop during the field period. We then determined the optimal cut-off points for each mode after which data collection can be terminated without increasing the risk of nonresponse bias and found that the optimal cut-off points differ by mode. Our study complements prior research by shifting the perspective in the investigation of the risk of nonresponse bias to panel data as well as to mixed-mode surveys, in particular. Our proposed method of using coefficients of variation to assess whether the risk of nonresponse bias decreases significantly with each additional day of fieldwork can aid survey practitioners in finding the optimal field period for their mixed-mode surveys.","PeriodicalId":17146,"journal":{"name":"Journal of Survey Statistics and Methodology","volume":" ","pages":""},"PeriodicalIF":2.1,"publicationDate":"2021-04-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1093/JSSAM/SMAB011","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"45924091","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：