Eliud Kibuchi, Patrick Sturgis, Gabriele B. Durrant, Olga Maslovskaya
{"title":"The efficacy of propensity score matching for separating selection and measurement effects across different survey modes","authors":"Eliud Kibuchi, Patrick Sturgis, Gabriele B. Durrant, Olga Maslovskaya","doi":"10.1093/jssam/smae017","DOIUrl":null,"url":null,"abstract":"\n Effective evaluation of data quality between data collected in different modes is complicated by the confounding of selection and measurement effects. This study evaluates the utility of propensity score matching (PSM) as a method that has been proposed as a means of removing selection effects across surveys conducted in different modes. Our results show large differences in estimates for the same variables between parallel face-to-face and online surveys, even after matching on standard demographic variables. Moreover, discrepancies in estimates are still present after matching between surveys conducted in the same (online) mode, where differences in measurement properties can be ruled out a priori. Our findings suggest that PSM has substantial limitations as a method for separating measurement and selection differences across modes and should be used only with caution.","PeriodicalId":17146,"journal":{"name":"Journal of Survey Statistics and Methodology","volume":null,"pages":null},"PeriodicalIF":1.6000,"publicationDate":"2024-04-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of Survey Statistics and Methodology","FirstCategoryId":"100","ListUrlMain":"https://doi.org/10.1093/jssam/smae017","RegionNum":4,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"SOCIAL SCIENCES, MATHEMATICAL METHODS","Score":null,"Total":0}
引用次数: 0
Abstract
Effective evaluation of data quality between data collected in different modes is complicated by the confounding of selection and measurement effects. This study evaluates the utility of propensity score matching (PSM) as a method that has been proposed as a means of removing selection effects across surveys conducted in different modes. Our results show large differences in estimates for the same variables between parallel face-to-face and online surveys, even after matching on standard demographic variables. Moreover, discrepancies in estimates are still present after matching between surveys conducted in the same (online) mode, where differences in measurement properties can be ruled out a priori. Our findings suggest that PSM has substantial limitations as a method for separating measurement and selection differences across modes and should be used only with caution.
期刊介绍：
The Journal of Survey Statistics and Methodology, sponsored by AAPOR and the American Statistical Association, began publishing in 2013. Its objective is to publish cutting edge scholarly articles on statistical and methodological issues for sample surveys, censuses, administrative record systems, and other related data. It aims to be the flagship journal for research on survey statistics and methodology. Topics of interest include survey sample design, statistical inference, nonresponse, measurement error, the effects of modes of data collection, paradata and responsive survey design, combining data from multiple sources, record linkage, disclosure limitation, and other issues in survey statistics and methodology. The journal publishes both theoretical and applied papers, provided the theory is motivated by an important applied problem and the applied papers report on research that contributes generalizable knowledge to the field. Review papers are also welcomed. Papers on a broad range of surveys are encouraged, including (but not limited to) surveys concerning business, economics, marketing research, social science, environment, epidemiology, biostatistics and official statistics. The journal has three sections. The Survey Statistics section presents papers on innovative sampling procedures, imputation, weighting, measures of uncertainty, small area inference, new methods of analysis, and other statistical issues related to surveys. The Survey Methodology section presents papers that focus on methodological research, including methodological experiments, methods of data collection and use of paradata. The Applications section contains papers involving innovative applications of methods and providing practical contributions and guidance, and/or significant new findings.
京公网安备 11010802042870号
京ICP备2023020795号-1
分享
求助内容：
应助结果提醒方式：
扫码关注我们
