Stefan Schneider, Pey-Jiuan Lee, Raymond Hernandez, Doerte U Junghaenel, Arthur A Stone, Erik Meijer, Haomiao Jin, Arie Kapteyn, Bart Orriens, Elizabeth M Zelinski
{"title":"Cognitive Functioning and the Quality of Survey Responses: An Individual Participant Data Meta-Analysis of 10 Epidemiological Studies of Aging.","authors":"Stefan Schneider, Pey-Jiuan Lee, Raymond Hernandez, Doerte U Junghaenel, Arthur A Stone, Erik Meijer, Haomiao Jin, Arie Kapteyn, Bart Orriens, Elizabeth M Zelinski","doi":"10.1093/geronb/gbae030","DOIUrl":null,"url":null,"abstract":"<p><strong>Objectives: </strong>Self-reported survey data are essential for monitoring the health and well-being of the population as it ages. For studies of aging to provide precise and unbiased results, it is necessary that the self-reported information meets high psychometric standards. In this study, we examined whether the quality of survey responses in panel studies of aging depends on respondents' cognitive abilities.</p><p><strong>Methods: </strong>Over 17 million survey responses from 157,844 participants aged 50 years and older in 10 epidemiological studies of aging were analyzed. We derived 6 common statistical indicators of response quality from each participant's data and estimated the correlations with participants' cognitive test scores at each study wave. Effect sizes (correlations) were synthesized across studies, cognitive tests, and waves using individual participant data meta-analysis methods.</p><p><strong>Results: </strong>Respondents with lower cognitive scores showed significantly more missing item responses (overall effect size ρ^ = -0.144), random measurement error (ρ^ = -0.192), Guttman errors (ρ^ = -0.233), multivariate outliers (ρ^ = -0.254), and acquiescent responses (ρ^ = -0.078); the overall effect for extreme responses (ρ^ = -0.045) was not significant. Effect sizes were consistent across studies, modes of survey administsration, and different cognitive functioning domains, although some cognitive domain specificity was also observed.</p><p><strong>Discussion: </strong>Lower-quality responses among respondents with lower cognitive abilities add random and systematic errors to survey measures, reducing the reliability, validity, and reproducibility of survey study results in aging research.</p>","PeriodicalId":56111,"journal":{"name":"Journals of Gerontology Series B-Psychological Sciences and Social Sciences","volume":null,"pages":null},"PeriodicalIF":4.8000,"publicationDate":"2024-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10998342/pdf/","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journals of Gerontology Series B-Psychological Sciences and Social Sciences","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.1093/geronb/gbae030","RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"GERIATRICS & GERONTOLOGY","Score":null,"Total":0}
引用次数: 0
Abstract
Objectives: Self-reported survey data are essential for monitoring the health and well-being of the population as it ages. For studies of aging to provide precise and unbiased results, it is necessary that the self-reported information meets high psychometric standards. In this study, we examined whether the quality of survey responses in panel studies of aging depends on respondents' cognitive abilities.
Methods: Over 17 million survey responses from 157,844 participants aged 50 years and older in 10 epidemiological studies of aging were analyzed. We derived 6 common statistical indicators of response quality from each participant's data and estimated the correlations with participants' cognitive test scores at each study wave. Effect sizes (correlations) were synthesized across studies, cognitive tests, and waves using individual participant data meta-analysis methods.
Results: Respondents with lower cognitive scores showed significantly more missing item responses (overall effect size ρ^ = -0.144), random measurement error (ρ^ = -0.192), Guttman errors (ρ^ = -0.233), multivariate outliers (ρ^ = -0.254), and acquiescent responses (ρ^ = -0.078); the overall effect for extreme responses (ρ^ = -0.045) was not significant. Effect sizes were consistent across studies, modes of survey administsration, and different cognitive functioning domains, although some cognitive domain specificity was also observed.
Discussion: Lower-quality responses among respondents with lower cognitive abilities add random and systematic errors to survey measures, reducing the reliability, validity, and reproducibility of survey study results in aging research.
期刊介绍:
The Journal of Gerontology: Psychological Sciences publishes articles on development in adulthood and old age that advance the psychological science of aging processes and outcomes. Articles have clear implications for theoretical or methodological innovation in the psychology of aging or contribute significantly to the empirical understanding of psychological processes and aging. Areas of interest include, but are not limited to, attitudes, clinical applications, cognition, education, emotion, health, human factors, interpersonal relations, neuropsychology, perception, personality, physiological psychology, social psychology, and sensation.