Response Rates, Reliability, and Convergent Validity in Experience Sampling Data.

IF 3.5 2区心理学 Q1 PSYCHOLOGY, CLINICAL Assessment Pub Date : 2024-12-10 DOI:10.1177/10731911241300002

Jeffrey S Simons, Stephen A Maisto, Raluca M Simons, Jessica A Keith, Tibor P Palfai, Kyle J Walters, Surabhi Swaminath, Kawon Kim, Patrick J Ronan

{"title":"Response Rates, Reliability, and Convergent Validity in Experience Sampling Data.","authors":"Jeffrey S Simons, Stephen A Maisto, Raluca M Simons, Jessica A Keith, Tibor P Palfai, Kyle J Walters, Surabhi Swaminath, Kawon Kim, Patrick J Ronan","doi":"10.1177/10731911241300002","DOIUrl":null,"url":null,"abstract":"<p><p>This study examined associations of compliance rate with the reliability and convergent validity of intoxication and negative affect assessments in experience sampling method (ESM) data in three samples (Veterans, Sexual Minority Men, and College Students). Convergent validity was operationalized as within-person associations between daily aggregates of random in situ assessments and retrospective daily assessments or transdermal alcohol assessments. Measures with lower ICC require more assessments for a reliable aggregate (e.g., daily mean). In this regard, the number of completed assessments and intraclass correlation (ICC), rather than compliance with the protocol per se, determines reliability. Although convergent validity was correlated with compliance rate, the relatively weak associations reflect that there are individuals with excellent compliance yet poor convergent validity as well as individuals with poor compliance and excellent convergent validity. The pattern of results does not show a clear threshold for compliance (e.g., 80%) that differentiates good versus poor validity.</p>","PeriodicalId":8577,"journal":{"name":"Assessment","volume":" ","pages":"10731911241300002"},"PeriodicalIF":3.5000,"publicationDate":"2024-12-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Assessment","FirstCategoryId":"102","ListUrlMain":"https://doi.org/10.1177/10731911241300002","RegionNum":2,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"PSYCHOLOGY, CLINICAL","Score":null,"Total":0}

引用次数: 0

Abstract

This study examined associations of compliance rate with the reliability and convergent validity of intoxication and negative affect assessments in experience sampling method (ESM) data in three samples (Veterans, Sexual Minority Men, and College Students). Convergent validity was operationalized as within-person associations between daily aggregates of random in situ assessments and retrospective daily assessments or transdermal alcohol assessments. Measures with lower ICC require more assessments for a reliable aggregate (e.g., daily mean). In this regard, the number of completed assessments and intraclass correlation (ICC), rather than compliance with the protocol per se, determines reliability. Although convergent validity was correlated with compliance rate, the relatively weak associations reflect that there are individuals with excellent compliance yet poor convergent validity as well as individuals with poor compliance and excellent convergent validity. The pattern of results does not show a clear threshold for compliance (e.g., 80%) that differentiates good versus poor validity.

查看原文

微信好友朋友圈 QQ好友复制链接

本刊更多论文

经验抽样数据的响应率、信度和收敛效度。

本研究在退伍军人、性少数男性和大学生三个样本中，考察了经验抽样法（ESM）数据中依从率与醉酒和负面影响评估的信度和收敛效度的关系。聚合效度被操作为随机现场评估每日总量与回顾性每日评估或透皮酒精评估之间的人内关联。具有较低ICC的措施需要更多的评估以获得可靠的总数（例如，每日平均值）。在这方面，决定可靠性的是完成评估的数量和类内相关性（ICC），而不是是否遵守方案本身。虽然趋同效度与依从率存在相关性，但相关性较弱，反映出既有依从性优异但收敛效度较差的个体，也有依从性较差但收敛效度优异的个体。结果的模式没有显示出一个明确的符合性阈值（例如，80%）来区分好效度和差效度。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文去求助

来源期刊

Assessment PSYCHOLOGY, CLINICAL-

CiteScore

8.90

自引率

2.60%

发文量

期刊介绍： Assessment publishes articles in the domain of applied clinical assessment. The emphasis of this journal is on publication of information of relevance to the use of assessment measures, including test development, validation, and interpretation practices. The scope of the journal includes research that can inform assessment practices in mental health, forensic, medical, and other applied settings. Papers that focus on the assessment of cognitive and neuropsychological functioning, personality, and psychopathology are invited. Most papers published in Assessment report the results of original empirical research, however integrative review articles and scholarly case studies will also be considered.