{"title":"评分者间可靠性研究中的协议类内相关系数的渐近置信区间、样本量公式和比较测试","authors":"Abderrahmane Bourredjem, Hervé Cardot, Hervé Devilliers","doi":"10.1002/sim.10217","DOIUrl":null,"url":null,"abstract":"The agreement intra‐class correlation coefficient (ICCa) is a suitable statistical index for inter‐rater reliability studies. With balanced Gaussian data, we prove the explicit form of ICCa asymptotic normality (ASN), valid both with analysis of variance (ANOVA), maximum likelihood (ML), or restricted ML (REML) estimates. An asymptotic confidence interval is then derived and its performances are examined by simulation compared to the most commonly used methods, under small, moderate and large sample size designs. Then, we deduce sample size calculation formulas, for the number of subjects and observers needed, to achieve a desired confidence interval width or an acceptable ICCa value test power and give concrete examples of their use. Finally, we propose a likelihood ratio test (LRT) to compare two ICCa's from two distinct subpopulations of patients (or raters) and study by simulation its first order risk and power properties. These methods are illustrated using data from two inter‐rater reliability studies, one in physiotherapy with 42 patients and 10 raters and the second in neonatology with 80 subjects and 14 raters. In conclusion, we made recommendations to employ the proposed confidence interval for medium to large samples combined with the quantification of the minimal required sample size at the planning step, or the posterior‐power at the analysis step, using simple dedicated formulas. Furthermore, with sufficient sizes, the proposed LRT seems suitable to compare inter‐rater reliability between two patient subpopulations. Used wisely, this proposed methods toolbox can remedy common current issues in inter‐rater reliability studies.","PeriodicalId":21879,"journal":{"name":"Statistics in Medicine","volume":null,"pages":null},"PeriodicalIF":1.8000,"publicationDate":"2024-09-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Asymptotic Confidence Interval, Sample Size Formulas and Comparison Test for the Agreement Intra‐Class Correlation Coefficient in Inter‐Rater Reliability Studies\",\"authors\":\"Abderrahmane Bourredjem, Hervé Cardot, Hervé Devilliers\",\"doi\":\"10.1002/sim.10217\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"The agreement intra‐class correlation coefficient (ICCa) is a suitable statistical index for inter‐rater reliability studies. With balanced Gaussian data, we prove the explicit form of ICCa asymptotic normality (ASN), valid both with analysis of variance (ANOVA), maximum likelihood (ML), or restricted ML (REML) estimates. An asymptotic confidence interval is then derived and its performances are examined by simulation compared to the most commonly used methods, under small, moderate and large sample size designs. Then, we deduce sample size calculation formulas, for the number of subjects and observers needed, to achieve a desired confidence interval width or an acceptable ICCa value test power and give concrete examples of their use. Finally, we propose a likelihood ratio test (LRT) to compare two ICCa's from two distinct subpopulations of patients (or raters) and study by simulation its first order risk and power properties. These methods are illustrated using data from two inter‐rater reliability studies, one in physiotherapy with 42 patients and 10 raters and the second in neonatology with 80 subjects and 14 raters. In conclusion, we made recommendations to employ the proposed confidence interval for medium to large samples combined with the quantification of the minimal required sample size at the planning step, or the posterior‐power at the analysis step, using simple dedicated formulas. Furthermore, with sufficient sizes, the proposed LRT seems suitable to compare inter‐rater reliability between two patient subpopulations. Used wisely, this proposed methods toolbox can remedy common current issues in inter‐rater reliability studies.\",\"PeriodicalId\":21879,\"journal\":{\"name\":\"Statistics in Medicine\",\"volume\":null,\"pages\":null},\"PeriodicalIF\":1.8000,\"publicationDate\":\"2024-09-17\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Statistics in Medicine\",\"FirstCategoryId\":\"3\",\"ListUrlMain\":\"https://doi.org/10.1002/sim.10217\",\"RegionNum\":4,\"RegionCategory\":\"医学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q3\",\"JCRName\":\"MATHEMATICAL & COMPUTATIONAL BIOLOGY\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Statistics in Medicine","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.1002/sim.10217","RegionNum":4,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q3","JCRName":"MATHEMATICAL & COMPUTATIONAL BIOLOGY","Score":null,"Total":0}
Asymptotic Confidence Interval, Sample Size Formulas and Comparison Test for the Agreement Intra‐Class Correlation Coefficient in Inter‐Rater Reliability Studies
The agreement intra‐class correlation coefficient (ICCa) is a suitable statistical index for inter‐rater reliability studies. With balanced Gaussian data, we prove the explicit form of ICCa asymptotic normality (ASN), valid both with analysis of variance (ANOVA), maximum likelihood (ML), or restricted ML (REML) estimates. An asymptotic confidence interval is then derived and its performances are examined by simulation compared to the most commonly used methods, under small, moderate and large sample size designs. Then, we deduce sample size calculation formulas, for the number of subjects and observers needed, to achieve a desired confidence interval width or an acceptable ICCa value test power and give concrete examples of their use. Finally, we propose a likelihood ratio test (LRT) to compare two ICCa's from two distinct subpopulations of patients (or raters) and study by simulation its first order risk and power properties. These methods are illustrated using data from two inter‐rater reliability studies, one in physiotherapy with 42 patients and 10 raters and the second in neonatology with 80 subjects and 14 raters. In conclusion, we made recommendations to employ the proposed confidence interval for medium to large samples combined with the quantification of the minimal required sample size at the planning step, or the posterior‐power at the analysis step, using simple dedicated formulas. Furthermore, with sufficient sizes, the proposed LRT seems suitable to compare inter‐rater reliability between two patient subpopulations. Used wisely, this proposed methods toolbox can remedy common current issues in inter‐rater reliability studies.
期刊介绍:
The journal aims to influence practice in medicine and its associated sciences through the publication of papers on statistical and other quantitative methods. Papers will explain new methods and demonstrate their application, preferably through a substantive, real, motivating example or a comprehensive evaluation based on an illustrative example. Alternatively, papers will report on case-studies where creative use or technical generalizations of established methodology is directed towards a substantive application. Reviews of, and tutorials on, general topics relevant to the application of statistics to medicine will also be published. The main criteria for publication are appropriateness of the statistical methods to a particular medical problem and clarity of exposition. Papers with primarily mathematical content will be excluded. The journal aims to enhance communication between statisticians, clinicians and medical researchers.