{"title":"Estimating the Diagnostic Accuracy from Multiple Raters Based on a Bivariate Random Effects Model","authors":"H. Saeki, T. Tango, Jinfang Wang","doi":"10.5691/JJB.37.23","DOIUrl":null,"url":null,"abstract":"In clinical investigations designed to demonstrate the efficacy of a diagnostic procedure, the procedure is usually evaluated by multiple independent raters. Although the sensitivity and specificity may be estimated by considering consensus evaluations to treat results from multiple raters as if there were a single rater, raters are not consid-ered independent in consensus evaluations. Typically, estimation methods are based on an “average rater” or a “majority rater” to account for multiple raters. In this paper, we propose a method for summarizing sensitivities and specificities evaluated from multiple independent raters based on a bivariate random effects model (BVRM) to account between-rater variance and correlation between sensitivity and specificity. In addition, we propose methods to draw joint confidence regions of sensitivity and specificity based on the BVRM. Simulation results show that the differences in the biases between the proposed method and the average rater method are small and that the empirical coverage probabilities of the proposed joint confidence regions are close to the nominal level. The proposed methods are illustrated using data from florbetapir F 18 positron emission tomographic imaging to predict the presence of β -amyloid in the brains of subjects with Alzheimer’s We an If each of exceeded , the was to otherwise, it was to zero. Similarly, if each element of the matrix for the specificities exceeded z p b ) , the outcome of the element was set to one; otherwise, it was set to zero. Here, z − p ) is the 100 p percentile of the standard distribution. our simulation study, 10,000 data sets were","PeriodicalId":365545,"journal":{"name":"Japanese journal of biometrics","volume":null,"pages":null},"PeriodicalIF":0.0000,"publicationDate":"2016-07-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"1","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Japanese journal of biometrics","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.5691/JJB.37.23","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 1
Abstract
In clinical investigations designed to demonstrate the efficacy of a diagnostic procedure, the procedure is usually evaluated by multiple independent raters. Although the sensitivity and specificity may be estimated by considering consensus evaluations to treat results from multiple raters as if there were a single rater, raters are not consid-ered independent in consensus evaluations. Typically, estimation methods are based on an “average rater” or a “majority rater” to account for multiple raters. In this paper, we propose a method for summarizing sensitivities and specificities evaluated from multiple independent raters based on a bivariate random effects model (BVRM) to account between-rater variance and correlation between sensitivity and specificity. In addition, we propose methods to draw joint confidence regions of sensitivity and specificity based on the BVRM. Simulation results show that the differences in the biases between the proposed method and the average rater method are small and that the empirical coverage probabilities of the proposed joint confidence regions are close to the nominal level. The proposed methods are illustrated using data from florbetapir F 18 positron emission tomographic imaging to predict the presence of β -amyloid in the brains of subjects with Alzheimer’s We an If each of exceeded , the was to otherwise, it was to zero. Similarly, if each element of the matrix for the specificities exceeded z p b ) , the outcome of the element was set to one; otherwise, it was set to zero. Here, z − p ) is the 100 p percentile of the standard distribution. our simulation study, 10,000 data sets were
在旨在证明诊断程序有效性的临床调查中,该程序通常由多个独立评价者进行评估。虽然可以通过考虑共识评价来估计敏感性和特异性,将多个评分者的结果视为单一评分者,但在共识评价中,评分者并不被认为是独立的。通常,评估方法基于“平均评分者”或“多数评分者”来考虑多个评分者。在本文中,我们提出了一种基于二元随机效应模型(BVRM)的方法来总结从多个独立评分者评估的敏感性和特异性,以考虑评分者之间的方差和敏感性和特异性之间的相关性。此外,我们提出了基于brvrm的灵敏度和特异性联合置信区域的绘制方法。仿真结果表明,所提出的联合置信区域的经验覆盖概率与平均加权方法的偏差差异较小,接近于名义水平。采用florbetapir f18正电子发射断层成像的数据来说明所提出的方法,以预测阿尔茨海默病患者大脑中β -淀粉样蛋白的存在。如果每一个都超过,则为零,否则为零。同样,如果矩阵的每个元素的特异性超过z p b),则该元素的结果设为1;否则,它被设为0。这里,z−p)是标准分布的100p个百分位数。我们的模拟研究,10000个数据集