{"title":"Calibration of a Confidence Interval for a Classification Accuracy","authors":"S. Magnussen","doi":"10.4236/OJF.2021.111002","DOIUrl":null,"url":null,"abstract":"Coverage of nominal 95% confidence intervals of a \nproportion estimated from a sample obtained under a complex survey design, or a \nproportion estimated from a ratio of two random variables, can depart \nsignificantly from its target. Effective calibration methods exist for \nintervals for a proportion derived from a single binary study variable, but not \nfor estimates of thematic classification accuracy. To promote a calibration of \nconfidence intervals within the context of land-cover mapping, this study first \nillustrates a common problem of under and over-coverage with standard \nconfidence intervals, and then proposes a simple and fast calibration that more \noften than not will improve coverage. The demonstration is with simulated \nsampling from a classified map with four classes, and a reference class known \nfor every unit in a population of 160,000 units arranged in a square array. The \nsimulations include four common probability sampling designs for accuracy \nassessment, and three sample sizes. Statistically significant over- and \nunder-coverage was present in estimates of user’s (UA) and producer’s accuracy \n(PA) as well as in estimates of class area proportion. A calibration with Bayes \nintervals for UA and PA was most efficient with smaller sample sizes and two \ncluster sampling designs.","PeriodicalId":63552,"journal":{"name":"林学期刊(英文)","volume":null,"pages":null},"PeriodicalIF":0.0000,"publicationDate":"2021-01-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"1","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"林学期刊(英文)","FirstCategoryId":"1087","ListUrlMain":"https://doi.org/10.4236/OJF.2021.111002","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 1
Abstract
Coverage of nominal 95% confidence intervals of a
proportion estimated from a sample obtained under a complex survey design, or a
proportion estimated from a ratio of two random variables, can depart
significantly from its target. Effective calibration methods exist for
intervals for a proportion derived from a single binary study variable, but not
for estimates of thematic classification accuracy. To promote a calibration of
confidence intervals within the context of land-cover mapping, this study first
illustrates a common problem of under and over-coverage with standard
confidence intervals, and then proposes a simple and fast calibration that more
often than not will improve coverage. The demonstration is with simulated
sampling from a classified map with four classes, and a reference class known
for every unit in a population of 160,000 units arranged in a square array. The
simulations include four common probability sampling designs for accuracy
assessment, and three sample sizes. Statistically significant over- and
under-coverage was present in estimates of user’s (UA) and producer’s accuracy
(PA) as well as in estimates of class area proportion. A calibration with Bayes
intervals for UA and PA was most efficient with smaller sample sizes and two
cluster sampling designs.