Evaluating intra and inter-observer bias in the cosmetic rating for random vs. serial assessment of breast photographs

Abstract

Background

This study was done to assess inter and intra-rater bias in rating of cosmesis, when breast photographs were evaluated serially or randomly by a panel of six members having varying years of experience.

Methods

Cosmetic assessment was done subjectively for 175 images [of 50 unilateral breast cancer patients for whom at least 3 images were collected], that were arranged serially from baseline to follow up in chronological order termed ‘serial assessment setting’ [SAS]. For ‘random assessment setting’ [RAS], all images was randomly arranged for assessment. Objectively assessment was also done using BCCT.core. Kappa index was calculated for agreement between the RAS and SAS rating for the 3 panellists' groups and with BCCT.core.

Results

Good agreement [kappa 0.659] was found between the mean panel cosmetic scores for both SAS and RAS. Fair agreement was found when subjective RAS [k = 0.301] and SAS [k = 0.343] scores were compared with the BCCT.core, which was highest for the most experienced panellists with SAS k = 0.387 and RAS k = 0.436. Both SAS and RAS had good intra-rater reliability.

Conclusions

SAS improves the agreement with BCCT.core rating and may be used if validated in a larger cohort. The clinical experience of the panellist impacts cosmetic rating and must be considered before forming a panel.