A Visualization and Radiation Treatment Plan Quality Scoring Method for Triage in a Population-Based Context

Purpose

Our purpose was to develop a clinically intuitive and easily understandable scoring method using statistical metrics to visually determine the quality of a radiation treatment plan.

Methods and Materials

Data from 111 patients with head and neck cancer were used to establish a percentile-based scoring system for treatment plan quality evaluation on both a plan-by-plan and objective-by-objective basis. The percentile scores for each clinical objective and the overall treatment plan score were then visualized using a daisy plot. To validate our scoring method, 6 physicians were recruited to assess 60 plans, each using a scoring table consisting of a 5-point Likert scale (with scores ≥3 considered passing). Spearman correlation analysis was conducted to assess the association between increasing treatment plan percentile rank and physician rating, with Likert scores of 1 and 2 representing clinically unacceptable plans, scores of 3 and 4 representing plans needing minor edits, and a score of 5 representing clinically acceptable plans. Receiver operating characteristic curve analysis was used to assess the scoring system's ability to quantify plan quality.

Results

Of the 60 plans scored by the physicians, 8 were deemed as clinically acceptable; these plans had an 89.0th ± 14.5 percentile value using our scoring system. The plans needing minor edits or deemed unacceptable had more variation, with scores falling in the 62.6nd ± 25.1 percentile and 35.6th ± 25.7 percentile, respectively. The estimated Spearman correlation coefficient between the physician score and treatment plan percentile was 0.53 (P < .001), indicating a moderate but statistically significant correlation. Receiver operating characteristic curve analysis demonstrated discernment between acceptable and unacceptable plan quality, with an area under the curve of 0.76.

Conclusions

Our scoring system correlates with physician ratings while providing intuitive visual feedback for identifying good treatment plan quality, thereby indicating its utility in the quality assurance process.