Assessing the predictive accuracy of ChatGPT-based image analysis in forecasting long-term scar characteristics from 3-month assessments – A pilot study

Abstract

Introduction

Scarring significantly impacts patient quality of life, yet traditional assessments often rely on subjective evaluations, resulting in variability in predictions. This study aimed to evaluate the predictive accuracy of a Smart Image Analysis ChatGPT model in forecasting scar characteristics.

Methods

This single-institution prospective cohort study included 40 patients who underwent plastic surgery. Scar images were captured at 3 and 12 months, assessing characteristics such as vascularity, pigmentation, height, and width. The ChatGPT model predicted binary outcomes (good vs. bad scars) and continuous outcomes. Predictive accuracy was measured using metrics including mean absolute error (MAE), mean squared error (MSE), root mean squared error (RMSE), and R-squared (R²).

Results

The model achieved an overall accuracy of 97.5% for binary classifications of scars. McNemar's test confirmed no significant differences between predicted and actual outcomes. For continuous outcomes, the MAE was 0.65, with an MSE of 0.9 and RMSE of 0.95, indicating moderate accuracy. Vascularity predictions yielded an R² of 0.234, whereas height and width showed stronger correlations with R² values of 0.857 and 0.956, respectively. Statistically significant differences in paired t-tests were observed for pigmentation (t = 4.356, p = 9.319e-05) and width (t = 2.896, p = 0.0062).

Conclusion

The Smart Image Analysis ChatGPT model demonstrates excellent predictive accuracy in binary scar classification and provides valuable insights for scar characteristics. Further refinement is necessary for improving predictions of dynamic features such as vascularity.