Comparison of the accuracy of GPT-4 and resident physicians in differentiating benign and malignant thyroid nodules.

Abstract

Objective: To assess the diagnostic performance of the GPT-4 model in comparison to resident physicians in distinguishing between benign and malignant thyroid nodules using ultrasound images.

Methods: This study analyzed 1,145 ultrasound images, including 632 malignant and 513 benign nodules. Both the GPT-4 model and two resident physicians independently classified the nodules using ultrasound images. The diagnostic accuracy of the resident physicians was determined by calculating the average of the individual accuracy rates of the two physicians and this was compared with the performance of the GPT-4 model.

Results: The GPT-4 model correctly identified 367 out of 632 malignant nodules (58.07%) and 343 out of 513 benign nodules (66.86%). Resident physicians identified 467 malignant (73.89%) and 383 benign nodules (74.66%). There was a statistically significant difference in the classification of malignant nodules (p < 0.001) and benign nodules (p = 0.048) between the GPT-4 model and residents. GPT-4 performed better for larger nodules (>1 cm) at 65.38%, compared to 53.77% for smaller nodules (≤1 cm, p = 0.004). The AUC for GPT-4 was 0.67, while residents achieved 0.75.

Conclusion: The GPT-4 model shows potential in classifying thyroid nodules, but its diagnostic accuracy remains significantly lower than that of resident physicians, particularly for smaller malignant nodules.