Assessing the accuracy of artificial intelligence in the diagnosis and management of orbital fractures: Is this the future of surgical decision-making?

Orbital fractures are common, but their management remains controversial. The aim of the present study was to assess the accuracy of an advanced artificial intelligence (AI) model, ChatGPT-4, in surgical decision-making, with a focus on orbital fracture diagnosis and management.

A retrospective observational analysis was conducted by involving a sample of 30 orbital fracture cases diagnosed and managed at the Geneva University Hospital, Switzerland. The process involved creating patient vignettes from anonymised medical records and presenting them to ChatGPT-4 in three stages: initial diagnosis, refinement with radiological reports and surgical intervention decisions. The performance of ChatGPT-4 in providing the appropriate surgical strategy was evaluated through measures of sensitivity, specificity, positive predictive value and negative predictive value, with the actual management used as the benchmark for accuracy.

The AI model could correctly diagnose the fracture in 100 % of the cases. It demonstrated a specificity of 100 % and sensitivity of 57 % for treatment recommendation, indicating its effectiveness in recognising patients who truly required an intervention; however, it demonstrated a moderate performance in correctly identifying cases that were better suited for conservative treatment. Cohen's Kappa statistic for interrater reliability of the choice of treatment was 0.44, indicating a weak level of agreement between ChatGPT and the physician's choice of treatment.

The study demonstrates that AI tools such as ChatGPT-4 can offer a high degree of accuracy in diagnosing orbital fractures and recognising patients requiring surgical intervention; however, it performed less satisfactorily in correctly identifying patients who were better suited for non-surgical treatment.