Artificial Intelligence-Assisted Surgical Scene Recognition: A Comparative Study Among Health Care Professionals.

Abstract

Objective: To compare the ability of a deep-learning platform (the MACSSwin-T model) with health care professionals in detecting cerebral aneurysms from operative videos. Secondly, we aimed to compare health care professionals' ability to detect cerebral aneurysms with and without artificial intelligence (AI) assistance.

Background: Modern microscopic surgery enables the capture of operative video data on an unforeseen scale. Advances in computer vision, a branch of AI, have enabled automated analysis of operative video. These advances are likely to benefit clinicians, health care systems, and patients alike, yet such benefits are yet to be realized.

Methods: In a cross-sectional comparative study, neurosurgeons, anesthetists, and operating room nurses, all at varying stages of training and experience, reviewed still frames of aneurysm clipping operations and labeled frames as "aneurysm not in frame" or "aneurysm in frame." Frames then underwent analysis by the AI platform. A second round of data collection was performed, whereby the neurosurgical team had AI assistance. The accuracy of aneurysm detection was calculated for human-only, AI-only, and AI-assisted human groups.

Results: A total of 5154 individual frame reviews were collated from 338 health care professionals. Health care professionals correctly labeled 70% of frames without AI assistance, compared with 78% with AI assistance (odds ratio: 1.77, P < 0.001). Neurosurgical Attendings showed the greatest improvement, from 77% to 92% correct predictions with AI assistance (odds ratio: 4.24, P = 0.003).

Conclusions: AI-assisted human performance surpassed both human and AI alone. Notably, across health care professionals surveyed, frame accuracy improved across all subspecialties and experience levels, particularly among the most experienced health care professionals. These results challenge the prevailing notion that AI primarily benefits junior clinicians, highlighting its crucial role throughout the surgical hierarchy as an essential component of modern surgical practice.