Developing an Effective Off-the-job Training Model and an Automated Evaluation System for Thoracoscopic Esophageal Atresia Surgery

Abstract

Background

Pediatric minimally invasive surgery requires advanced technical skills. Off-the-job training (OJT), especially when using disease-specific models, is an effective method of acquiring surgical skills. To achieve effective OJT, it is necessary to provide objective and appropriate skill assessment feedback to trainees. We aimed to construct a system that automatically evaluates surgical skills based on forceps movement using deep learning (DL).

Methods

Using our original esophageal atresia OJT model, participants were tasked with performing esophageal anastomosis. All tasks were recorded for image analysis. Based on manual objective skill assessments, each participant's surgical skills were categorized into two groups: good and poor. The motion of the forceps in both groups was used as training data. Employing this training data, we constructed an automated system that recognized the movement of forceps and determined the quality of the surgical technique.

Results

Thirteen participants were assigned to the good skill group and 32 to the poor skill group. These cases were validated using an automated skill assessment system. This system showed a precision of 75%, a specificity of 94%, and an area under the receiver operating characteristic curve of 0.81.

Conclusions

We constructed a system that automatically evaluated the quality of surgical techniques based on the movement of forceps using DL. Artificial intelligence diagnostics further revealed the procedures important for suture manipulation.

Levels of Evidence

Level IV