A convolutional neural network-based system for identifying neuroendocrine neoplasm and multiple types of lesions in the pancreas via endoscopic ultrasound (with videos).

Background and aims: Endoscopic ultrasound (EUS) is sensitive in detecting pancreatic neuroendocrine neoplasm (pNEN). However, the endoscopic diagnosis of pNEN is operator-dependent and time-consuming since pNEN mimics normal pancreas and other pancreatic lesions. We intended to develop a convolutional neural network (CNN)-based system named iEUS for identifying pNEN and multiple types of pancreatic lesions via EUS.

Methods: Retrospective data of 12,200 EUS images obtained from pNEN and non-pNEN pancreatic lesions, including pancreatic ductal adenocarcinoma (PDAC), autoimmune pancreatitis (AIP), and pancreatic cystic neoplasm (PCN), were used to develop iEUS. It was composed of a two-category (pNEN/ non-pNEN pancreatic lesion) classification model (CNN1) and a four-category (pNEN/ PDAC/ AIP/ PCN) classification model (CNN2). Videos from consecutive patients were prospectively collected for a human-iEUS contest to evaluate the performance of iEUS.

Results: A total of 573 patients were enrolled in this study. In the human-iEUS contest containing 203 videos, CNN1 and CNN2 showed an accuracy of 84.2% and 88.2% for diagnosing pNEN, respectively, which were significantly higher than that of novices (75.4%) and comparable with intermediate endosonographers (85.5%) and experts (85.5%). In addition, CNN2 showed an accuracy of 86.2%, 97.0%, and 97.0% for diagnosing PDAC, AIP, and PCN, respectively. With the assistance of iEUS, the sensitivity of endosonographers at all three levels in diagnosing pNEN has significantly improved (64.6% vs. 44.8%, 87.5% vs. 71.9%, 74.0% vs. 57.6%, respectively).

Conclusions: The iEUS precisely diagnosed pNEN and other confusing pancreatic lesions, thus could assist endosonographers in achieving more accessible and accurate endoscopic diagnoses via EUS.