{"title":"Unsupervised binocular depth prediction network for laparoscopic surgery.","authors":"Ke Xu, Zhiyong Chen, F. Jia","doi":"10.1080/24699322.2018.1560082","DOIUrl":null,"url":null,"abstract":"Minimally invasive surgery (MIS) is characterized by less trauma, shorter recovery time, and lower postoperative infection rate. The two-dimensional (2D) laparoscopic imaging lacks depth perception and does not provide quantitative depth information, thereby limiting precise and complex surgical operations. Three-dimensional (3D) laparoscopic imaging provides surgeons depth perception. This study aims to 3D reconstruction of the surgical scene based on the disparity map generated by the depth estimation algorithm. An unsupervised learning autoencoder method was proposed to calculate the accurate disparity with a 101-layer residual convolutional network. The loss function included three parts: left-right consistency loss, structure similarity loss, and reconstruction error loss, the combination can improve reconstruction accuracy and robustness. The method was validated on a Hamlyn Center Laparoscopic/Endoscopic Video Dataset. The structural similarity index (SSIM) is 0.8349 ± 0.0523 and the peak signal-to-noise ratio (PSNR) is 14.4957 ± 1.9676. The depth prediction network has high accuracy and robustness. The average time to produce each disparity map is about 16 ms. The experimental result shows that the proposed depth estimation method can offer dense disparity map, and can meet surgical real-time requirement. Future work will focus on network structure optimization and loss function design, transfer learning to improve the robustness and accuracy further.","PeriodicalId":56051,"journal":{"name":"Computer Assisted Surgery","volume":null,"pages":null},"PeriodicalIF":1.5000,"publicationDate":"2019-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Computer Assisted Surgery","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.1080/24699322.2018.1560082","RegionNum":4,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q3","JCRName":"SURGERY","Score":null,"Total":0}
引用次数: 0
Abstract
Minimally invasive surgery (MIS) is characterized by less trauma, shorter recovery time, and lower postoperative infection rate. The two-dimensional (2D) laparoscopic imaging lacks depth perception and does not provide quantitative depth information, thereby limiting precise and complex surgical operations. Three-dimensional (3D) laparoscopic imaging provides surgeons depth perception. This study aims to 3D reconstruction of the surgical scene based on the disparity map generated by the depth estimation algorithm. An unsupervised learning autoencoder method was proposed to calculate the accurate disparity with a 101-layer residual convolutional network. The loss function included three parts: left-right consistency loss, structure similarity loss, and reconstruction error loss, the combination can improve reconstruction accuracy and robustness. The method was validated on a Hamlyn Center Laparoscopic/Endoscopic Video Dataset. The structural similarity index (SSIM) is 0.8349 ± 0.0523 and the peak signal-to-noise ratio (PSNR) is 14.4957 ± 1.9676. The depth prediction network has high accuracy and robustness. The average time to produce each disparity map is about 16 ms. The experimental result shows that the proposed depth estimation method can offer dense disparity map, and can meet surgical real-time requirement. Future work will focus on network structure optimization and loss function design, transfer learning to improve the robustness and accuracy further.
期刊介绍:
omputer Assisted Surgery aims to improve patient care by advancing the utilization of computers during treatment; to evaluate the benefits and risks associated with the integration of advanced digital technologies into surgical practice; to disseminate clinical and basic research relevant to stereotactic surgery, minimal access surgery, endoscopy, and surgical robotics; to encourage interdisciplinary collaboration between engineers and physicians in developing new concepts and applications; to educate clinicians about the principles and techniques of computer assisted surgery and therapeutics; and to serve the international scientific community as a medium for the transfer of new information relating to theory, research, and practice in biomedical imaging and the surgical specialties.
The scope of Computer Assisted Surgery encompasses all fields within surgery, as well as biomedical imaging and instrumentation, and digital technology employed as an adjunct to imaging in diagnosis, therapeutics, and surgery. Topics featured include frameless as well as conventional stereotactic procedures, surgery guided by intraoperative ultrasound or magnetic resonance imaging, image guided focused irradiation, robotic surgery, and any therapeutic interventions performed with the use of digital imaging technology.