3D Hyperspectral Data Analysis with Spatially Aware Deep Learning for Diagnostic Applications

Abstract

Nowadays, with the rise of artificial intelligence (AI), deep learning algorithms play an increasingly important role in various traditional fields of research. Recently, these algorithms have already spread into data analysis for Raman spectroscopy. However, most current methods only use 1-dimensional (1D) spectral data classification, instead of considering any neighboring information in space. Despite some successes, this type of methods wastes the 3-dimensional (3D) structure of Raman hyperspectral scans. Therefore, to investigate the feasibility of preserving the spatial information on Raman spectroscopy for data analysis, spatially aware deep learning algorithms were applied into a colorectal tissue data set with 3D Raman hyperspectral scans. This data set contains Raman spectra from normal, hyperplasia, adenoma, carcinoma tissues as well as artifacts. First, a modified version of 3D U-Net was utilized for segmentation; second, another convolutional neural network (CNN) using 3D Raman patches was utilized for pixel-wise classification. Both methods were compared with the conventional 1D CNN method, which worked as baseline. Based on the results of both epithelial tissue detection and colorectal cancer detection, it is shown that using spatially neighboring information on 3D Raman scans can increase the performance of deep learning models, although it might also increase the complexity of network training. Apart from the colorectal tissue data set, experiments were also conducted on a cholangiocarcinoma data set for generalizability verification. The findings in this study can also be potentially applied into future tasks regarding spectroscopic data analysis, especially for improving model performance in a spatially aware way.