IRR-Net: A Joint Learning Framework for Image Reconstruction and Recognition of Photoacoustic Tomography

In photoacoustic tomography (PAT), object identification and classification are usually performed as postprocessing processes after image reconstruction. Since useful information about the target implied in the raw signal can be lost during image reconstruction, this two-step scheme can reduce the accuracy of tissue characterization. For learning-based methods, it is time consuming to train the network of each subtask separately. In this paper, we report on an end-to-end joint learning framework for simultaneous image reconstruction and object recognition, named IRR-Net. It establishes direct mapping of raw photoacoustic signals to high-quality images with recognized targets. The network consists of an image reconstruction module, an optimization module, and a recognition module, which achieved signal-to-image, image-to-image, and image-to-class conversion, respectively. We built simulation, phantom and in vivo data sets to train and test IRR-Net. The results show that the proposed method successfully yields concurrent improvements in both the quality of the reconstructed images and the accuracy of target recognition at a lower time cost compared to the separately trained networks.