Semantic ghost imaging based on semantic coding

To address the challenge of reconstructing high-quality images under low sampling during the Ghost Imaging (GI) procedure, we propose a semantic GI method based on semantic encoding. Through training, the obtained continuous weights of the convolution kernels are served as the speckle patterns. A processing module and a Recurrent Neural Network (RNN) are then employed to decode and reconstruct the images. Both the speckle patterns and their corresponding bucket values are optimized, so that the semantic information of the target object can be better extracted. With different gated recurrent units (GRU) layers for the target objects in different datasets, the feasibility of the proposed GI method is validated by the numerical simulations and experiments on the simple target objects (the handwriting digits “0” to “9”) and more complex target objects (“NUPT”). The results show that the target objects (the handwriting digits) can be reconstructed with higher quality even at a lower sampling rate of 1.28%. Additionally, the proposed method has applicability for more complex objects (such as “NUPT”) in real applications. In comparison with those results by using traditional ghost imaging (TGI), deep convolution auto-encoder network (DCAN), and RNN-based GI (GI-RNN), the proposed GI method shows a better performance in terms of the quality of reconstructed images and the training time, that is, the proposed method can have both good reconstructed image quality and less training time. By introducing the concept of semantic communication into GI, the proposed method provides a new idea for the model-based GI.