Context-Aware Transformer GAN for Direct Generation of Attenuation and Scatter Corrected PET Data

Abstract

We present a context-aware generative deep learning framework to produce photon attenuation and scatter corrected (ASC) positron emission tomography (PET) images directly from nonattenuation and nonscatter corrected (NASC) images. We trained conditional generative adversarial networks (cGANs) on either single-modality (NASC) or multimodality (NASC+MRI) input data to map NASC images to pixel-wise continuously valued ASC PET images. We designed and evaluated four cGAN models including Pix2Pix, attention-guided cGAN (AG-Pix2Pix), vision transformer cGAN (ViT-GAN), and shifted window transformer cGAN (Swin-GAN). Retrospective 18F-fluorodeoxyglucose (18F-FDG) full-body PET images from 33 subjects were collected and analyzed. Notably, as a particular strength of this work, each patient in the study underwent both a PET/CT scan and a multisequence PET/MRI scan on the same day giving us a gold standard from the former as we investigate ASC for the latter. Quantitative analysis, evaluating image quality using peak signal-to-noise ratio (PSNR), multiscale structural similarity index (MS-SSIM), normalized mean-squared error (NRMSE), and mean absolute error (MAE) metrics, showed no significant impact of input type on PSNR ( $p=0.95$ ), MS-SSIM ( $p=0.083$ ), NRMSE ( $p=0.72$ ), or MAE ( $p=0.70$ ). For multimodal input data, Swin-GAN outperformed Pix2Pix ( $p=0.023$ ) and AG-Pix2Pix ( $p \lt 0.001$ ), but not ViT-GAN ( $p=0.154$ ) in PSNR. Swin-GAN achieved significantly higher MS-SSIM than ViT-GAN ( $p=0.007$ ) and AG-Pix2Pix ( $p=0.002$ ). Multimodal Swin-GAN demonstrated reduced NRMSE and MAE compared to ViT-GAN ( $p=0.023$ and 0.031, respectively) and AG-Pix2Pix (both $p \lt 0.001$ ), with marginal improvement over Pix2Pix ( $p \lt 0.064$ ). The cGAN models, in particular Swin-GAN, consistently generated reliable and accurate ASC PET images, whether using multimodal or single-modal input data. The findings indicate that this methodology can be used to generate ASC data from standalone PET scanners or integrated PET/MRI systems, without relying on transmission scan-based attenuation maps.