An uncertainty-based Collaborative Weakly Supervised Segmentation Network for Positron emission tomography-Computed tomography images

Abstract

Weakly supervised segmentation has emerged as an alternative to mitigate the necessity for large volumes of annotated data in semantic segmentation, particularly crucial in medical image analysis where pixel-level labeling is time-consuming and labor-intensive. At present, many weakly supervised methods for single modality medical image segmentation have been proposed, but there is a lack of research on multimodality, especially for Positron emission tomography-Computed tomography images. In Positron emission tomography-Computed tomography images, objects may be easily distinguishable in one modality but indistinguishable in another modality. Therefore, we propose an Uncertainty-based Collaborative Weakly Supervised Segmentation Network. First, we propose a Self-Refine module to output a more precise Class Activation Map for a single modality. Then, an uncertainty collaborative learning strategy is proposed, which follows the principle of “who claim, who burden the evidence”. Uncertainty collaborative learning strategy leverages the uncertainty dispersion module to make the modalities considered to have tumors dominate in the final segmentation results fusion. If both modalities are considered to have a tumor, the cross-modal consistency constraint is utilized to obtain more precise segmentation results. Finally, we extend the uncertainty collaborative learning strategy to the fully supervised segmentation task. We do experiments on two datasets of soft tissue sarcoma and liver tumors. The experimental results prove that the proposed method is superior to other weakly supervised methods in Positron emission tomography-Computed tomography images. In addition, experiments on fully supervised segmentation also demonstrate that the uncertainty collaborative learning strategy proposed can improve the segmentation results of different segmentation networks. The code is available at https://github.com/HarriesDZS/UCWS-Net.