Occlusion Segmentation: Restore and Segment Invisible Areas for Particle Objects

Abstract

The occlusion problem has consistently posed a significant challenge in the field of segmentation. Most existing segmentation methods require additional annotations and fail to capture the contour information of occluded regions, thus not truly addressing the occlusion issue. Although segmentation tasks involving particle objects also suffer from occlusion problems, the homogeneity of particle objects offers new possibilities for overcoming this challenge. In this paper, we propose an occlusion segmentation framework for particle objects that does not require additional annotations. This framework only necessitates instance-level segmentation labels to obtain complete contour information of particle objects, including occluded regions. First, we decompose the occlusion segmentation task into a generic instance segmentation task and an occlusion repair task for occluded objects. Then, in order to train the occlusion repair model with only instance segmentation-level labels, we quantitatively analyze the occlusion phenomenon, including the mathematical descriptions of occlusion relationships, degrees, and distributions. Next, we geometrically transform and layer overlay the unobscured samples to construct occlusion samples containing labeling information of the occluded regions. These sample sets are used to train a generative model that predicts the contour information of occluded regions. Finally, we fine-tune or post-process the pre-segmentation model with the particle objects containing restored complete contour information to achieve the final occlusion segmentation. We conducted extensive ablation experiments on both the ore-particle dataset and publicly available cell-particle datasets. The experimental results validate the effectiveness, accuracy, and generalizability of our method. Note to Practitioners—Particle segmentation has been faced with the occlusion problem. In this paper, inspired by the similarity between particle objects, we propose a self-supervised occlusion segmentation framework that does not require additional annotation of occlusion layers. Our approach requires only instance segmentation level annotation without more complex additional manual annotation, which is crucial for practical applications. In addition, we decouple the complex occlusion relation modeling into a binary classification problem without knowing precisely the occlusion hierarchy between particles, which further reduces the difficulty of practical applications. Then, we also propose shading transformations to characterize the inter-particle shading distribution to construct shading sample sets from existing samples. Finally, we use these learned and constructed occlusion sample sets to pre-train the generative model for regenerating the occluded objects to complete the final occlusion segmentation. Although in a generic segmentation task, our approach may have some limitations because the segmented objects may not have an apparent similarity. However, our approach using self-supervision and the objects’ properties provides valuable ideas for solving the occlusion problem. In the future, we will solve the occlusion problem regarding the properties of each class of objects rather than just considering the similarity among particle objects.