DGT: Depth-guided RGB-D occluded target detection with transformers

In occluded urban environments, the traditional object detection algorithm relies solely on RGB as input, making it challenging to discern the spatial relationship of occluded objects and consequently affecting the target detection accuracy. Previous studies primarily focused on fusing depth and RGB information at the feature level, resulting in the loss of detailed features from the original data, such as occlusion boundaries. This leads to blurred fusion features and degraded model detection performance. Therefore, this paper proposes a depth-guided RGB-D occluded target detection framework based on transformers (DGT) to effectively extract occlusion boundary information and guide the occlusion discrimination via data-level fusion of depth and RGB information. In particular, a multimodal data-level fusion model is proposed for a two-part task. One is to generate dense depth images with strengthened occlusion edge features by extracting the depth difference of object edges in the point cloud data. The other is to dilute the influence of useless information using RGB-D data-level fusion. A depth-guided occlusion layered detection network with transformers was designed to obtain the cross-module guided feature vector by exchanging the weights of the residual and interaction vectors. Extensive experiments showed that DGT achieves state-of-the-art performance in occluded environments.