F2RAIL: panoptic segmentation integrating Fpn and transFormer towards RAILway

Abstract

Panoptic segmentation method enables precise identification and localization of various elements in railway scenes by assigning unique masks to each object in the image, thereby providing crucial data support for autonomous perception tasks in railway environments. However, existing segmentation methods fail to effectively leverage the prominent boundary and linear features of objects such as railway tracks and guardrails, resulting in unsatisfactory segmentation performance in railway scenes. Moreover, the inherent structural limitations of generic segmentation methods lead to weak feature extraction capabilities. Accordingly, this paper proposes the F2RAIL panoptic segmentation network, which achieves a unified approach to multi-scale detection and high-precision recognition through an innovative fusion of Feature Pyramid Networks (FPN) and transformer networks. By introducing an edge feature enhancement module, we address the insufficient utilization of linear features in railway scenes by segmentation models; By introducing a multi-dimensional enhancement module, we resolve the issues of weakened or even lost deep feature information in segmentation models. Based on the aforementioned structural innovations and methodological improvements, F2RAIL achieved a panoptic quality(PQ) of 43.74% on our custom railway dataset, representing a 2.2% improvement over existing state-of-the-art(SOTA) methods. Additionally, it demonstrated comparable performance to SOTA methods on public benchmark datasets.