Language-guided reasoning segmentation for underwater images

Abstract

In this paper, we introduce Language-Guided Reasoning Segmentation (LGRS), a framework that leverages human language instructions to guide underwater image segmentation. Unlike existing methods, that rely solely on visual cues or predefined categories, LGRS enables segmentation at underwater images based on detailed, context-aware textual descriptions, allowing it to tackle more challenging scenarios, such as distinguishing visually similar objects or identifying species from complex queries. To facilitate the development and evaluation of this approach, we create an underwater image-language segmentation dataset, the first of its kind, which pairs underwater images with detailed textual descriptions and corresponding segmentation masks. This dataset provides a foundation for training models capable of processing both visual and linguistic inputs simultaneously. Furthermore, LGRS incorporates reasoning capabilities through large language models, enabling the system to interpret complex relationships between objects in the scene and perform accurate segmentation in dynamic underwater environments. Notably, our method also demonstrates strong zero-shot segmentation capabilities, enabling the model to generalize to unseen categories without additional training. Experimental results show that LGRS outperforms existing underwater image segmentation methods in both accuracy and flexibility, offering a foundation for further advancements.