Self-prompt contextual learning with AxialMamba for multi-label segmentation in carotid ultrasound

Abstract

Plaque and vessel segmentation in carotid ultrasound videos is critical for assessing carotid artery stenosis and providing essential information for doctors’ diagnostic and treatment planning. However, most existing methods segment vessels and plaques without distinguishing the plaque types and their corresponding vascular segments. To address this limitation, we define a novel multi-label carotid ultrasound video segmentation task that categorizes vessels based on their anatomical locations and classifies plaques according to their echo characteristics. To address this task, we constructed a novel dataset, CAUS45, comprising 7479 annotated frames from 45 patients. In this dataset, vessels are segmented into three categories: the internal carotid artery (ICA), external carotid artery (ECA), and common carotid artery (CCA). Plaques are classified based on echogenicity into three types: weakly echogenic, moderately echogenic, and strongly echogenic. To further advance this task, we propose a self-prompt contextual segmentation framework, termed SPCNet. To address the challenges posed by the significant variability in ultrasound images, we leveraged foundational models pretrained on large-scale ultrasound datasets as part of our video clip encoder to extract features from individual frames. To effectively utilize the inter-frame contextual information within a clip, we propose a novel AxialMamba module designed for extracting inter-frame features. Additionally, to fully exploit the correlation between different clips within a video, we introduce a self-prompted contextual learning strategy to establish contextual dependencies across clips. Experiments demonstrate that SPCNet achieves a Dice coefficient of 89.08%, with a 3.04% improvement over the current state-of-the-art method. Additionally, SPCNet achieves a Hausdorff Distance (HD) of 5.04 and an Average Surface Distance (ASD) of 1.21 on our private CAUS45 dataset. Our method shows the great potential to be applied in practical large-scale screening.