Reliable multi-modal prototypical contrastive learning for difficult airway assessment

Recent advancements in facial image-based prediction for difficult airway assessment show significant clinical promise. However, existing methods often struggle to accurately distinguish subtle facial features, contend with limited label information, and address the uncertainty in correlating facial features with airway difficulty. In this study, we propose a Reliable Multimodal Prototypical Contrastive Learning Network (RMP-Net) for difficult airway assessment, which aims to overcome these challenges. RMP-Net integrates multiple modalities, including facial images processed by a Convolutional Neural Network (CNN) and keypoint graphs processed by a Graph Convolutional Network (GCN). In addition to the commonly used image modality, we innovatively build a graph based on the keypoints modality for prediction. It not only captures comprehensive facial information but also targets critical anatomical features, enhancing feature representation and model interpretability. During the training, features extracted from laryngoscopic images serve as a priori prototype, which are further aligned with facial image and keypoint features for a clearer feature representation. Importantly, the laryngoscopic modality is used exclusively during training since it is obtained intraoperatively. This ensures that RMP-Net remains a preoperative prediction method while leveraging detailed anatomical insights during learning. Furthermore, we introduce a uncertainty learning process to validate the correlation between facial features and airway difficulty, improving the model’s robustness by focusing on reliable data. We construct a comprehensive multi-modal dataset, including facial images, laryngoscopic images, and facial keypoints. Five-fold cross-validation experiments demonstrate that RMP-Net achieves significant improvements in diagnostic AUC, sensitivity, and specificity compared to traditional and state-of-the-art (SoTA) methods. The code for this study is available at https://github.com/a6177738/RMP-Net.