Performance of machine learning models in predicting difficult laryngoscopy in the emergency department: a single-centre retrospective study comparing with conventional regression method.

Abstract

Background: Emergency endotracheal intubation is a critical skill for managing airway emergencies in the emergency department (ED). Accurate prediction of difficult laryngoscopy is essential for improving first-attempt success, minimizing complications, optimizing resource utilization, and enhancing patient outcomes. Traditional methods, such as the LEMON criteria, have limited predictive accuracy. Machine learning (ML) offers advanced predictive capabilities by analyzing large datasets and identifying complex variable interactions. This study aimed to develop and validate the performance of ML models for predicting difficult laryngoscopy in the ED, comparing it with a conventional regression model.

Methods: A retrospective cohort study was conducted on 4,370 adult patients who underwent intubation in the ED at Thammasat University Hospital. Difficult laryngoscopy was defined as a Cormack-Lehane grade III or IV. Patients were divided into development (training, 70%) and validation (testing, 30%) cohorts. Predictors of difficult laryngoscopy were identified using multivariable stepwise backward elimination logistic regression and were used to develop ML models, including Logistic Regression, Decision Tree, Random Forest, and XGBoost. Model performance was evaluated using the area under the receiver operating characteristic curve (AuROC), accuracy, precision, recall, and F1-score. Validation was performed on the validation cohort to confirm model accuracy.

Results: Nine significant predictors were identified: male sex, trauma, absence of neuromuscular blocking agents, large incisors, large tongue, limited mouth opening, short thyrohyoid distance, obstructed airway, and poor neck mobility. The Random Forest model demonstrated the highest predictive performance, with an AuROC of 0.82 (95% CI: 0.78-0.85), accuracy of 0.89, recall of 0.89, and F1-score of 0.87, outperforming conventional regression (AuROC 0.76, 95% CI: 0.73-0.78) and other ML models. DeLong's test confirmed a statistically significant difference in AuROC between the two models (p = 0.002). The Decision Tree showed limited performance due to overfitting, while XGBoost demonstrated strong precision. No significant differences were found when comparing the two models with conventional regression (p = 0.498 and 0.496, respectively).

Conclusion: The Random Forest model provides the most robust prediction of difficult laryngoscopy, outperforming both conventional and other ML methods. While ML models improve predictive accuracy, logistic regression remains a practical option in resource-limited settings. Integrating ML into clinical workflows could enhance decision-making, resource allocation, and patient safety in emergency airway management. Future research should prioritize external validation and real-world implementation.