Robustness assessment of an automated AI-based white blood cell morphometric analysis system using different smear preparation methods

Introduction

Numerous AI-based systems are being developed to evaluate peripheral blood (PB) smears, but the feasibility of these systems on different smear preparation methods has not been fully understood. In this study, we assessed the impact of different smear preparation methods on the robustness of the deep learning system (DLS).

Methods

We collected 193 PB samples from patients, preparing a pair of smears for each sample using two systems: (1) SP50 smears, prepared by the DLS recommended fully automated slide preparation with double fan drying and staining (May–Grunwald Giemsa, M–G) system using SP50 (Sysmex) and (2) SP1000i smears, prepared by automated smear preparation with single fan drying by SP1000i (Sysmex) and manually stained with M–G. Digital images of PB cells were captured using DI-60 (Sysmex), and the DLS performed cell classification. Sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were used to evaluate the performance of the DLS.

Results

The specificity and NPV for all cell types were 97.4%–100% in both smear sets. The average sensitivity and PPV were 88.9% and 90.1% on SP50 smears, and 87.0% and 83.2% on SP1000i smears, respectively. The lower performance on SP1000i smears was attributed to the intra-lineage misclassification of neutrophil precursors and inter-lineage misclassification of lymphocytes.

Conclusion

The DLS demonstrated consistent performance in specificity and NPV for smears prepared by a system different from the recommended method. Our results suggest that applying an automated smear preparation system optimized for the DLS system may be important.