End-User Confidence in Artificial Intelligence-Based Predictions Applied to Biomedical Data.

Abstract

Applications of Artificial Intelligence (AI) are revolutionizing biomedical research and healthcare by offering data-driven predictions that assist in diagnoses. Supervised learning systems are trained on large datasets to predict outcomes for new test cases. However, they typically do not provide an indication of the reliability of these predictions, even though error estimates are integral to model development. Here, we introduce a novel method to identify regions in the feature space that diverge from training data, where an AI model may perform poorly. We utilize a compact precompiled structure that allows for fast and direct access to confidence scores in real time at the point of use without requiring access to the training data or model algorithms. As a result, users can determine when to trust the AI model's outputs, while developers can identify where the model's applicability is limited. We validate our approach using simulated data and several biomedical case studies, demonstrating that our approach provides fast confidence estimates ([Formula: see text] milliseconds per case), with high concordance to previously developed methods (f-[Formula: see text]). These estimates can be easily added to real-world AI applications. We argue that providing confidence estimates should be a standard practice for all AI applications in public use.