Influence of Data Curation and Confidence Levels on Compound Predictions Using Machine Learning Models.

Abstract

While data curation principles and practices are a major topic in data science, they are often not explicitly considered in machine learning (ML) applications in chemistry. We have been interested in evaluating the potential effects of data curation on the performance of molecular ML models. Therefore, a sequential curation scheme was developed for compounds and activity data, and different ML classification models were generated at increasing data confidence levels and evaluated. Sequential data curation was found to systematically increase classification performance in an incremental manner due to cumulative effects of individual data curation criteria. The analysis of chemical space distributions of compound subsets at different data confidence levels revealed that the separation of compounds with different class labels in chemical space generally increased during sequential activity data curation, which was mostly due to subsequent elimination of singletons rather than compounds from analogue series. These findings provided a rationale for increasing the classification performance of ML models as a consequence of increasingly stringent data curation. Taken together, the results reported herein suggest that further attention should be paid to varying data curation and confidence levels when deriving and assessing ML models for chemical applications.