Leveraging ChemBERTa and machine learning for accurate toxicity prediction of ionic liquids

Abstract

Background:

Accurately predicting the toxicity of ionic liquids is essential for promoting sustainable chemical applications while mitigating environmental and health risks. The increasing complexity and volume of data inherent in toxicology have stimulated interest in machine learning models because they are attractive approaches that can identify patterns among predictors and responses that may not be obvious through classical statistical methodologies.

Methods:

This study introduces a hybrid framework that combines ChemBERTa-based chemical structure embeddings with Convolutional Neural Networks (CNNs), XGBoost, and Support Vector Regression (SVR). ChemBERTa embeddings, derived from SMILES strings, were enriched with molecular descriptors and fingerprints, with dimensionality reduced using Principal Component Analysis (PCA). To further enhance performance, model optimization was conducted through Optuna, ensuring the best configuration of hyperparameters.

Significant Findings:

CNNs demonstrated superior performance, achieving an R-squared value of 0.865, a Root Mean Squared Error (RMSE) of 0.390, and a Pearson correlation coefficient of 0.937. XGBoost followed closely with an R-squared value of 0.824, an RMSE of 0.462, and a Pearson correlation of 0.923. SVR also performed competitively, with an R-squared value of 0.797 and an RMSE of 0.496. Notably, the inclusion of ChemBERTa embeddings significantly enhanced model accuracy, as evidenced by the results of ablation studies. This study highlights the potential of hybrid frameworks that combine deep learning with classical machine learning approaches to predict ionic liquid (IL) toxicity. These findings offer valuable insights for safer chemical design, promoting sustainable innovation while supporting regulatory decision-making.