ChemBERTa embeddings and ensemble learning for prediction of density and melting point of deep eutectic solvents with hybrid features

Abstract

Deep eutectic solvents (DESs) are sustainable alternatives to traditional solvents, but their structural complexity makes accurate prediction of melting points and densities challenging. This study utilizes ChemBERTa, a pre-trained Transformer model, to extract high-dimensional embeddings from Simplified Molecular Input Line Entry System (SMILES) strings, effectively capturing complex molecular interactions and subtle structural features. Through feature importance analysis, we identified missing molecular information in the ChemBERTa embeddings and supplemented it with select physicochemical descriptors from RDKit, creating a feature set that enhances both interpretability and predictive accuracy. Optimized ensemble models, including ExtraTreesRegressor (ETR) and XGBRegressor (XGBR), are then applied to this enriched feature set, achieving notable improvements in prediction accuracy for DES melting point and density. Rigorous grid search and ten-fold cross-validation ensure model robustness and generalizability. Experimental results confirm the effectiveness of this approach, underscoring the transformative role of pre-trained deep learning models in chemical informatics and supporting scalable, sustainable DESs design.