Formation Energy Prediction of Neutral Single-Atom Impurities in 2D Materials using Tree-based Machine Learning

We applied tree-based machine learning algorithms to predict the formation energy of impurities in 2D materials, where adsorbates and interstitial defects are investigated. Regression models based on random forest (RF), gradient boosting regression (GBR), histogram-based gradient-boosting regression (HGBR), and light gradient-boosting machine (LightGBM) algorithms are employed for training, testing, cross validation, and blind testing. We utilized chemical features from fundamental properties of atoms and supplemented them with structural features from the interaction of the added chemical element with its neighboring host atoms via the Jacobi-Legendre (JL) polynomials. Overall, the prediction accuracy yields optimal $\text{MAE} \approx 0.518$, $\text{RMSE} \approx 1.14$, and $R^2 \approx 0.855$. When trained separately, we obtained lower residual errors RMSE and MAE, and higher $R^2$ value for predicting the formation energy in the adsorbates than in the interstitial defects. In both cases, the inclusion of the structural features via the JL polynomials improves the prediction accuracy of the formation energy in terms of decreasing RMSE and MAE, and increasing $R^2$. This work demonstrates the potential and application of physically meaningful features to obtain physical properties of impurities in 2D materials that otherwise would require higher computational cost.