Exploring how base model combination affects the results of a “stacking” ensemble machine learning model: An applied study on optimization of heteroatom doped carbon data

Abstract

This study explores stack models for electrochemical analysis, incorporating base models (decision trees, linear regression, and k-nearest neighbors) and a meta-model. It reveals that the order of stacking base models affects predictions, often yielding multiple solutions. To address this “uncertainty,” a novel “sorting” technique was applied during meta-model training. This approach significantly reduced model uncertainty, achieving the most accurate predictions and minimizing order deviations (mean absolute error of 37.92388; standard deviation reduced from 6.19 × 10⁻¹⁵ to 0). The refined model was applied to analyze synergies in electrochemical and material properties using feature importance tools, such as SHAP, Feature Permutation Importance (FPI), and Partial Dependence Plots (PDP). Key insights for heteroatom-doped carbon supercapacitors suggest maximizing surface area and nitrogen, sulfur, and boron doping while minimizing current density and acidic electrolyte concentration. Optimal oxygen and phosphorus doping levels were ∼ 15 % and ∼ 2.5 %, respectively. FPI ranked nitrogen > surface area > electrolyte concentration > oxygen > current density > defect ratio > sulfur > boron > phosphorus. PDP revealed that dual heteroatom doping (e.g., nitrogen and oxygen) may outperform doping with five heteroatoms. These findings enhance machine learning's reliability in materials science, offering pathways for efficient synthesis and optimization in two-dimensional materials.