Verifying the value and veracity of extreme gradient boosted decision trees on a variety of datasets

Abstract

Learning models are used widely in both, industries and in areas of our daily lives. They thus witness a large amount of improvement and research. Gradient Boosted Machines (GBM) was one approach, which was known to give accurate solutions, and used ensemble trees to build upon weak learners for classifying the data. Over time the need for a more scalable, modifiable, and accurate system was felt, and building upon GBMs an improved variant called eXtreme GBM (XGBoost) was proposed. XGBoost gave highly accurate results in many international competitions and presented itself as an ideal learning model ready to be adapted for wide usage. Our objective was to experimentally verify the value and veracity of this new approach, and towards this, we analyzed and compared it with traditional and benchmark algorithms, on a variety of datasets. XGBoost outperformed its counterparts, attesting to the fact that it indeed holds promise.