Hierarchical Classification and Regression with Feature Selection

Abstract

Previous studies proposed different hierarchical estimation approaches for solving certain specific domain problems. They usually combine two or more estimation models in a hierarchical fashion. Therefore, in our previous work [2], we proposed a hierarchical approach for generic purposes, the Hierarchical Classification and Regression (HCR), that incorporates classification and estimation techniques. The HCR [2] approach significantly outperformed three benchmark flat estimation models. Having seen the potential of the proposed HCR as a generic hierarchical regression scheme, we propose to further improve the HCR by introducing feature selection (FS) techniques to the HCR. In order to thoroughly investigate the effect of FS on the HCR, we examine different numbers of attributes remained after feature selection with respect to datasets of various sizes. The results showed that the HCR with linear regression performed significantly better than the other HCRs while feature selection helped lower the RMSE slightly with only 50% of the original features.