Generalised Entropies for Decision Trees in Classification Under Monotonicity Constraints

Abstract

Several decision-making approaches involve ordinal labelling between feature values and decision outcomes. These issues refer to ordinal classification under monotonicity constraints. Recently, some machine learning approaches have been designed to deal with these kinds of problems. Indeed, numerous experiments have shown that these algorithms are widely used in real-life applications because of their flexibility and efficiency in terms of interpretation and predictions. In this paper, we introduce novel approaches for measuring feature quality and information quantity, called Rényi-Tsallis Monotonic Tree (RTMT), which uses the advantages of Rényi and Tsallis entropies while incorporating monotonicity constraints through an optimisation framework. Moreover, we introduce Mono-CART, a variant of the CART approach adapted for monotonic classification. New decision tree algorithms are designed on the basis of aforementioned entropies while considering the monotonicity constraints within an optimisation system. The experiments conducted using some benchmark datasets demonstrate the superiority of the proposed approaches compared to existing methods.