Compressing Interpretable Representations of Piecewise Linear Neural Networks using Neuro-Fuzzy Models

Abstract

We present Rectified Linear Unit based Local Linear Model Tree (ReLUMoT). A model that bridges the gap between Piecewise Linear Neural Networks (PLNN) and Local Model Networks (LMN) like those resulting from the LoLiMoT algorithm. Essentially, we perform the input space partitioning of LoLiMoT by training a PLNN and extracting its linear regions. These become the input space partitions of ReLUMoT. From the perspective of PLNNs our approach compresses and smoothens low-dimensional models, while making them interpretable. From the perspective of LoLiMoT, our approach replaces the incremental and heuristic input space partitioning scheme with gradient-based training of a neural network, which is considerably more flexible.