Converting Neural Networks to Rule Foam

Abstract

A system of rules can approximate a trained neural classifier after sampling from that classifier. The rules define a generalized probability mixture that then describes the classifier. The size or granularity of the rule if-parts defines a foam-like structure with a few large rule if-part set bubbles in patternclass centers and many smaller if-part sets near class borders. The rule foam's mixture gives a Bayesian posterior over the rules. The posterior describes the relative importance of each rule for each observed input and output. The foam's mixture also gives the conditional variance that measures the uncertainty in its output. So the rule base is statistically interpretable as well as modular and adaptive. A rule foam with 1000 Gaussian rules approximated a 96.85% accurate MNIST neural classifier and had itself 95.66% classification accuracy. Foams can also approximate other foams. Some approximator foams out-performed the target foam that generated their training data. The rule foam's granularity mitigates the rule explosion inherent in the rule-based approximator's graph-covering structure