Class Incremental Learning for Visual Task using Knowledge Distillation

Abstract

The Artificial Agent's ability to enhance knowledge incrementally for new data is challenging in class incremental learning because of catastrophic forgetting in which new classes make the trained model quickly forget old classes knowledge. Knowledge distilling techniques and keeping subset of data from the old classes have been proposed to revamp models to accommodate new classes. These techniques allow models to sustain their knowledge without forgetting everything they already know but somewhat alleviate the catastrophic forgetting problem. In this study we propose class incremental learning using bi-distillation (CILBD) method that effectively learn not only the classes of the new data but also previously learned classes. The proposed architecture uses knowledge distillation in such a way that the student model directly learns knowledge from two teacher model and thus alleviate the forgetting of the old class. Our experiments on the iCIFAR-100 dataset showed that the proposed method is more accurate at classifying, forgets less, and works better than state-of-the-art methods.