Learning to Instruct Learning

One reason why deep neural networks require lots of data is that most current training methods are only driven by the task goal information. We propose a novel instructor which can guide networks to learn useful abstraction. Since the instructor provides additional learning power, the efficiency of data is significantly improved. To get appropriate instructor, we design a generative instructor mechanism which supports learning an instructor generator from multiple tasks. The generator can generate the corresponding instructor for different tasks by using fast weights. Experiment results demonstrate the efficiency and robustness of the generated instructor. Meanwhile, our generator also shows the property relating to continuous learning.