Multi-attribute dynamic attenuation learning improved spiking actor network

Abstract

Deep reinforcement learning (DRL) has shown promising results in solving robotic control and decision tasks, which can learn the high-dimensional state and action information well. Despite their successes, conventional neural-based DRL models are criticized for low energy efficiency, making them laborious to be widely applied in low-power electronics. With more biologically plausible plasticity principles, spiking neural networks (SNNs) are now considered an energy-efficient and robust alternative. The most existing dynamics and learning paradigms for spiking neurons with a common Leaky Integrate-and-Fire (LIF) neuron model often result in relatively low efficiency and poor robustness. To address these limitations, we propose a multi-attribute dynamic attenuation learning improved spiking actor network (MADA-SAN) for reinforcement learning to achieve effective decision-making. The resistance, membrane voltage and membrane current of spiking neurons are updated from a fixed value into dynamic attenuation. By enhancing the temporal relation dependencies in neurons, this model can learn the spatio-temporal relevance of complex continuous information well. Extensive experimental results show MADA-SAN performs better than its counterpart deep actor network on six continuous control tasks from OpenAI gym. Besides, we further validated the proposed MADA-LIF can achieve comparable performance with other state-of-the-art algorithms on MNIST and DVS-gesture recognition tasks.