An Elastic Neural Network Toward Multi-Grained Re-configurable Accelerator

A bisection topology of neural networks (NN) is developed instead of conventional full connection (FC) fashion for NNs. Each neuron only communicates with two synapses from its previous neurons in adjacent, and outputs the data to two neurons in the post layer. A large amount of neurons and synapses are expected to symmetrically implement by the computational hardware in parallel. In this manner, the entire network can be partitioned into arbitrary diamond-shaped pieces (seen as DiaNet) for behaving the NN functions without any redundancy theoretically. Assuming such topology is implemented on-chip in parallel, the DiaNets perform multi-grained re-configuration to offer flexible function units. Various behaviors of conventional NNs are efficiently retrieved by the proposed DiaNet topology while maintaining high fidelity of results. Also, two optimization technologies such as overlapping and reshaping are proposed to further reduce the synapses. On the Wine dataset, our results show that the number of synapses is reduced to 36.3% without accuracy loss. Finally, the bit precision of the DiaNet is investigated to suggest the guideline toward efficient hardware implementations.