Algorithmic mapping of neural network models onto parallel SIMD machines

Abstract

The authors consider parallel implementation of neural network computations of fine grain SIMD machines. The authors show a mapping of a neural network having n nodes and e connections onto a parallel machine having (n+e) PEs arranged in an array of square root n+e* square root n+e PEs such that routing for each update iteration of the recall phase can be performed in 24( square root n+e-1) elemental data shifts. The array uses simple PEs and few local registers to perform the routing and computations. The method is simple and is well suited for implementation of various classes of neural networks on many currently available parallel machines.<>