A 4-way Matrix Multiply Unit for High Throughput Machine Learning Accelerator

Abstract

With the rapid growth of modern applications based on machine learning, neural network (NN) algorithm has been widely used in various fields. Accordingly, machine learning accelerators with high performance based on FPGA and ASIC design have become necessary. Machine learning accelerators generally include a matrix multiply unit that performs arithmetic. However, despite the development of dedicated hardware, some NN algorithms still suffer from performance degradation due to computation bounds in the matrix multiply units. Resolving the computation bound is crucial for high throughput machine learning accelerator. In this paper, we propose a 4-way matrix unit to resolve the computation bound by minimizing idle state operation logic and improving overall utilization. A 4-way matrix multiply unit resulted in an average throughput improvement of 29 percent and a 24 percent increase in the total area, comparing to the conventional systolic array-based matrix multiply unit.