Modeling and optimizing PE utilization rate for systolic array based CNN accelerators

Due to its efficiency, energy-saving, and abundant data reuse, systolic array has been a popular choice for Convolutional Neural Network (CNN) accelerators. Dataflow of the systolic array defines computation mapping strategy and memory access and it is one of the most important design points of accelerators. Most conventional accelerator designs choose a single dataflow and optimize around it. This may influence the Processing Element (PE) utilization rate and cause waste of computing resources and energy. This work introduces a self-paced method to alleviate this problem. We analyse and quantify the PE utilization rate related to the three basic dataflows and build a model called PEU-sim to explore workload-oriented flexible dataflow. Experiments show by combining three dataflows, we are able to raise more than 10% of PE utilization rate for most neural networks and we get the highest of 12.4% for MobileNet.