An Automatic Vision Transformer Pruning Method Based on Binary Particle Swarm Optimization

Abstract

This paper presents an automatic vision transformer pruning method that aims to alleviate the difficulty of deploying vision transformer models on resource-constrained devices. The proposed method aims to automatically search for the optimal pruned model by removing irrelevant units while maintaining the original accuracy. Specifically, the model pruning is formulated as an optimization problem using binary particle swarm optimization. To demonstrate its effectiveness, our method was tested on the DeiT Transformer model with CIFAR-10 and CIFAR-100 datasets. Experimental results demonstrate that our method achieves a significant reduction in computational cost with slight performance degradation.