CS-Mixer: A Cross-Scale Vision Multilayer Perceptron With Spatial–Channel Mixing

Abstract

Despite simpler architectural designs compared with vision transformers (ViTs) and convolutional neural networks, vision multilayer perceptrons (MLPs) have demonstrated strong performance and high data efficiency for image classification and semantic segmentation. Following pioneering works such as MLP-Mixers and gMLPs, later research proposed a plethora of vision MLP architectures that achieve token-mixing with specifically engineered convolution- or attentionlike mechanisms. However, existing methods such as $\text{S}^{\text{2}}$ -MLPs and PoolFormers typically model spatial information in equal-sized spatial regions and do not consider cross-scale spatial interactions, thus delivering subpar performance compared with transformer models that employ global token mixing. Further, these MLP token-mixers, along with most ViTs, only model one- or two-axis correlations among space and channels, avoiding simultaneous three-axis spatial–channel mixing due to its computational demands. We, therefore, propose CS-Mixer, a hierarchical vision MLP that learns dynamic low-rank transformations for tokens aggregated across scales, both locally and globally. Such aggregation allows for token-mixing that explicitly models spatial–channel interactions, made computationally possible by a multihead design that projects to low-dimensional subspaces. The proposed methodology achieves competitive results on popular image recognition benchmarks without incurring substantially more computing. Our largest model, CS-Mixer-L, reaches 83.2% top-1 accuracy on ImageNet-1k with 13.7 GFLOPs and 94 M parameters.