Quaternion Vector Quantized Variational Autoencoder

Abstract

Vector quantized variational autoencoders, as variants of variational autoencoders, effectively capture discrete representations by quantizing continuous latent spaces and are widely used in generative tasks. However, these models still face limitations in handling complex image reconstruction, particularly in preserving high-quality details. Moreover, quaternion neural networks have shown unique advantages in handling multi-dimensional data, indicating that integrating quaternion approaches could potentially improve the performance of these autoencoders. To this end, we propose QVQ-VAE, a lightweight network in the quaternion domain that introduces a quaternion-based quantization layer and training strategy to improve reconstruction precision. By fully leveraging quaternion operations, QVQ-VAE reduces the number of model parameters, thereby lowering computational resource demands. Extensive evaluations on face and general object reconstruction tasks show that QVQ-VAE consistently outperforms existing methods while using significantly fewer parameters.