Virtual try-on with Pose-Aware diffusion models

Abstract

Image-based virtual try-on (VTON) refers to the task of synthesizing realistic images of a person wearing a target garment based on reference images. Existing approaches use diffusion models that demonstrate outstanding performance in image synthesis tasks but often fail in preserving the pose and body features of the reference person in certain cases. To address these limitations, we propose Pose-Aware Virtual Try-ON (PA-VTON), a methodology that uses a pretrained diffusion-based VTON framework and additional modules that specify in preserving the information of a person’s attributes. Our proposed module, PoseNet, adds spatial conditioning controls to the VTON process to enhance pose consistency preservation. Experimental results on two benchmark datasets demonstrate that our proposed method quantitatively improves image synthesis performance while qualitatively resolving issues such as ghosting effects and improper generation of body parts that previous methods struggled with.