Multi-scale information transport generative adversarial network for human pose transfer

Abstract

Human pose transfer, a challenging image generation task, aims to transfer a source image from one pose to another. Existing methods often struggle to preserve details in visible regions or predict reasonable pixels for invisible regions due to inaccurate correspondences. In this paper, we design a novel multi-scale information transport generative adversarial network, composed of Information Transport (IT) blocks to establish and refine the correspondences progressively. Specifically, we compute a transport matrix to warp the source image features by integrating an optimal transport solver in our proposed IT block, and use IT blocks to refine the correspondences in different resolutions to preserve rich details of the source image features. The experimental results and applications demonstrate the effectiveness of our proposed method. We further present an image-specific optimization using only a single image. The code is available for research purposes at https://github.com/Zhangjinso/OT-POSE.