Gaits Generation from a Mimetic Word based on Sound Symbolism

Abstract

The Japanese language is known to have a rich vocabulary of mimetic words, which have the property of sound symbolism; Phonemes that compose the mimetic words are strongly related to the impression of various phenomena. Especially, human gait is one of the most commonly represented phenomena by mimetic words expressing its visually dynamic state. Sound symbolism is useful for modeling the relation between gaits and mimetic words intuitively, but there has been no study on their intuitive generation. Most previous gait generation methods set specific class labels such as “elderly” but have not considered the intuitiveness of the generation model. Thus, in this paper, we propose a framework to generate gaits from a mimetic word based on sound symbolism. This framework enables us to generate gaits from one or more mimetic words. It leads to the construction of a generation model represented in a continuous feature space, which is similar to human intuition. Concretely, we train an encoder-decoder model conditioned by a “phonetic vector”, a quantitive representation of mimetic words, with an adaptive instance normalization module inspired by style transfer. The phonetic vector is a dense description of the intuitive impression of a corresponding gait and is calculated from many mimetic words in the HOYO dataset, which includes gait motion data and corresponding mimetic word annotations. Through experiments, we confirmed the effectiveness of the proposed framework.