Exploiting Locality Sensitive Hashing - Clustering and gloss feature for sign language production

Abstract

The automatic Sign Language Production (SLP), which converts spoken language sentences into continuous sign pose sequences, is crucial for the digital interactive application of sign language. Long text sequence inputs make current deep learning-based SLP models inefficient and unable to fully take advantage of the intricate information conveyed by sign language, resulting in the fact that the generated skeleton pose sequences may not be well comprehensible or acceptable to individuals with hearing impairments. In this paper, we propose a sign language production method that utilizes Locality Sensitive Hashing-Clustering to automatically aggregate the similar and identical embedded word vectors, capture long-distance dependencies, thereby enhance the accuracy of SLP. And a multi-scale feature extraction network is designed to extract local feature of gloss and combine it with embedded text vectors to enhance text in-formation. Extensive experimental results on the challenging RWTH-PHOENIX-Weather 2014T (PHOENIX14T) dataset show that our model outperforms the baseline method.