Sign Language Video Generation from Text Using Generative Adversarial Networks

Abstract

This work presents a technique developed by utilizing Generative Adversarial Networks (GANs) to generate Sign Language videos. Sign Language is the main mode of communication for people in the hearing impaired community. The process of teaching sign language is difficult as there are not a lot of tools available for this purpose. Generative artificial intelligence can be very helpful for this task as it is able to learn from the limited data and is able to generate various images and videos. In this work, Conditional GANs (cGANs) were employed to generate videos for Indian Sign Language (ISL) based on a text input. It is found that the results obtained from cGANs exhibit superior quality and control based on the performance metrics such as SSIM, FID and MSE values. The effectiveness of the cGANs in generating accurate and visually appealing sign language videos highlights their potential for teaching sign language and improving sign language communication systems.