Co-speech gesture generation for a hindi conversing virtual agent/robot

Abstract

Humans make use of gestures while interacting to enhance the effectiveness of their communication. With the increasing use of humanoid robots and virtual agents, researchers have been trying to make the robots more human-like, improving their perceived likeability and anthropomorphize them. Recent works in this domain have used learning-based approaches to generate co-speech gestures, however, most of this work has been done for the English language. Our work aims at creating a dataset to study the correlation between gestures and audio and text for the Hindi language. We further aim to create an end to end model for co-speech gesture generation for Hindi conversing virtual agents or humanoid robots.