A Signing Avatar on Stage in Realtime

Topic: A Signing Avatar on Stage in Realtime

When: 17 January at 12.00-13.15

Speaker: Henrik Garde, Systems developer, Humanities Lab, Lund University 

Where: Online

Spoken language: English
The presentation and  following discussion was interpreted from English speech into Swedish sign language. Wi will tyy to returv with a video of the presentation hat include the sign interpretation.

Abstract

As a research infrastructure, Lund University Humanities Lab is constantly developing skills and methodology to support a wide range of researchers. The product is workflows or pipelines that describes how data is captured and transformed or computed in several steps to finally end up as a final result, which could be a visualisation or something else. To evaluate the quality and validity of data going through that process step by step can be accomplished only in collaboration with lab users - who are experts in their field. Immediate visualisations seem very helpful in that process. 

Teaming up with deaf actors, directors and producers from the Swedish national theatre (Riksteatern and Crea), appeared to be a perfect setup for improving and assessing workflows for sign language being captured from a real person and instantly applied to a virtual reality (VR) character - an avatar. 

A signing avatar in real-time requires a constant and accurate capturing of synchronized fingers, body and eye movement with face expressions to be streamed to a VR engine. Many methods including fuzzy logic needed to be explored. Inspired by the mindset of ‘production lines’ from this collaboration, with strict procedures and constant assessment has improved other lab pipelines already employed by researchers from fields other than sign language, e.g., multimodal communication and gesture research, research in education and training. Efficiency and transparency is much appreciated by researchers in the national infrastructures (Infravis, Huminfra, etc) who travels long distances to collaborate with the most appropriate experts in their facilities.