AVATAR

Sign languages are highly structured languages with linguistic rules distinct from their spoken counterparts without a standard written form. However, the vast majority of information available online is provided in spoken or written language, which excludes sign languages. Many communication barriers exist for sign language users, and signing avatars (computer animations of humans) have the potential to break down these barriers for Deaf people who prefer sign language or have lower literacy in written language.

AVATAR proposes creating a signing 3D avatar able to synthesize Portuguese Sign Language from European Portuguese text. We will leverage a an annotated (phonetic-phonological, morphological and syntactic) corpus consisting of video-recorded interactions between deaf signers. This project is supported by Funda\c{c}~{a}o para a Ci\^{e}cia e Tecnologia (FCT), project CORPUS LINGISTICO E AVATAR (PTDC/LLT-LIN/29887/2017).

Project Details

Title: AVATAR

Date: Apr 9, 2020

Authors: Hugo Nicolau, Luisa Coheur, Carolina Neves, Matilde Gon\c{c}alves, Pedro Cabral

Keywords: Portuguese Sign Language, animation, accessibility


Related Publications


    • HamNoSyS2SiGML: Translating HamNoSys Into SiGML
    • Neves, Carolina and Coheur, Luisa and Nicolau, Hugo
    • Proceedings of the International Conference on Language Resources and Evaluation, , 2020
    • [ABSTRACT] [PDF]
    • Sign Languages are visual languages and the primary means of communication used by Deaf people. However, the majority of the information available online is presented through written form. Hence, it is not of easy access to the Deaf community. Avatars have gained an increase of interest due to their potential in automatically generating signs from text. Synthetic animation of conversational agents can be achieved through the use of notation systems. HamNoSys is one of these systems, which describes movements of the body through symbols. SiGML is an XML-compliant machine-readable format that enables avatars to animate HamNoSys symbols. However, there are no freely available open-source libraries that allow the conversion from HamNoSys to SiGML. In this paper, we present our open-source and cross-platform tool that performs such conversion. This system represents a crucial intermediate step in the broader pipeline of animating signing avatars. Finally,we describe two cases studies to illustrate different applications of our tool.


    • PE2LGP Animator: A Tool to Animate a Portuguse Sign Language Avatar
    • Cabral, Pedro and Gonçalves, Matilde and Santos, Ruben and Nicolau, Hugo and Coheur, Luisa
    • Proceedings of the 9th Workshop on the Representation and Processing of Sign Languages at the International Conference on Language Resources and Evaluation, , 2020
    • [ABSTRACT] [PDF]
    • Software for the production of sign languages is much less common than for spoken languages. Such software usually relies on 3D humanoid avatars to produce signs which, inevitably, necessitates the use of animation. One barrier to the use of popular animation tools is their complexity and steep learning curve, which can be hard to master for inexperienced users. Here, we present PE2LGP, an authoring system that features a 3D avatar that signs Portuguese Sign Language. Our Animator is designed specifically to craft sign language animations using a key frame method, and is meant to be easy to use and learn to users without animation skills. We conducted a preliminary evaluation of the Animator, where we animated seven Portuguese Sign Language sentences and asked four sign language users to evaluate their quality. This evaluation revealed that the system, in spite of its simplicity, is indeed capable of producing comprehensible messages.

Social Links

My Office

Instituto Superior Técnico,
Computer Science and Engineering Department,
Room 2-N9.21 - TagusPark
Avenida Professor Cavaco Silva,
2744-016 Porto Salvo,
Portugal