Being given €5.6 million from the European Commission, 17 European partners are working on an app called SignON; the app will make Communication easier, all over Europe, between people with hearing disabilities and those who can hear.
Each spoken language has its own unique sign language, making Communication highly difficult across Europe. The project is focused on Irish, British, Dutch, Flemish, and Spanish sign languages. In terms of spoken languages, the group is concentrating on Irish, English, Dutch, and Spanish.
Dr. Dimitar Shterionov, scientific project leader from Tilburg University in the Netherlands, and one of the app’s researchers said, “The project is extensive. From a technological research perspective, it is mainly about recognition, analysis, and an understanding of the text, speech, and sign language. The app automatically translates into the chosen language using a model that is a hybrid of a deep learning and rule-based model.”
“When something is translated into sign language, it is represented by a 3D avatar. When sign language is translated into a spoken language, it is articulated using artificial speech,” he added. So like a Siri or Alexa type voice.
The project is relying heavily on the advice and reactions from the deaf and hearing disabled community. “Communication with people who are deaf and hard of hearing will steer the development of the app. We will evaluate the app in the interim so that we can incorporate the feedback from this group,” Shterionov said.
This cooperation makes the project unique, ensuring that the app will be relevant to the needs of the user target group.
The four Dutch partners involved in the project are: Radboud University, Nederlandse TaalUnie (Dutch Language Association), the Dutch Language Institute, and Tilburg University. The projects involve various experts in the field of Natural Language Processing (NLP), Machine Learning (ML), Deep Learning (DL), Machine Translation (MT), Linguistics, Deaf Studies, education, and 3D graphics.
