Top of page
Technology

Siberian Federal University student develops AI method to understand sign language

woman communicates in sign language online at a laptop

Vladislava Zhukovskaya, a student at the School of Space and Information Technology at Siberian Federal University, has developed a method for recognizing Russian sign language gestures from video sequences, taking into account the language dialect of the Siberian region.

Sign language is not a universal language for all deaf people in the world. Depending on the location of the hard-of-hearing, sign language dialects differ. To ensure communicative interaction, Vladislava developed methods for recognizing sign language from visual data using neural networks.

“In order to implement the gesture recognition software, I used a modified LSTM neural network model that can highlight the key points of the palm then tracks the movements of these points. The system can reliably understand 6 different gestures. Currently, I continue filling the database of gestures so that the system can understand more words. Then I will have to figure out how to recognize sentences, because often pauses in sign speech are minimal or absent. I am going to continue my work during my master’s degree training,” said Vladislava.

This work was carried out within the framework of interdisciplinary cooperation between the School of Space and Information Technology and the School of Philology and Language Communication of Siberian Federal University.

“Lyudmila Kulikova, director of the School of Philology and Language Communication, contacted me in September. We discussed possible interdisciplinary cooperation between our schools. Among all the tasks, Vladislava was interested in gesture recognition based on visual data. It turned out that sign language differs significantly in different regions of Russia, and our Siberian region is no exception, so gesture recognition is a complex and interesting task,” shared Anna Pyataeva, supervisor, assistant professor of the Department of Artificial Intelligence Systems at the School of Space and Information Technology.

The thesis titled “Development of methods for recognizing the Russian language from video sequences on the basis of the language dialect of the Siberian region” by Vladislava Zhukovskaya became a winner of the All-Russian Engineering Competition in the Best Bachelor’s Thesis nomination.

You might also like

Young woman in wheelchair using computer at home Young woman in wheelchair using computer at home

Tech jobs offering new opportunities for persons with disabilities

A team of workers at Makeables, an employment arm of…

Blind person using computer with braille display Blind person using computer with braille display

New guide to assist the higher education sector to procure accessible ICT

A new guide designed to assist the higher education sector…

Global CP community submit ideas for better, more accessible technology solutions

October 6th marks World Cerebral Palsy Day and people living…

a girl uses a smartwatch a girl uses a smartwatch

New research tests wearable sensors for children with cerebral palsy

Children with cerebral palsy will wear specially designed wearable sensors…