Top of page
Technology

Siberian Federal University student develops AI method to understand sign language

woman communicates in sign language online at a laptop

Vladislava Zhukovskaya, a student at the School of Space and Information Technology at Siberian Federal University, has developed a method for recognizing Russian sign language gestures from video sequences, taking into account the language dialect of the Siberian region.

Sign language is not a universal language for all deaf people in the world. Depending on the location of the hard-of-hearing, sign language dialects differ. To ensure communicative interaction, Vladislava developed methods for recognizing sign language from visual data using neural networks.

“In order to implement the gesture recognition software, I used a modified LSTM neural network model that can highlight the key points of the palm then tracks the movements of these points. The system can reliably understand 6 different gestures. Currently, I continue filling the database of gestures so that the system can understand more words. Then I will have to figure out how to recognize sentences, because often pauses in sign speech are minimal or absent. I am going to continue my work during my master’s degree training,” said Vladislava.

This work was carried out within the framework of interdisciplinary cooperation between the School of Space and Information Technology and the School of Philology and Language Communication of Siberian Federal University.

“Lyudmila Kulikova, director of the School of Philology and Language Communication, contacted me in September. We discussed possible interdisciplinary cooperation between our schools. Among all the tasks, Vladislava was interested in gesture recognition based on visual data. It turned out that sign language differs significantly in different regions of Russia, and our Siberian region is no exception, so gesture recognition is a complex and interesting task,” shared Anna Pyataeva, supervisor, assistant professor of the Department of Artificial Intelligence Systems at the School of Space and Information Technology.

The thesis titled “Development of methods for recognizing the Russian language from video sequences on the basis of the language dialect of the Siberian region” by Vladislava Zhukovskaya became a winner of the All-Russian Engineering Competition in the Best Bachelor’s Thesis nomination.

You might also like

Close-up of Mother Holding Feet of Her Little Baby Close-up of Mother Holding Feet of Her Little Baby

Tech-based peer therapy effective for perinatal depression in LMICs

A University of Liverpool study conducted in rural Pakistan in…

Apple Watch on man hand - display showing apps Apple Watch on man hand - display showing apps

Equal access to digital tech could improve heart health

The use of technology-based health solutions, such as fitness trackers,…

Machine Deep learning algorithms, Artificial intelligence AI , Automation and modern technology in business as concept Machine Deep learning algorithms, Artificial intelligence AI , Automation and modern technology in business as concept

AI predicts dementia risk in American Indian and Alaska Native elders

Machine learning algorithms utilizing electronic health records can effectively predict…

person reading with Focus 40 Blue Braille Display person reading with Focus 40 Blue Braille Display

New guidelines to help digital tech meet accessibility compliance

The Australian Human Rights Commission has released new guidelines to…