Top of page
Technology

Siberian Federal University student develops AI method to understand sign language

woman communicates in sign language online at a laptop

Vladislava Zhukovskaya, a student at the School of Space and Information Technology at Siberian Federal University, has developed a method for recognizing Russian sign language gestures from video sequences, taking into account the language dialect of the Siberian region.

Sign language is not a universal language for all deaf people in the world. Depending on the location of the hard-of-hearing, sign language dialects differ. To ensure communicative interaction, Vladislava developed methods for recognizing sign language from visual data using neural networks.

“In order to implement the gesture recognition software, I used a modified LSTM neural network model that can highlight the key points of the palm then tracks the movements of these points. The system can reliably understand 6 different gestures. Currently, I continue filling the database of gestures so that the system can understand more words. Then I will have to figure out how to recognize sentences, because often pauses in sign speech are minimal or absent. I am going to continue my work during my master’s degree training,” said Vladislava.

This work was carried out within the framework of interdisciplinary cooperation between the School of Space and Information Technology and the School of Philology and Language Communication of Siberian Federal University.

“Lyudmila Kulikova, director of the School of Philology and Language Communication, contacted me in September. We discussed possible interdisciplinary cooperation between our schools. Among all the tasks, Vladislava was interested in gesture recognition based on visual data. It turned out that sign language differs significantly in different regions of Russia, and our Siberian region is no exception, so gesture recognition is a complex and interesting task,” shared Anna Pyataeva, supervisor, assistant professor of the Department of Artificial Intelligence Systems at the School of Space and Information Technology.

The thesis titled “Development of methods for recognizing the Russian language from video sequences on the basis of the language dialect of the Siberian region” by Vladislava Zhukovskaya became a winner of the All-Russian Engineering Competition in the Best Bachelor’s Thesis nomination.

You might also like

Global IT Challenge for Youth with Disabilities continues amid resurgence of COVID

Amidst the resurgence of COVID-19, the Global IT Challenge for…

The Yang-Tan Institute of Employment and Disability has joined a multi-institution team that has received a $5 million grant from the National Science Foundation (NSF) to help create better job outcomes for people with autism. The Yang-Tan Institute of Employment and Disability has joined a multi-institution team that has received a $5 million grant from the National Science Foundation (NSF) to help create better job outcomes for people with autism.

Research shows AI can improve stroke diagnostics

A new study presented on Thursday at the Society of…

Person using the mobile App Person using the mobile App

TESDA introduces online skills training program Mobile App

The Technical Education and Skills Development Authority (TESDA) has launched…

baby with app baby with app

South Korea’s hospitals to develop AI-based Platform for autism diagnosis

South Korea’s two biggest hospitals plan to develop an artificial…