People who lack the ability to use their hands from conditions like motor neurone disease and cerebral palsy may one day be able to play video games that require a handheld controller through the use of customisable, wearable 3D printed bracelet.
Made with a type of resin, the bracelet works by picking up tiny movements in the user’s wrist when they move their fingers. These movements are sent wirelessly to a computer, and are then interpreted, classified and adapted using machine learning. The interpreted information can then be used to play a game, control a computer interface or communicate using a smart device.
The bracelet and program have been developed by a team of engineers in the University of Sydney’s School of Computer Science, led by undergraduate honours student Mr Stephen Lin under the supervision of Dr Anusha Withana.
“We have 3D printed a sensor bracelet that can be easily customised for individual users. It accurately detects subtle finger movements through vibrations in the carpal tunnel – an area of the wrist that contains the tendons which control the hand,” said Dr Withana.
“These subtle movements are then transmitted from the sensors to a program using Bluetooth, which interprets an individual’s movement patterns using machine learning. It then communicates this with the game. It does so almost instantaneously, allowing the user to play games that would otherwise require the use of a handheld controller.”
Mr Stephen Lin said: “We use machine learning to interpret an individual’s movements, which vary from user to user. Currently, these signals are transmitted to a computer-based program, but we hope to progress this to a free phone app so it can be easily downloadable.”
In 2020, Dr Withana received an ARC DECRA fellowship to investigate novel sensor fabrication techniques for wearable applications. Currently, the research team is funded by Cerebral Palsy Alliance and Neurodisability Assist Trust to further investigate how this technology can be used to help people living with cerebral palsy.
“We know that assistive technology holds the key to a brighter future for many children with cerebral palsy and similar disabilities, with the potential to transform communication, mobility and participation in society. Cerebral palsy is the most common physical disability in childhood globally, meaning it is vital that these tech advancements are accessible, customisable and as widely available as possible,” said Professor Nadia Badawi, CP Alliance Chair of Cerebral Palsy Research at The University of Sydney and Medical Director and Co-Head at Westmead Hospital’s Grace Centre for Newborn Intensive Care.
Cerebral palsy can cause spasticity in muscles and affect a person’s ability to move. It can have profound impacts on the ability of people to communicate with 50 percent of people living with the condition finding speech difficult or impossible, while two-thirds have difficulty with movement in one or both arms.
“This invaluable project can have a real impact in assisting children with cerebral palsy to play, learn and express themselves. At Cerebral Palsy Alliance, we’re proud to support innovative projects such as this through our grants program, which has committed more than $59 million in funding to leading researchers in 38 countries around the world,” said Professor Badawi.
The sensors were designed using computational fabrication techniques, with the components able to be printed using a low-cost, commercial 3D printer. The team has also developed a simple, easy-to-use tool that allows users to customise the sensor to fit their needs.
The researchers plan to release the tools to create sensor bracelets as open-source software, with the aim of improving accessibility for people living with disability worldwide.
“Accessibility shouldn’t come at a huge cost. Our mission is to provide an affordable, easy-to-use solution to assist people around the world who are living with disability. We want this technology to be available to anyone who needs it, which is why we plan to release it publicly without IP,” said Mr Lin.
Gopi Kitnasamy, Head of Rehabiliation Services at MJF Charitable Foundation, a Sri Lankan not-for-profit organisation the researchers are partnered with, says that new technology is helping individuals with disabilities to perform functions that might otherwise be difficult or impossible.
He hopes the device will also benefit his son, who has cerebral palsy.
“Access to assistive devices is essential for many people with disabilities. However, access to such devices has been a major issue in low-to-middle income countries,” he said.
“My 23-year-old son lives with cerebral palsy and uses a wheelchair. He loves watching car and bike races and playing video games, with a keen interest in car racing and shooting games.
“As he has lot of dyskinetic (involuntary) movements, he finds it difficult to use joysticks or controllers, so he ends up just watching as we play for him. He selects the tracks, cars, and drivers, but is never able to participate by actually playing. This is very frustrating for both him and us.
“We have long dreamt of finding a way for him to fully play the game himself. We have tried various controllers with sensors, but these devices are very expensive, and they didn’t help in the way he wanted.
“I was so thrilled and excited to learn about the researchers’ sensor bracelet, which appears promising in terms of functionality and affordability. We hope the research progresses, not only to give people like my son access to video games, but also to improve rehabilitative services, and to make the impossible possible.
