Sign Language-to-Speech Glove

Over the past few years there has been a very rapid growth in the technological industry. However, there seems to be one specific niche that hasn?t been tapped to its fullest potential. Very little work has been done on assistive technologies, especially for auditory and speech impairments. Thus, th

Project Title

Sign Language-to-Speech Glove

Project Area of Specialization

Wearables and Implantable

Project Summary

Over the past few years there has been a very rapid growth in the technological industry. However, there seems to be one specific niche that hasn’t been tapped to its fullest potential. Very little work has been done on assistive technologies, especially for auditory and speech impairments. Thus, there exists a wide communication deficit in our community and our project aims to bridge that gap.

This project aims to create a fully functional, modular, and durable glove that can be used to convert American Sign Language (ASL) to audible speech. It is made using affordable and accessible parts that can be easily replaced in the event of any malfunctioning. An Android App is also to be developed along with an Arduino chip where the main processing would take place. This chip would utilize machine learning algorithms to reinforce correct results and thus increase the accuracy of the system.

Unlike the devices that already exist to convert sign language to audio with the help of image processing, we provide a method that will neither require proper lighting conditions, nor accurate camera positioning as it uses machine learning algorithms to provide results on the go. All that needs to be done is to wear the portable glove. We aim to use off the shelf sensors readily available to achieve the maximum accuracy rate possible unlike other gloves that use specially designed yarn-based sensors that are difficult to replace hence hinder maintenance

Project Objectives

The first major objective of this project is to attain a useful signal from the flex sensors, giving an accuratemeasure ofthe bend ofthe users’finger. Also, to gauge the movement and positioning of the wrist additional sensors like accelerometer and gyroscope are to be calibrated.

Next, we aim to employ noise reduction techniques such as digital filters to efficiently remove any undesirable background electrical noise, in order to achieve a clean and smooth signal.

Furthermore, we aim to develop a smart machine learning algorithm embedded in our Arduino Nano to match the acquired signal with the ones stored in our database.

Lastly, we aim to design an Android application which utilizes Bluetooth Low Energy communication protocols accurately display the acquired gesture.

Project Implementation Method

Our design consists of three basic building blocks (Sensory circuit (flex sensor, accelerometer, and gyroscope), Arduino block (Data Categorization, Machine Learning, and Bluetooth Loe Energy Send), and the Android App (Bluetooth Loe Energy Send and Audio/visual output), first two of which reside on the glove itself and third within the smartphone app. The first two modules are realized using a microprocessor and sensing circuitry embedded on the glove. These acquire the change in the user’s wrist movements and finger-bends to get a picture of what symbol the user is trying to make; this is done using flex sensors, a gyroscope, and an accelerometer. It then checks this symbol with a local library of symbols using machine learning to find the closest match. The detected symbol is then sent to the smartphone via Bluetooth Low Energy (BLE) where it is displayed on the users’ screen along with a text-to-speech software that converts the symbol to audio to be played on the phone’s speaker

Benefits of the Project

The sole purpose of this project is to give back to the community by bridging the communication deficit that exists between the able and the impaired. More effective communication means that people who do have speech impairments will have better employment opportunities, more confidence, and easier social interactions. Not only will this project allow people with auditory and speech disabilities to communicate more effectively and more confidently, but it will also allow everyone else to use it as a tool to learn the American Sign Language (ASL). Currently only 1% of the US population knows how to speak ASL, and that number would be much lower for less developed countries. Providing an intuitive way to learn ASL would also greatly benefit the community by normalizing the use of the language and thus making people with auditory and speech impairment feel less outcasted.

Technical Details of Final Deliverable

• Produce a successful simulation of the smartglove
• Deliver a fully functional Android application with BLE communication protocols.
• An Arduino code that employs machine learning and pattern recognition techniques to convert signal received from sensors (flex sensors, accelerometer, gyroscope) into information that can be sent to the Android application using BLE.
• Curate an ASL database along with their corresponding English words and audio signals.
• Deliver a working prototype of our product.

Final Deliverable of the Project

HW/SW integrated system

Core Industry

Health

Other Industries

Core Technology

Wearables and Implantables

Other Technologies

Sustainable Development Goals

Good Health and Well-Being for People

Required Resources

If you need this project, please contact me on contact@adikhanofficial.com