Hand Gesture Vocalizer for Mute People

Those with speech impairment due to autism or cerebral palsy, use sign language for communication, and ? understandably ? find a lot of difficulties in conveying their messages. This difficulty equally holds at the receiver?s side for trusted understanding and interpretation. To overcome their

Project Title

Hand Gesture Vocalizer for Mute People

Project Area of Specialization

Robotics

Project Summary

Those with speech impairment due to autism or cerebral palsy, use sign language for communication, and – understandably – find a lot of difficulties in conveying their messages. This difficulty equally holds at the receiver’s side for trusted understanding and interpretation. To overcome their communication barrier, an Arduino UNO based augmentative and alternative communication (AAC) system is proposed, which is capable of translating hand-gestures into text, which are later vocalized. The system requires the end-users to be familiar with hand gestures for their essential needs a priori; they are normally taught by speech-language pathologists. The core component of the system is a flex sensor, whose resistance increases with its bend. We stitch such sensors on a hand glove, and each bend of a finger – due to a different resistance – encodes a unique gesture; thereby allowing multiple gestures to be communicated by each finger. These coded messages are wirelessly transmitted using a Bluetooth transmitter-receiver pair, where the latter is connected to another Arduino UNO module, responsible for correct decoding. Once decoded, the message is displayed on an LCD, and subsequently vocalized by means of text-to-speech converter.

The proposed system has a potential to be shaped into a telephone set for persons with speech impairments, where hand-gestures will be converted into text and voice, and the voice, on return, will be converted into visual gestures, displayed on a screen in real-time. This will require natural language processing, which is something we plan to do in future.

Project Objectives

The primary objective of the proposed system is to assist persons with speech disability and impairments, in conveying their messages to the people around them. The specific phase-wise milestones of the project are as follows:

1. Designing a wearable flex-glove
2. Programming the coding of varying resistances of flex-sensor
3. Wirelessly transmitting the encoded message using Bluetooth module
4. Convert the decoded message into text and speech  

Project Implementation Method

The implementation of the project involves software and hardware components, and is based on a suitable microcontroller. For the first objective, flex sensors will be stitched on a glove, whose resistance, for each bend, will be digitized and conveyed to a microcontroller. The latter, each at the transmitter and the receiver end, will be responsible for encoding and decoding the message behind each bend of the finger respectively, where the transmitter and receiver pair will communicate using a Bluetooth module. Once correctly decoded by the receiver, the message will be displayed on a screen and later vocalized for conveying the message.

Benefits of the Project

Societal contributions

1. An autonomous system capable of giving voice to mute people, thereby serving a neglected segment of our society.
2. Requires minimal human intervention for communication

Literary contributions

1. The algorithm for encoding and decoding has a potential to be published.
2. In future, an updated version of the same will incorporate natural language processing, which will be a significant contribution for the scientific and research community.

Commercial benefits

1. The system to be developed, if successfully transformed into a product, will be submitted for patent. We are confident that a full-duplex system will have a potential to be commercialized.

Technical Details of Final Deliverable

1. Easily scalable and modifiable for the inclusion of new gestures
2. Off-the-shelf components make debugging and replacing of faulty components convenient and cheap.
3. Communication range, in the first prototype will be ten meters, which may be increased by means of Wi-Fi and/or GSM connectivity.
4. 45-125K ohm resistance of flex sensors
5. 5 volt and 40 Amp Arduino UNO will make the system energy efficienct.

Final Deliverable of the Project

Hardware System

Type of Industry

Others , Health

Technologies

Robotics

Sustainable Development Goals

Industry, Innovation and Infrastructure

Required Resources

If you need this project, please contact me on contact@adikhanofficial.com