SMART BARRIER FREE HAND GESTURE COMMUNICATION SYSTEM FOR HEARING AND SPEECH IMPAIRED

Hearing and speech impaired people can communicate with each other by sign language. But it can be difficult for common people because not all common people understand sign language. Disable people indulged in a problem and cannot communicate with common people without sign language. To

Project Title

SMART BARRIER FREE HAND GESTURE COMMUNICATION SYSTEM FOR HEARING AND SPEECH IMPAIRED

Project Area of Specialization

Electrical/Electronic Engineering

Project Summary

Hearing and speech impaired people can communicate with each other by sign language. But it can be difficult for common people because not all common people understand sign language. Disable people indulged in a problem and cannot communicate with common people without sign language. To cover this issue translators are used but that has very much cost and only the person could carry who know the sign language which is also a problem. In that case this smart glove project can resolve all the problems for hearing and speech impaired people. It is the sign language based communication system, that can convert into text and speech. As name clearly defined this is a kind of system that gives voice to voice less. especially speech impaired people sharing  feelings with others. This system can be customized by the individual according to his needs as the messages are derived from a data based developed after predictive analysis. There are two types of hand gesture dynamic and static. Dynamic gesture consists of sequence of movement of the hand such as waving. While static hand gestures imply to the stability of hand shape. In this static gesutres are used. hence, by using hand gestures disable people communicate with others in the form of text (shown in mobile app) and speech (in terms of voice).Hearing and speech impaired people can communicate with each other by sign language. But it can be difficult for common people because not all common people understand sign language. Disable people indulged in a problem and cannot communicate with common people without sign language. To cover this issue translators are used but that has very much cost and only the person could carry who know the sign language which is also a problem. In that case this smart glove project can resolve all the problems for hearing and speech impaired people. It is the sign language based communication system, that can convert into text and speech. As name clearly defined this is a kind of system that gives voice to voice less. especially speech impaired people sharing  feelings with others. This system can be customized by the individual according to his needs as the messages are derived from a data based developed after predictive analysis. There are two types of hand gesture dynamic and static. Dynamic gesture consists of sequence of movement of the hand such as waving. While static hand gestures imply to the stability of hand shape. In this static gesutres are used. hence, by using hand gestures disable people communicate with others in the form of text (shown in mobile app) and speech (in terms of voice).

Project Objectives

The goal of this project is to develop a gestural system by which we can easily understood with hearing and speech impaired peoples. Converting sign language into text and speech. Hearing and speech impaired people able to convey their needs, feelings and emotions. It is lower power consumption, portable and efficient tool easily use by hearing and speech impaired masses. After this there is no restriction for hearing and speech impaired to live in a small social circle and no need of a third person to translate between disable (hearing and speech impaired peoples). Light weight and easy gripping for hand. Also it would be helpful in educational institutes.

Project Implementation Method

We are using ESP-32 on both sides for 2- way communication (means for transmitting and receiving purpose). We are also using its hotspot to access the internet. On the other hand we are using IR sensors for fingers movement in our glove which we are making and what module we are making on the sender side in which IR sensor will be fixed and the IR sensor works on the principle of zeros and ones. Basically there are two types of IR sensors, the first one is analog and the other one is digital. We are using digital because we want to work on zeros and ones. When we touch our finger on IR sensor, it will work as a button and one will go to receive side from transmitter side. When one will transmit it will automatically on the speaker and as you release your finger, it will become zero and it automatically off the speaker. We are using IR sensor here due to accuracy. As we touch it will show 1 and on release it will show 1 and on release it will show 0. We are using MPU-6050 that will give X-axis and Y-axis instructions and also we are using DF mini player which is a MP3 module player that will provide simplified output directly to the speaker. So for this our main implementation is our APP in which we can show any language in a text format. An APP provides a facility for receiver to view what the people need.

Benefits of the Project

• By this gestural based communication system, messages easily convey recognizably to the hearing and speech impaired. It saves from danger and hazards and from environment situations.
• Mobility and comfortably.
• Self education tool.
• Easy to acquire data.
• Enhanced the accuracy and reliability of recognition system.
• Flexible to users.
• Easy to operate.
• Real time translation

Technical Details of Final Deliverable

In our project we are using below components to make this idea happens.

• ESP-32

We are using this ESP-32 for 2-way communication and for internet access through its hotspot. It can perform as a complete standalone system or a slave device to host MCU, reducing communication stack overhead on the main application processor and it has a Wi-Fi and Bluetooth functionality onboard.

• NRF 24L01

This is wireless transceiver module which works on SPI protocol. By using this in our project we can sending and receiving data at an operating radio frequency of 2.4 to 2.5ghz ISM band and it connect with Esp-32 through Wi-Fi Gateway.

• Speaker

For listening the messages, help or simply what disable people needs.

• IR sensor

We are using IR senor for fingers movement. Basically 2 types of IR sensor Analog & Digital, we use Digital because we want to work with zeros and ones and digital IR sensor working on zeros and ones, it will serve as a button to finger for transmitting and receiving data automatically to the speaker.

• MPU 6050

It is a micro electromechanical system consists of three-axis accelerometer and three-axis gyroscope. It helps us to measure velocity, orientation, acceleration, displacement and other motion like features.

• DF mini Player

DF mini player is simply a MP3 module player that will provide simplified output directly to the speaker.

Final Deliverable of the Project

HW/SW integrated system

Core Industry

IT

Other Industries

Health

Core Technology

Internet of Things (IoT)

Other Technologies

Others

Sustainable Development Goals

Decent Work and Economic Growth

Required Resources

If you need this project, please contact me on contact@adikhanofficial.com