Smart glove Hand Gesture Vocalizer for deaf and dumb people

Communication plays a vital role for human beings. In the present time, we don?t have suitable source for deaf and dumb people that help them to convey their message. The people who cannot speak makes use of the sign languages to communicate with other fellow who knows the meanings of sign languages

Project Title

Smart glove Hand Gesture Vocalizer for deaf and dumb people

Project Area of Specialization

Electrical/Electronic Engineering

Project Summary

Communication plays a vital role for human beings. In the present time, we don’t have suitable source for deaf and dumb people that help them to convey their message. The people who cannot speak makes use of the sign languages to communicate with other fellow who knows the meanings of sign languages or an interpreter is needed to translate the meanings of sign languages. However, hiring interpreters is not easy. Another alternative approach that is easy and not expensive is that we use technology that is to minimize the communication gap between the normal & impaired people.

This work is addressing the three UN SDGs: first one is “Quality Education” which “ensure inclusive and equitable quality education and promote lifelong opportunities for all”. Hence, the smart hand should provide the quality education to special students that meet the equal opportunity strategy between normal and special people. While the second one is “Innovation” means “build innovative techniques and promote inclusive, sustainable and foster innovation”. Hence, the project should bring the innovation in training centers and the institutes related to deaf and dumb. And the last is “Reduce Inequalities” which defines “reduced inequality among the local community and creating equal opportunity for all to enhance their life according to their own”. Hence, this project should lease the inequality between both the normal and special people to reduce the communication gap.

Project Objectives

The main objective of this project is that it can be used in hospitals that need several measurement systems which can investigate physiological parameters of the patients. It can act as a communication aid for the hearing- and speech-impaired persons. Using this we can control any electronic machines and devices wirelessly by adding some others gestures and commands. This project can be also used as Signed Language Convertor/Interpreter. For gaming purposes like 3D gaming, at the place of joysticks. It can be used in military actions based on hand gestures, which can be used for squad communication. The hand gesture recognition system can be used in robotics, desktop and tablet PC applications and gaming. The Handtalk is also can be used for HOME AUTOMATION.

Project Implementation Method

Step 1: Start

Step 2: Input: For a particular gesture, 8 sensor data inputs (5 fl ex sensors data and 3 accelerometer data)

 Step 3: Convert raw sensor data to meaningful data

Step 4: Set the baud rate

Step 5: Send data to the serial port

Step 6: Then data is collected through ESP 32 DEV-KIT module.

Step 7: Go to step B

Step B

Step 1: Start

Step 2: Train the model using the training dataset.

Step 3: Set the baud rate similar to Arduino serial baud rate and open the serial connection.

Step 4: Is there data at serial port? If no, wait till data available then continue.

Else, go to step 5

Step 5: Input: Collect data from the serial port at real time.

Step 6 Write the collected data the new file in CSV format.

Step 7:Close the Serial port connection.

Step 8: Pass the temporary dataset through the model.

Step 9: Trained machine predicts the output.

Step 10: Display the prediction on the screen and play the audio of the predicted word.

Step 11: Add 1-second delay

Step 12: End

 This is all our Project Implementation Method.

Benefits of the Project

The outcome of this work was estimated where all the Pakistan standard sign language with their respective alphabets and some words were successfully displayed in to text and voice which can help speech impaired and hearing impaired people to communicate with normal people easily.

This proposed system will support 4 th of SGDs (Quality Education) to provide quality education to special students that provide equal opportunity to get quality education as that of other normal students. The development of our system will bring the innovation in training centers, learning institutions for deaf and dumb people and thus help to meet the 9 th of SGDs (Innovation). Further it will reduce inequality between the normal and special people and help them to convey their message, this will help to meet the 10th of SGDs (Reduced Inequalities)

Social Impact:

It is a project which aims to bridge the gap between deaf and dumb community and normal masses.

This project converts sign language to text and speech. It is also a cost effective.it is Useful for physically challenged people.

This technology makes it possible to obtain accuracy for converting sign to speech language. This makes the user feel like a normal person just with the help of smart glove.

This system will help special people in improving their quality of life significantly.

Technical Details of Final Deliverable

The project proposes a system for speech impaired and hearing impaired people using glove technology and enable normal people to communicate with them too. The use of five flex sensor, touch sensors and an accelerometer on to a glove demonstrate that it is helpful to beak the gap between speech impaired hearing impaired and normal people. This device will be an apt tool not only for the people got such disability naturally rather it also helps disability due to oral diseases and accidental cause make them to learn gesture through application easily. The project can be enhanced further by including more words and different standard sign language.

Smart gloves which take input from the users’ gestures and gives a textual output. It helps the mute people to communicate with one another and with the normal people. The mute people use their standard sign language to communicate which is not easily understandable by common people without the knowledge of the sign language. Thus, this system converts the sign language into audio, pictorial and into text form, improving their life in a signifi cant way Image processing can be applied to take in new gestures from the user themselves and store it in the dataset. Make proper, well fi nished and a wireless wearable technology to bridge the communication gap with accurate and precise result.

Final Deliverable of the Project

HW/SW integrated system

Core Industry

Education

Other Industries

Medical

Core Technology

Robotics

Other Technologies

Wearables and Implantables

Sustainable Development Goals

Quality Education, Gender Equality, Industry, Innovation and Infrastructure

Required Resources

If you need this project, please contact me on contact@adikhanofficial.com