Hand Gesture and Recognition system for mute people

Communications between deaf-mute and a normal person have always been a challenging task. Muteness creates a speech affliction where in the patient is unable to have the normal capacity to speak resulting in the complete absence or at least a significant loss of verbal communic

Project Title

Hand Gesture and Recognition system for mute people

Project Area of Specialization

Artificial Intelligence

Project Summary

Communications between deaf-mute and a normal person have always been a challenging task. Muteness creates a speech affliction where in the patient is unable to have the normal capacity to speak resulting in the complete absence or at least a significant loss of verbal communication under neurological diseases. Sometimes major cause of muteness is deafness. Around 466 million people worldwide have disabling hearing loss, and 34 million of these are children. The communication between a deaf and hearing person poses to be a serious problem compared to communication between blind and normal visual people.This creates a very little room for them with communication being a fundamental aspect of human life. Sign language is a non-verbal form of intercourse which is found amongst deaf communities in world. The sign languages do not have a common origin and hence difficult to interpret. Deaf-Mute communication interpreter is a device that translates the hand gestures to auditory speech. The project aims to facilitate people by means of a deaf-mute communication interpreter system. A gesture in a sign language is a particular movement of the hands to express different words and tasks. The proposed model is expected to translate alphabets into meaningful words made from hand gestures. The Training mode will also be offered in the device to fit every user need.The device will be capable of translating large gestures that require single hand movement.

Project Objectives

During the first phase of our research, we conducted a survey at Bahawalpur School of Special children. We interacted with the principal and staff of the school and
discussed their teaching methodologies which helped us in understanding the mindset and special requirements of the disabled students and the problems faced by them in
communicating with rest of the world. We also got the opportunity to sit in the classrooms and observe the teaching sessions. We attended different classes starting from kindergarten to Grade 5. We observed the changes in the students and the major problems that were faced by the teachers in teaching the students with special needs.
We felt that to meet the needs of these underprivileged people in Pakistani society, it is essential to develop an automated model that can effectively and efficiently translate Pakistani sign language used by speech impaired persons to overcome the communication barrier faced by them. The existing application available on the android market does not address this problem. They are limited to the English language and not all people in Pakistan can understand and talk in English, especially in rural areas.

Project Implementation Method

The project uses the methodology of a Kinect based assistive device that can be used as a sensor device to detect motion and gestures. The workflow requires implementing image processing technique in an open CV environment to detect the signs. The design is motivated by preliminary studies with mute people, and it is small-scale and mobile, which enables a more manageable operation with little setup.
Hardware & Software used:
1. Kinect Sensor Device
2. Kinect Adopter
3. Arduino
4. Portable speaker for voice output
5. Cables for connection
6. Python for the interference of Sensors, in Open Cv.
Steps Involved:
1. The first step is to install the SDK file of Kinect in your Laptop.
2. The second step will be skeleton detection by the Kinect sensor.
3. The Third Step is to install Anaconda3-2018.12-Windows-x86_64.
4. After that detect the upper skeleton with Kinect sensor device and take live gestures at live time and take output in form of voice.
5. And then the audio output is taken.

Benefits of the Project

The project will be highly benefical fpr the mute people because it will be providing real time support by converting their signs to the Voice Signals. 

The project will be providing solution to the millions of people which shows its scalibilty that means it can be adopted in any part of the world.

Technical Details of Final Deliverable

HAND gesture recognition is of great importance for human-computer interaction (HCI), because of its extensive applications in virtual reality, sign language recognition, and computer games. In this project we will build a prototype of hand gesture recognition system for detection of human gestures, which will help mute peolple in their communication. Upon finalization of the project we will provide

a) Prototype of Hand Gesture recognition system for mute people using Kinect based on hardware and software

b) Conference paper

c) Final Report

Final Deliverable of the Project

HW/SW integrated system

Type of Industry

Telecommunication

Technologies

Artificial Intelligence(AI)

Sustainable Development Goals

Good Health and Well-Being for People

Required Resources

If you need this project, please contact me on contact@adikhanofficial.com