Realization and Implementation of Communication for hearing impaired and inarticulate people

Natural language processing is one of the most growing fields of research. It deals with human-computer interaction and natural languages. Humans share ideas and thoughts with people around them using speech and hearing abilities. But this is not the case for hearing impaired and inarticulate people

Project Title

Realization and Implementation of Communication for hearing impaired and inarticulate people

Project Area of Specialization

Computer Science

Project Summary

Natural language processing is one of the most growing fields of research. It deals with human-computer interaction and natural languages. Humans share ideas and thoughts with people around them using speech and hearing abilities. But this is not the case for hearing impaired and inarticulate people. Through sign recognition, communication is possible with hearing impaired and inarticulate people. This project aims to develop a mobile application-based system for recognizing sign language and converting it into text and speech, which provides smooth communication between the deaf-mute community and normal people, thereby reducing the communication barrier between them. Our proposed system will consist of a mobile application that would be used for communication. The project will mainly consist of two parts, i.e., sign-to-speech conversion and mobile application development. In sign-to-speech conversion, the sign will be recognized using a mobile camera with the help of computer vision, image processing and machine learning algorithms. The recognized sign will be converted into text, and text will be used to generate speech. The whole system will be integrated with easy to use mobile application. With the incorporation of the android mobile application, this system will ease people with disabilities to communicate with non-disabled people and reduce the communication barrier between them.

Project Objectives

Natural Language Processing (NLP) is an emerging field that helps in conveying information and meaning with semantic cues such as words, signs, or images. Sign Language is a very convenient way for hearing impaired and inarticulate people. But for other world, communication with hearing impaired and inarticulate people is difficult because a normal person cannot understand their sign language in normal circumstances. This difference in society can be minimized if we can program a machine in such a way that it translates sign language into text or speech. According to a survey, there are more than 10 million people in Pakistan who cannot express their thoughts and feelings in speech. Therefore, we proposed a project that utilizes computer vision and image processing techniques to develop a system that can convert sign language into textual and audio form with a mobile application interface, thereby, reducing the communication barrier between specially-abled and non-disabled people. This project aims to meet UN’s SDGs 8.5, 10.2 and 10.3. It aims to reduce inequalities and communication barriers in society. The second purpose of this project is to increase the productive employment of all persons including persons with disabilities.

Project Implementation Method

For the implementation of our project, we have to collect the video dataset based on sign gestures (from specially-abled people) to train the machine learning model. Sign language depends on continuous gestures of hands movement and facial expressions rather than static images. So we have to collect data in the form of videos based on daily life activities. This video dataset will be fed to a machine learning model which will be used to classify the sign gestures. After successful results and satisfactory accuracy of the model, the whole system will be integrated with a mobile application for easy useability. The application will display the predicted text from sign gesture and generate speech as the final output.

Benefits of the Project

We are part of society and we should try to solve problems in our societies. Communication is a basic right of each person in society. But this is not the case for hearing-impaired and inarticulate people. There are more than 10 million people in Pakistan who cannot communicate or participate in many social, political and professional events due to their inability to speak and hear. There is a lack of accessibility for these people in nearly every field of life such as markets, job interviews, and day-to-day communication. This project utilized computer vision and image processing techniques to develop a system that can convert sign language into textual and audio form with a mobile application interface, thereby, reducing the communication barrier between specially-abled and non-disabled people in the society. It is a reliable alternate mode of communication that provides the following benefits:

• It does not need a voice and is beneficial for individuals suffering from hearing and speech disabilities to improve their quality of life.
• The system will translate sign gestures into words and sentences which are important for communication.
• This system does not require any hardware and is based on a mobile application that is easy to use and available anywhere. The user does not need to carry any additional equipment with him in order to communicate.

This system will bridge the communication barrier in society and will open doors to employment and opportunities for specially-able people. By bridging the communication gap, the life experience of the deaf-mute will be enhanced by allowing them to express themselves and being understood better.

Technical Details of Final Deliverable

We will deliver a software-based mobile application that will be easy to use and have a friendly user interface. This mobile application will provide facilities of:

1. Recording a sign gesture video of deaf-mute people.
2. Uploading existing sign gesture video.
3. Prediction of the sign in the form of Text and Speech.

Final Deliverable of the Project

Software System

Core Industry

IT

Other Industries

Others

Core Technology

Others

Other Technologies

Others

Sustainable Development Goals

Reduced Inequality

Required Resources

If you need this project, please contact me on contact@adikhanofficial.com