Automated communication device for deaf and dumb

Communication is a basic human need, and language is its source. Most people have the ability to listen and speak and they use different languages to communicate. But, whenever deaf and dumb people need to communicate with non-disabled people in any situation, they need to hire an interpreter or ask

Project Title

Automated communication device for deaf and dumb

Project Area of Specialization

Information & Communication Technology

Project Summary

Communication is a basic human need, and language is its source. Most people have the ability to listen and speak and they use different languages to communicate. But, whenever deaf and dumb people need to communicate with non-disabled people in any situation, they need to hire an interpreter or ask people with disabilities to communicate in writing. Current translators are very difficult to find at all times and expensive as well. The interaction of deaf people and ordinary people becomes slow and embarrassing. At the same time, deaf and dumb people find it difficult to do what they want in shopping malls, banks and post offices and so on. In addition, deaf and dumb people in Pakistan will soon end up with friends who are also deaf and dumb. There is definitely a need for an automated system that can facilitate communication between the deaf/dumb and the disabled.

In this project, we try to develop a system to converts gestures into voice and voice into gestures. The converted voice is in National language(URDU). Main objective is to facilitate a large population of hearing-impaired persons and making them an integral part of the society.

The proposed device is based on the human skeletal key points that are estimated by third part library OpenCV. The OpenCV is an open source toolkit for real-time multi-person key point detection. The system takes images of user making particular sign through webcam and uses skeletal tracking combined with machine learning library(Tenser flow)to detect what sign they are making. Python is used as programming language , there is no device or application in Pakistan for this purpose so this, will be the first project that will work on that It will also allow users to learn PSL interactively.

This project can serve both speech-disordered people and the others. Yet it is designed especially for the speech-disordered people’s convenient use.

Project Objectives

The aim of the project is to bridge the communication gap between signers and non-signers by developing an interpretation system which is cost effective and easy to use. The system is expected to have the ability to converts sign into speech. In this project, we try to build a hardware device that converts gestures into voice and voice into gestures. The converted voice is in National language(URDU).

The main objectives of this system are:

• Using OpenCV, we detect the gesture of deaf and dumb people.
• Using machine learning, we convert the gestures into voice and voice into gestures.
• Configure Raspberry Pi for machine learning technique.
• Propose communication device for deaf and dumb people.

Project Implementation Method

The methodology is divided into two modules:

1. Gesture to speech:

•Acquire images using the camera of the device.

•Perform vision-based algorithms and provides speech output through the audio device(headphones).

2. Speech to gesture:

•Acquire speech output using the microphone of the device.

•Perform speech analysis algorithms and provides gesture output thorough display device.

Benefits of the Project

Benefits of our project are:

1. Deaf and dumb people use sign language as their primary means to express their thoughts and ideas to the people around them with different hand and body gestures.

2. Keep conversation ordered and coherent between deaf and dumb People.

3. This device is portable as wel as use for any organization such as bank..to make easy communication with deaf and dumb people.

4. Provide an interesting application that guarantees ultimate communication with the disabled users.

5. Build the confidence in these people.

6. These people can interact face to face with normal people.

Technical Details of Final Deliverable

The tools that we used in our project:

1. Hardware Based:

• Raspberry pi
• Raspberry pi camera
• Raspberry pi adapter
• Headphones
• Microphones
• Glass
• Connected Wires
• Display Device

2. Software Based:

• OpenCV
• Tenserflow lite
• Python

The final delivery will be a device that used to converts speech into gesture and vice versa.The device is placed between the daef-dumb and normal people during communication. The deaf-dumb People performs gestures and device  converts that  into speech and normal people speek and that speech converts into gesture.

Final Deliverable of the Project

Hardware System

Core Industry

Telecommunication

Other Industries

Core Technology

Others

Other Technologies

Sustainable Development Goals

Decent Work and Economic Growth, Reduced Inequality, Peace and Justice Strong Institutions, Partnerships to achieve the Goal

Required Resources

If you need this project, please contact me on contact@adikhanofficial.com