An Interactive System for Pakistan Sign Language for Recognition for Deaf

Sign Language Recognition is a breakthrough for helping deaf-mute people and has been researched for many years. Unfortunately, every research has its own limitations and are still unable to be used commercially. Some of the researches have known to be successful for recognizing sign language, but r

Project Title

An Interactive System for Pakistan Sign Language for Recognition for Deaf

Project Area of Specialization

Artificial Intelligence

Project Summary

Sign Language Recognition is a breakthrough for helping deaf-mute people and has been researched for many years. Unfortunately, every research has its own limitations and are still unable to be used commercially. Some of the researches have known to be successful for recognizing sign language, but require an expensive cost to be commercialized. Nowadays, researchers have gotten more attention for developing Sign Language Recognition that can be used commercially. Researchers do their researches in various ways and cultures. As we know that sign language is different for each culture and country. According to our knowledge and research, nobody has researched on Pakistan Sign Language (PSL). Sign Language recognition starts from the data acquisition methods. The data acquisition method varies because of the cost needed for a good device, but cheap method is needed for the Sign Language Recognition System to be commercialized. The methods used in developing Sign Language Recognition are also varied between researchers. Each method has its own strength compare to other methods and researchers are still using different methods in developing their own Sign Language Recognition. Each method also has its own limitations compared to other methods. The aim of this project is to recognize actions of the subject through simple camera. Hence other researchers can get more information about the methods used and could develop better Sign Language Application Systems in the for other culture in future

Project Objectives

To proposes an interactive system for PSL recognition based on the deep learning through videos.

To develop a public dataset for researchers and students to further research on PSL

Development of a holistic system for PSL recognition for deaf. 

To make deaf people capable to communicate globally. 

To facilitate the deaf with understanding their points.

Project Implementation Method

In order to do that we are using few key models so we are using media pipe holistic to be able to extract keypoints so this allow us to extract key points from our hands, body and our face. Furthermore, we are using tensorflow and keras and build up a lstm model to be able to predict the action or a sign language’s sign and then we are going to put it all together. We take media pipe holistic, trained lstm model and actually go on ahead and predict signs in real time.

There are some keypoints that shows how it works:

• Collect a bunch of data on all of our different keypoints so we are going to be collecting the data on our hands, body and face and save all the data as numpy arrays
• Train a deep neural network using lstm layers to go on ahead and predict that temporal component so that we will be able to predict the action from multiple set of frames.
• Stick it all together using opencv and predict in real time using a webcam.

Benefits of the Project

• To proposes an interactive system for PSL recognition based on the deep learning through videos.
• To develop a public dataset for researchers and students to further research on PSL 
• Development of a holistic system for PSL recognition for deaf.
• To make deaf people capable to communicate globally.
• To facilitate the deaf with understanding their points.

Technical Details of Final Deliverable

Phase

1

2

3

Final Deliverable of the Project

HW/SW integrated system

Core Industry

IT

Other Industries

Core Technology

Artificial Intelligence(AI)

Other Technologies

Sustainable Development Goals

Good Health and Well-Being for People, Industry, Innovation and Infrastructure, Reduced Inequality

Required Resources

If you need this project, please contact me on contact@adikhanofficial.com