An Interactive System for Pakistan Sign Language for Recognition for Deaf
Sign Language Recognition is a breakthrough for helping deaf-mute people and has been researched for many years. Unfortunately, every research has its own limitations and are still unable to be used commercially. Some of the researches have known to be successful for recognizing sign language, but r
2025-06-28 16:25:06 - Adil Khan
An Interactive System for Pakistan Sign Language for Recognition for Deaf
Project Area of Specialization Artificial IntelligenceProject SummarySign Language Recognition is a breakthrough for helping deaf-mute people and has been researched for many years. Unfortunately, every research has its own limitations and are still unable to be used commercially. Some of the researches have known to be successful for recognizing sign language, but require an expensive cost to be commercialized. Nowadays, researchers have gotten more attention for developing Sign Language Recognition that can be used commercially. Researchers do their researches in various ways and cultures. As we know that sign language is different for each culture and country. According to our knowledge and research, nobody has researched on Pakistan Sign Language (PSL). Sign Language recognition starts from the data acquisition methods. The data acquisition method varies because of the cost needed for a good device, but cheap method is needed for the Sign Language Recognition System to be commercialized. The methods used in developing Sign Language Recognition are also varied between researchers. Each method has its own strength compare to other methods and researchers are still using different methods in developing their own Sign Language Recognition. Each method also has its own limitations compared to other methods. The aim of this project is to recognize actions of the subject through simple camera. Hence other researchers can get more information about the methods used and could develop better Sign Language Application Systems in the for other culture in future
Project ObjectivesTo proposes an interactive system for PSL recognition based on the deep learning through videos.
To develop a public dataset for researchers and students to further research on PSL
Development of a holistic system for PSL recognition for deaf.
To make deaf people capable to communicate globally.
To facilitate the deaf with understanding their points.
Project Implementation MethodIn order to do that we are using few key models so we are using media pipe holistic to be able to extract keypoints so this allow us to extract key points from our hands, body and our face. Furthermore, we are using tensorflow and keras and build up a lstm model to be able to predict the action or a sign language’s sign and then we are going to put it all together. We take media pipe holistic, trained lstm model and actually go on ahead and predict signs in real time.
There are some keypoints that shows how it works:
- Collect a bunch of data on all of our different keypoints so we are going to be collecting the data on our hands, body and face and save all the data as numpy arrays
- Train a deep neural network using lstm layers to go on ahead and predict that temporal component so that we will be able to predict the action from multiple set of frames.
- Stick it all together using opencv and predict in real time using a webcam.
- To proposes an interactive system for PSL recognition based on the deep learning through videos.
- To develop a public dataset for researchers and students to further research on PSL
- Development of a holistic system for PSL recognition for deaf.
- To make deaf people capable to communicate globally.
- To facilitate the deaf with understanding their points.
| Phase | Requirement | Resource | Timeframe | Estimated | |||||
| time | of | ||||||||
| completion | |||||||||
| 1 | Information | Requirement | One week | June 21 | |||||
| Gathering | Engineer | ||||||||
| 2 | Planning | Project | Two week | July 4 | |||||
| Manager | |||||||||
| 3 | Designing | Product | Two week | July 18 | |||||
Phase
1
2
3
Final Deliverable of the Project HW/SW integrated systemCore Industry ITOther IndustriesCore Technology Artificial Intelligence(AI)Other TechnologiesSustainable Development Goals Good Health and Well-Being for People, Industry, Innovation and Infrastructure, Reduced InequalityRequired Resources| Elapsed time in (days or weeks or month or quarter) since start of the project | Milestone | Deliverable |
|---|---|---|
| Month 1 | Information Gathering | No |
| Month 2 | Planning | No |
| Month 3 | Designing | No |
| Month 4 | ContentWritingAssembly | No |
| Month 5 | Coding | yes |
| Month 6 | Testing,Reviewand Launch | yes |