Isharay
A mobile application which helps deaf people communicate with other people by working as a translator from sign language to text and vice versa using mobile camera and 3d modelling.
Project Title
Isharay
Project Area of Specialization
Project Summary
A mobile application which helps deaf people communicate with other people by working as a translator from sign language to text and vice versa using mobile camera and 3d modelling.
Project Objectives
- Converting sign language to text.
- Converting text to sign language.
Project Implementation Method
PHASE1:
Conversion of Sign language to text.
DATA COLLECTION:
Collection of video data of words/phrases in sign language from different websites.
DATA CLEANING:
Organizing those videos into a single usable format with unified frames per seconds (fps) for our model.
KEYPOINT DETECTION:
Detection of hand motion and collection of 21 keypoints of each hand in a single frame.
DATA TRANSLATION:
Converting the above mentioned keypoints into vectors for our machine learning model.
MODEL TRAINING:
Training of model on these vectors to classify videos against words/phrases of sign language.
PHASE2:
Conversion of text to sign language.
3D MODEL:
Making of a 3D Human Model to demonstrate sign language words/phrases.
ANIMATION:
Animating the model in sign language gestures against the words/phrases (text) from user.
PUBLIC PORTAL:
Allowing users to add new words in the vocabulory considering the regional differences in sign language.
Benefits of the Project
- Diminishing the barrier between normal and deaf people .
- Making the lives of hearing-impaired more independent.
- Creating job opportunities for deaf people.
- Making the learning of sign language easy for deaf people.
Technical Details of Final Deliverable
A mobile application that will convert text to sign language and sign language to text using mobile device.
The keypoint detection is being done by using a TensorFlow model in C++
After that these keypoints are converted to a vector which will be used to train a model in python using a neural network.
This model will be used in the mobile device or cloud to convert the sign language video to text
The 3D model will be used to convert text to sign language gestures by animation of the model.
There will also be an option to add more sign language words/phrases by user to overcome the regional differences of sign language.
Final Deliverable of the Project
Core Industry
Other Industries
Core Technology
Other Technologies
Sustainable Development Goals
Required Resources
| Item Name | Type | No. of Units | Per Unit Cost (in Rs) | Total (in Rs) |
|---|---|---|---|---|
| External Docker for Graphic Card EXP GDC Beast External Graphics Card Dock | Equipment | 1 | 20000 | 20000 |
| Nvidia GTX 1070 | Equipment | 1 | 50000 | 50000 |
| Total in (Rs) | 70000 |
If you need this project, please contact me on contact@adikhanofficial.com
Walletic
WALLETIC is an Electronic Money Institution (EMI), The EMIs in Pakistan are expected to of...
Leap motion control robot and robotic arm
With technology advancing at an astronomical rate, we are witnessing events today that a f...
RFID based library management system
RFID-based frameworks go far beyond security to become monitoring systems that integrate s...
Solar Car with Intelligent reverse braking system
Title : SOLAR CAR WITH INTELLIGENT BRAKING SYSTEM Project is composed of two basic fields...
Design and Fabrication of Hydro Vortex Turbine
A Gravitation vortex turbine is a type of hydraulic turbine which converts the potential e...