Isharay
A mobile application which helps deaf people communicate with other people by working as a translator from sign language to text and vice versa using mobile camera and 3d modelling.
Project Title
Isharay
Project Area of Specialization
Project Summary
A mobile application which helps deaf people communicate with other people by working as a translator from sign language to text and vice versa using mobile camera and 3d modelling.
Project Objectives
- Converting sign language to text.
- Converting text to sign language.
Project Implementation Method
PHASE1:
Conversion of Sign language to text.
DATA COLLECTION:
Collection of video data of words/phrases in sign language from different websites.
DATA CLEANING:
Organizing those videos into a single usable format with unified frames per seconds (fps) for our model.
KEYPOINT DETECTION:
Detection of hand motion and collection of 21 keypoints of each hand in a single frame.
DATA TRANSLATION:
Converting the above mentioned keypoints into vectors for our machine learning model.
MODEL TRAINING:
Training of model on these vectors to classify videos against words/phrases of sign language.
PHASE2:
Conversion of text to sign language.
3D MODEL:
Making of a 3D Human Model to demonstrate sign language words/phrases.
ANIMATION:
Animating the model in sign language gestures against the words/phrases (text) from user.
PUBLIC PORTAL:
Allowing users to add new words in the vocabulory considering the regional differences in sign language.
Benefits of the Project
- Diminishing the barrier between normal and deaf people .
- Making the lives of hearing-impaired more independent.
- Creating job opportunities for deaf people.
- Making the learning of sign language easy for deaf people.
Technical Details of Final Deliverable
A mobile application that will convert text to sign language and sign language to text using mobile device.
The keypoint detection is being done by using a TensorFlow model in C++
After that these keypoints are converted to a vector which will be used to train a model in python using a neural network.
This model will be used in the mobile device or cloud to convert the sign language video to text
The 3D model will be used to convert text to sign language gestures by animation of the model.
There will also be an option to add more sign language words/phrases by user to overcome the regional differences of sign language.
Final Deliverable of the Project
Core Industry
Other Industries
Core Technology
Other Technologies
Sustainable Development Goals
Required Resources
| Item Name | Type | No. of Units | Per Unit Cost (in Rs) | Total (in Rs) |
|---|---|---|---|---|
| External Docker for Graphic Card EXP GDC Beast External Graphics Card Dock | Equipment | 1 | 20000 | 20000 |
| Nvidia GTX 1070 | Equipment | 1 | 50000 | 50000 |
| Total in (Rs) | 70000 |
If you need this project, please contact me on contact@adikhanofficial.com
AUST CS COMMUNITY
The project is an online knowledge sharing and resource center; it is an online system tha...
COAL Mine Investigator
Each year many coal mine workers die because of incidents in coal mines, majority of incid...
design and fabrication of evasive spiderbot
We are going to design and fabricate evasive spiderbot which will have the feature of SLAM...
effect of rainfall intensity on slope stability of Swat region
Landslide is one of the most common Hazard on Earth, although affected area is smaller but...
IOT based arduino smart parking system with covid prevention property
Smart car parking project aims at providing a confusion free and easy parking. Efficient a...