Isharay

Project Title

Isharay

Project Area of Specialization Artificial IntelligenceProject Summary

A mobile application which helps deaf people communicate with other people by working as a translator from sign language to text and vice versa using mobile camera and 3d modelling. 

Project Objectives

• Converting sign language to text.
• Converting text to sign language.

Project Implementation Method PHASE1:

      Conversion of Sign language to text.

DATA COLLECTION:

      Collection of video data of words/phrases in sign language  from different websites.

DATA CLEANING:

      Organizing those videos into a single usable format with unified frames per seconds (fps) for our model.

KEYPOINT DETECTION:

      Detection of hand motion and collection of 21 keypoints of  each hand in a single frame.

DATA TRANSLATION:

       Converting the above mentioned keypoints into vectors for our machine learning model.

MODEL TRAINING:

       Training of model on these vectors to classify videos against words/phrases of sign language.

PHASE2:

        Conversion of text to sign language.

3D MODEL:

        Making of a 3D Human Model to demonstrate sign language words/phrases.        

ANIMATION:

       Animating the model in sign language gestures against the words/phrases (text) from user.

PUBLIC PORTAL:

      Allowing users to add new words in the vocabulory considering the regional differences in sign language.

Benefits of the Project

• Diminishing the barrier between normal and deaf people .
• Making the lives of hearing-impaired more independent.
• Creating job opportunities for deaf people.
• Making the learning of sign language easy for deaf people.

Technical Details of Final Deliverable

A mobile application that will convert text to sign language and sign language to text using mobile device.

The keypoint detection is being done by using a TensorFlow model in C++

After that these keypoints are converted to a vector which will be  used to train a model in python using a neural network.

This model will be used in the mobile device or cloud to convert the sign language video to text

The 3D model will be used to convert text to sign language gestures by animation of the model.

There will also be an option to add more sign language words/phrases by user to overcome the regional differences of sign language. 

Final Deliverable of the Project Software SystemCore Industry EducationOther Industries IT , Others , Telecommunication Core Technology Artificial Intelligence(AI)Other Technologies Artificial Intelligence(AI)Sustainable Development Goals Good Health and Well-Being for People, Quality Education, Reduced InequalityRequired Resources

Item Name	Type	No. of Units	Per Unit Cost (in Rs)	Total (in Rs)
			Total in (Rs)	70000
External Docker for Graphic Card EXP GDC Beast External Graphics Card Dock	Equipment	1	20000	20000
Nvidia GTX 1070	Equipment	1	50000	50000