Sign Language Converter and Trainer

The Android Application ?Sign Language Converter & Trainer? mainly made to facilitate people who are in need to learn Sign Language or require Sign Language to be used in their daily life to be able to communicate with Deaf People and people with disabling hearing loss. The purpose of this proje

Project Title

Sign Language Converter and Trainer

Project Area of Specialization

Artificial Intelligence

Project Summary

The Android Application “Sign Language Converter & Trainer” mainly made to facilitate people who are in need to learn Sign Language or require Sign Language to be used in their daily life to be able to communicate with Deaf People and people with disabling hearing loss. The purpose of this project is to target Sign Language (ASL), help people learn Sign language, and spread awareness related to it so that whenever they meet someone who is deaf or have disabling hearing loss problem, they can easily communicate with them. Through this Mobile Application, the people would be able to convert their message in English to respective sign Language and learn how respective signs are made for each letter. The product’s main goal is to target ASL by providing convenient and expressive Sign Language Conversion and Training.
The main objective of the Application is to be a bugless, easy-to-use, convenient, accurate and reliable system to which people in need of Sign Language can rely on.
The mobile App. would be able to convert English language to Sign language of respective standard (ASL). It will take English sentence as an input through text or voice (Google Speech-to-Text API will be used for Speech Recognition) and will convert it into Sign language and show it in terms of images as well as it will also provide animated representation of each respective sign. The App. would also provide training of how to use sign language effectively by showing respective letters on screen and telling the user, how to express those letters in sign language. The user while taking the guide from the screen, will make respective signs for respective letters and the application will evaluate through camera (Image Detection and Classification) whether the signs made by the user are correct or wrong. In this way, one would be able to train oneself the art of using sign language. The App. would also provide an assessment feature through which one can assess whether he/ she can use sign language effectively. The App. will show random words or sentences on the screen and this time it will not provide any hint or guide and the user will have to express the words or sentences shown on screen in terms of sign language. Application would also contain a feature where one would be able to convert sign language into respective English Language.

Project Objectives

• Convenient conversion from English language to Sign Language (ASL) for interaction with Deaf community.
• Expression of sign language in images.
• Expression of sign language in animation (To get better idea of how to make signs).
• Convenient conversion from Signs to English Language through camera.
• Provide training of ASL Signs through Hand Detection and Image Classification
• Assessment model to evaluate one’s expertise of using sign language through Hand detection and image classification.
• Accuracy of image classification must be at least 80%.

Project Implementation Method

Project will be implemented by dividing the entire project in two time periods.

• FYP - I
• FYP - II

In each time period 3 phases will be followed:

• Design (SRS)
• Implement w/ Technical Document
• Testing

First of all an android application will be developed including screens of the app and then database will be constructed and connected with the app. Animations will be desgined on Blender for converter part. Converter will be made with images and animations and then for training and assessment part, a model will be trained using python, tensorflow and GPU. Then model will be integerated into the Applicaton and used for training, assessment and conversion from Sign to English Language.

Benefits of the Project

• Convenient and quick conversion from English (Text) to Sign Language (Images).
• Convenient and quick conversion from English (Speech) to Sign Language (Images).
• View 3D animations of converted Sign Language (letter by letter) with animation controller.
• Get training of each Sign of American Sign Language.
• Assess one's skill of using Sign Language through assessment mode of the Application.
• Direct conversion from Signs to english language through camera.

Technical Details of Final Deliverable

Final deliverable will contain an apk file which can be used to install the application on any smartphone having Android OS (v4.4) or latest. Documentation will also be provided to learn more about the application, how it was made and the technical details of the application and the trained model.

Documentation:

• SRS
• Technical Document
• Object Detection & Image Classification

Final Deliverable of the Project

Software System

Core Industry

Education

Other Industries

Core Technology

Artificial Intelligence(AI)

Other Technologies

Sustainable Development Goals

Quality Education

Required Resources

If you need this project, please contact me on contact@adikhanofficial.com