Speech enabled interactive simplification Hub in English and urdu

Our project is basically a Virtual Reality based Educational Android Application for students. By this application students will be able to visualize 3D structures of the topics with the help of VR Headsets. Along with visualization, it will also show the description of the topics in both English an

Project Title

Speech enabled interactive simplification Hub in English and urdu

Project Area of Specialization

Augmented and Virtual Reality

Project Summary

Our project is basically a Virtual Reality based Educational Android Application for students. By this application students will be able to visualize 3D structures of the topics with the help of VR Headsets. Along with visualization, it will also show the description of the topics in both English and Urdu Language. It will include text simplification feature which will simplify the text. Plus it will be capable to read aloud description and simplified sentences in Urdu. It will convert Urdu Text into Speech form. Anyone who is blind will be able to hear the description of the topics and simplified version in Urdu Language. The application will help non- native speakers and dyslexic children. Non-native speakers, poor readers and children suffering from aphasia, dyslexia, and autism have difficulty in understanding or remembering new and difficult words. They have problems translating language to thought (in listening or reading) and thought to language (in writing or speaking). Students are not able to learn new concepts of their curriculum by traditional teaching method. They cannot understand new concepts by just reading them and not visualizing them. And Anyone who is blind will be able to hear the written text in Urdu Language.

Project Objectives

1. To improve learning skills of students with 3D image visualization. With 3D visualization students can easily understand different topics in better way.
2. With the help of 3D visualization of new concepts students will be able to easily understand and remember them efficiently.
3. Visualization along with text simplification will enhance student’s learning capabilities.
4. Other than normal users, this framework is particularly useful for visually impaired people.
5. Simplification has a variety of important societal applications, for example increasing accessibility for those with cognitive disabilities such as aphasia, dyslexia, and autism, or for non-native speakers and children with reading difficulties.
6. Text simplification will improve readability and understanding.
7. People can listen Urdu Text in Speech form.

Project Implementation Method

Technologies to be used:

• Android application will be developed using android studio using Java.
• 3D Animations will be developed in blender and VR headsets will be used for visualization of 3D animations.
• Text simplification will be done through NLP which will first divide the sentences into small pieces and then convert difficult words to their easier synonyms.
• Urdu Wikipedia corpus will be used as a source to build a new simplification corpus to train the simplification model.
• Text-to-Speech System using Python, HTML, CSS, JavaScript and Java Language Toolkits.

Research Aspects:

• Models for Automated Text Simpli?cation.
• Sentence Simpli?cation with Core Vocabulary.
• Integrating a trained model into a GUI based application.
• Training and Testing a Text-to-Speech system for Urdu language using Urdu Speech Corpus having Transcribed Text for each Audio File.

Benefits of the Project

1. Text simplification will improve readability and understanding.
2. 3D image visualization will improve learning skills of dyslexia children.
3. Students can easily understand different topics in better way.
4. An extra benefit is that, just by using it, people can simplify any text with difficult vocabulary into simplified version and can understand it. 
5. Simplification has a variety of important societal applications, for example increasing accessibility for those with cognitive disabilities such as aphasia, dyslexia, and autism, or for non-native speakers and children with reading difficulties.
6. The application will help non- native speakers and dyslexia children. Non-native speakers, poor readers and children suffering from aphasia, dyslexia, and autism have difficulty in understanding or remembering new and difficult words. They have problems translating language to thought (in listening or reading) and thought to language (in writing or speaking).
7. Students are not able to learn new concepts of their curriculum by traditional teaching method. They cannot understand new concepts by just reading them and not visualizing them. 3D visualization will help them better understand new concepts.
8. The application implemented into mobile software is a contribution to improving the quality of life and working conditions of people of Pakistan.
9. The current generation has grown up in the digital age, and demands greater effectiveness and naturalness when using consumer devices such as laptops PCs mobile telephones, TV Steering sets, PDAs, car navigation kits. Human language is the richest and most effective way for interacting with machines.

Technical Details of Final Deliverable

Modules to be built:

• Text simplification model for Urdu to Urdu simplification.

• Preparing the data and building a simplification model.  The task involves defining the scope, training, tuning and testing of the model for simplification.

3D images creation:

• Learn and use the tool Blender for creating 3D images. Adding description to the 3D visualization.

Text-to-Speech Converter for Urdu Language:

• Preparing the data and building a converter.  The task involves defining the scope, training, tuning, and testing of the system for conversion of Text into Speech form.

• We will use Tacotron for Speech Generation.

Interface design:

• Interface of the whole project will be implemented in Java.

System integration:

• Learn to integrate trained simplification model and 3D visualization with description in Java GUI environment.

Final Deliverable of the Project

Software System

Core Industry

Education

Other Industries

Core Technology

Augmented & Virtual Reality

Other Technologies

3D/4D Printing, Others

Sustainable Development Goals

Quality Education, Reduced Inequality

Required Resources

If you need this project, please contact me on contact@adikhanofficial.com