Social Glasses for Deaf using Augmented Reality
The proposed system has two ends, i.e., a mobile application end and a hardware end. The hardware consists of augmented glasses headset which provide captions on the screen of the glasses. Augmented text will be shown as the subjects speak, on the Glasses Display. This sense of emersion will help th
2025-06-28 16:29:34 - Adil Khan
Social Glasses for Deaf using Augmented Reality
Project Area of Specialization Augmented and Virtual RealityProject SummaryThe proposed system has two ends, i.e., a mobile application end and a hardware end. The hardware consists of augmented glasses headset which provide captions on the screen of the glasses. Augmented text will be shown as the subjects speak, on the Glasses Display. This sense of emersion will help the deaf and mute be able to be a part of society with real-time translation through portable AR glasses headset.
The system will display text accurately by first converting the audio into speech and then the text. Once the text is found, the system will start detecting the speaker by checking the video for any person. Afterward, the system will associate the person with the text on a successful match between them both.
The system is also having emotions as emoji icons according to the different voice tones of the speaker’s speech. Once detected the expressions of the speaker will be augmented along with the speech text.
Similarly, the app can detect a user's friends which are saved by them with their pictures and other details. If the system successfully detects the speaker as a friend by comparing their facial features, then it will augment their details with the associated person.
Project Objectives- To build a system for deaf and mute people to help them with communication.
- Digitally capturing the sound and displaying the user's needed information in the augmented text is the main motive of this project.
- Removing the barrier between deaf and normal person so that be a part of society.
- Using state of the art technology because it is demanding in near future.
- Providing free app for benefiting society.
Agile Development Methodology, Scrum in Specific.
Software is based on iterative and incremental processes due to which Scrum works best. Scrum is an adaptable, fast, flexible, and effective agile framework. It supports teamwork and maintains project history(backlog) and is ideal for our group of 3 members. We are using Trello for maintaining our project. Daily meetings are conducted, and tasks are evaluated based on the sprint backlog. After each module, a new sprint is made.
Benefits of the ProjectThe system can be used by the public as a means to socialize in the future. Our system has the capability to be used by the wide public by introducing features of social media into their daily lives. Once successful in its mission to help socialize the deaf, features such as chat, meetings, navigation, reminders, games, advertisements, etc. can all be merged into this augmented system.
So far, there is no reliable solution for the deaf to get sound information in real-time so that they can easily interact with daily life. Such a system will create the initial funding for its growth. The go-to-market strategy can be achieved as this system will deal with all the communication gaps that arise with the deaf by encouraging other developers to further add new features.
It is our responsibility to give back to society, and this is how society lives. The Social Glasses are an initiative to help the deaf community around the globe, be able to use the prospects of technology for the betterment of their lost abilities and eventually help society as a whole.
Technical Details of Final DeliverableAR Glasses Headset
AR Glasses Headset use the phone’s hardware to process the image that is shown on the glasses screen.
Application
The Application is multiplatform and works on the AR glasses as well as mobile applications for Android & IOS platforms. It consists of 4 core functionalities:
Speech to Text Augmentation
Voice Emotion Detection
Alerts System
Friend/Identity Detection
There are three modes of Social Glasses:
- Mobile
- VR
- AR
Document
Complete FYP Documentation, inclusive of all that is mentioned above.
Final Deliverable of the Project HW/SW integrated systemCore Industry OthersOther Industries IT Core Technology Augmented & Virtual RealityOther Technologies Artificial Intelligence(AI), 3D/4D PrintingSustainable Development Goals Good Health and Well-Being for People, Industry, Innovation and InfrastructureRequired Resources| Item Name | Type | No. of Units | Per Unit Cost (in Rs) | Total (in Rs) |
|---|---|---|---|---|
| Total in (Rs) | 66000 | |||
| VR Headset | Equipment | 1 | 5000 | 5000 |
| Xiaomi Note 9s | Equipment | 1 | 35000 | 35000 |
| AR Headset | Equipment | 1 | 8000 | 8000 |
| Microphone | Equipment | 1 | 1000 | 1000 |
| Camera | Equipment | 1 | 5000 | 5000 |
| Headphones | Equipment | 1 | 2000 | 2000 |
| Miscellaneous | Miscellaneous | 1 | 10000 | 10000 |