A wearable self assistive device for Dumb, Deaf and Blind People using IoT
Focusing and addressing on the difficulties faced by the diverse skills people, such as blind deaf and dumb, it is a difficult job to solve their challenges using a single device. A numerous research on each problem has already been done and solutions proposed separately, but none of them addressed
2025-06-28 16:30:07 - Adil Khan
A wearable self assistive device for Dumb, Deaf and Blind People using IoT
Project Area of Specialization Internet of ThingsProject SummaryFocusing and addressing on the difficulties faced by the diverse skills people, such as blind deaf and dumb, it is a difficult job to solve their challenges using a single device. A numerous research on each problem has already been done and solutions proposed separately, but none of them addressed these challenges together.
This project provide a single solution device which is wearable, swift, precise and low cost. The key aim of this device is to make various disabilities suffering people more self-confident by seeing, listening and communicating for them. This project provides Google API and Raspberry Pi-based support for visual, audible and vocal dumb people. The live broadcast speech of audibly challenged people on the microphone is sent to Google API server which converts the audio signal into text and display onto a LCD display. For blind people, a captured image is converted to text and speech which further converts into an audio format that reads out the extracted text and translating documents, books and other accessible documents on a speaker. It helps visually impaired individuals to learn by translating image to text and speech synthesis is performed for transforming it into an audio format that reads the extracted text, translating records, books and other accessible documents. For the vocally impaired people, the text input from an on-screen keyboard is provided which will convert into speech and speaks out in speaker.
This proposed wearable device is small in size which can easily wearable on wrist or arm. This will enable the deaf, dumb and blind individuals to work efficiently and comfortably at office, home and any public place. The support works at low latency at good internet connectivity.
Project ObjectivesThe main objective of our project is to design a wearable self-assistive device for deaf, dumb, and blind people suffering from mild level deafness which leads to person dumbness and permanent blindness. The purpose of this device is to establish communication between the disabled person and a normal person through wireless technology. This will enable the deaf and blind persons to work effectively at the home, office, and any public places with ease. The Raspberry Pi is the support system of the device which connects the camera, microphone, Wi-Fi module, speaker, and LCD display. The device works for the visually impaired as the camera clicks a picture of the document in jpeg, pdf, png formats, and then converted into digital text through Google-vision API and This text is then converted into audio using the TTS (Text to Speech) library and voice converted output according to the written document or book is obtained the output is in audio format through the speaker, audibly impaired as the microphone takes the spoken words as input convert into text using API and displays it as text on the LCD display, and for the vocally impaired as the user types the message in the LCD and the speakers give the output as audio. We use technology for tone-deaf, dumb, and blind persons to link with the world.
Project Implementation MethodThe implementation of wearable self-assistive device using IOT includes the following three stages. Each mode is separately dedicated for the blind, deaf and dumb respectively in the device.
Blind Mode
Deaf Mode
Dumb Mode
In blind mode, the image is converted into voice with the use of hardware and software program tools.The three-way slider is set to the blind mode. The device takes a picture of the written document or book and placed them on the holder of the device. The picture is saved in JPEG format and is passed to the Google Cloud Vision API to be converted to text. The extracted text then gets converted into speech using the GTTS API and the required text is thus converted to the audio format. This audio is given as an output by the high-quality speaker connected to the Raspberry Pi and thus the device enables the visually impaired person to understand the written document or book through the audio.
In deaf mode, the voice is converted into text with the use of; hardware and software program tools. The three-way slider is set to the deaf mode. The audio or the words being spoken to the user, who in this case might be a deaf person, are recorded as input by the USB Microphone connected to the Raspberry Pi of the device and is saved as a file in mp3 format. This audio file is passed to the Google Speech API which converts the audio into text for the user to understand. The converted text is then displayed on the 5 inch HDMI LCD screen available in the device, as a pop-up window exclusively created using python Tkinter for this module.
In Dumb Mode, When the three-way slider is used to set the device on the dumb mode, a pop up is displayed along with a customized keyboard which has been created using python tkinter, in it on the HDMI screen connected to the Raspberry Pi. The user who possibly is vocally impaired can type whatever he wants to convey using the keyboard in the screen as text .The typed text is converted into audio format using the gTTS API and the audio file of the required text is obtained.
Internet of Things (IoT) is a new technology that has made an enormous impact on the modern world. This project adds mainly four features: convert text into audio, audio to text on LCD display, image (print/pdf/jpeg) to text conversion. This project will enable the deaf and blind persons to work effectively at the home, office, and any public places with ease.
Benefits of the Project:
- Addressing the problems faced by the differently abled people such as visually, audibly and vocally challenged, through a single device is a tough job.
- The aim of the project is to create a single device solution in such a way that is simple, fast, accurate and cost-effective.
- This is an IoT based system and hence can be made automated.
- By using Raspberry Pi, the system becomes scalable and flexible.
- The system can be modified easily without disturbing the other components in the system.
- As we will use Raspberry Pi to develop the system, the total system has become a low power system.
- New embedded technologies can be easily inserted into this development, due to the use of raspberry pi.
Features:
- Wearable:
It is easily wearable and easy to carry for a person.
- Technological link:
This device help the differently abled person to communicate with other person easily through these technology. This device will generate a link for differently able person to the world.
- Education Purposes:
This will enable the deaf, dumb and blind persons to work effectively at the school, colleges, universities, home, office, and any public places with ease.
- Barrier:
A Self Assistive wearable Device for Deaf & Blind and dumb People Using IOT: benefits from the increasing use of barrier-free facilities and assistive devices.
Technical Details of Final DeliverableThe final product will be an integrated system based on hardware and software. The software will be a google API system that is used as an assistant Software will be used for simulation to detect the image text and convert into audio, convert audio into text.
Hardware includes Raspberry pi, Camera module, wifi module, microphone, speaker, and screen display. IoT will be used to connect the hardware equipment to google cloud vision API.
1. Hardware and software integrated system
2. Project report
3. Publication in well-reputed journal/ conference
Final Deliverable of the Project HW/SW integrated systemCore Industry HealthOther Industries Education , IT , Medical Core Technology Internet of Things (IoT)Other Technologies Cloud Infrastructure, Wearables and Implantables, OthersSustainable Development Goals Good Health and Well-Being for People, Industry, Innovation and Infrastructure, Partnerships to achieve the GoalRequired Resources| Item Name | Type | No. of Units | Per Unit Cost (in Rs) | Total (in Rs) |
|---|---|---|---|---|
| Total in (Rs) | 68945 | |||
| Raspberry pi 4 b+ 8gb | Equipment | 1 | 24950 | 24950 |
| LCD Display | Equipment | 1 | 6890 | 6890 |
| Logitech HD Camera | Equipment | 1 | 7960 | 7960 |
| Super mini microphone | Equipment | 1 | 2550 | 2550 |
| SD-card convertor 3.0 | Equipment | 1 | 800 | 800 |
| Case for Raspberry pi | Equipment | 1 | 1890 | 1890 |
| Speaker | Equipment | 1 | 2875 | 2875 |
| SD card 32gb | Equipment | 1 | 2200 | 2200 |
| Power Bank | Equipment | 1 | 7000 | 7000 |
| Power Cable | Equipment | 1 | 510 | 510 |
| VGA to HDMI Converter | Equipment | 1 | 580 | 580 |
| Wires | Equipment | 30 | 40 | 1200 |
| Miscellaneous (Travelling, Printing etc) | Miscellaneous | 1 | 9540 | 9540 |