SMART GLASSES FOR VISUALLY IMPAIRED USING DEEP LEARNING

The world is progressing in every aspect now-a-days; it would not be possible without the help of technology. By technology we achieved things that were once a part of our imagination. With the help of technology we aim to build smart glasses for the people who are visually impaired or partially bli

Project Title

SMART GLASSES FOR VISUALLY IMPAIRED USING DEEP LEARNING

Project Area of Specialization

Artificial Intelligence

Project Summary

The world is progressing in every aspect now-a-days; it would not be possible without the help of technology. By technology we achieved things that were once a part of our imagination. With the help of technology we aim to build smart glasses for the people who are visually impaired or partially blind so they can see and interact with the world just like any normal person. Speaking about the project, the smart glasses will consist of a camera, number of sensors attached to it and a portable GPU for processing the data. On the software side, we will be using Machine learning and Artificial Intelligence based algorithms to detect objects with accuracy and with low latency which can help the user to make decisions while moving around. There is so much research on the proposed project from different universities all over the world. In China, researchers conducted a study to make a better and efficient version of smart glasses by adding a navigation system to it. Demographically, around 2.5 to 3 million people are blind in Pakistan and they are using traditional methods to lead their way around places which in return makes them face many problems and accidents because of the lack of knowhow about their surroundings. In order to address this problem, we came up with a solution of commercially manufacturing such glasses for the visually impaired or partially blind which would attract a plethora of buyers all across Pakistan. Moreover, by this venture, not only we will be carrying out an entrepreneurial endeavor but also contribute to society and help people with blindness.

Project Objectives

The following are the main objectives:

1. To provide virtual eyes to blind people.
2. To build a low-priced gadget
3. To achieve a low latency rate while identifying objects
4. To replace traditional canes with smart wearable glasses to get in-depth detail of the environment to the user. 

Deep learning (DL) methods are widely used to train easily and give accurate latency results. This project mainly focuses on developing smart guiding glasses that work on specific functionalities that reduce the image(s) matching time, more accuracy and latency, and fast response time. Smart glasses first detect the object(s) and then inform the blind people via audio output such as an earpiece. The blind person will be guided to turn left or right and will also tell the distance between the person and the objects. The glasses will work in limited indoor and outdoor areas. In this project, we will work to furnish a solution to improve the accuracy and quick response time that cannot be delayed using machine learning (ML), Deep Learning (DL), and Image Processing methods.

Project Implementation Method

The project is divided into two parts:

1. Software
2. Hardware

On the software side, we will implement the image recognition method to identify the objects and store them in datasets, as well as real-time object detection using deep learning. The CNN algorithm recognition method is slow as compared to YOLOv3 while YOLO detects from three different angles. Also, YOLOv3 executes multi-marks characterization for objects detected in pictures. To prepare our article finder, we have to manage its learning with bounding box explanations. We draw a crate around each item that we need the indicator to see and mark each container with the article class that we might want the identifier to anticipate. For upcoming work, we can say that YOLO has more accurate in comparison to CNN.

On the hardware side, we will use the stereo camera to get all the possible angle data from the real world that mimics a real eye. It can detect the objects in a specific range, which provides real-time circumstantial data that mimics a real eye. The raspberry pi 4b will be used for processing the data along with the Movidius NCS2 stick for neural network processing, the headphones and mic will use for assistance with a blind person and at last, the 3-D printed glasses will hold all the sensors and hardware for easy wearing.

Benefits of the Project

This project aims to help blind people in their ways as well as, it will be also helpful during sports for visually impaired people such that they can play slow games to wear those glasses which will help them to guide during the playoff. For further improvement, lots of students and readers would just scan the page and listen via audio output. Through this project, the majority of blind children and teenagers will be attracted to the schooling and learning phase.

There are some branded products available at the international market, but the disabled people have cost issues, therefore, they cannot buy that glasses and could not do their normal work to help them to live a normal life and bring them to the normal fold of life to enable them to work in industries in academia and other parts of the societies and make themselves useful, our cheap smart glasses will also help to decrease the depression in the visually impaired people. Moreover, we can expand it into world marketing at a low cost that will play a vital role in Pakistan.

Technical Details of Final Deliverable

Smart glasses for visually impaired people will provide the required assistance to both blind as well as partially blind people. Providing real-time object detection capability with audio output will allow users to get an idea of their surroundings with greater precision than using a blind man’s cane.

The stereo camera will mimic the human eye to get all possible angles of real-world and capture its surroundings for further processing and get better results from the machine learning algorithm. The ML algorithm YOLO will be trained on our custom dataset to detect objects quickly and precisely with a low latency rate. The YOLOv3 then implemented on the Raspberry Pi 4b along with the neural network processing Intel Movidius NCS2 stick for detecting objects in realtime. The blind person will also be able to find a free path along with the directions which he is walking towards and also he knows the distance to the objects that how far is the XYZ object. The audio output PCM5122 for Raspberry Pi will be used for assistance to the blind person and a mic is to get interacted with the custom build python based smart assistant to use the features of gadgets.

At last, all the hardware components are mounted on specially designed 3D printed glasses for easy wearing and light weight.

Final Deliverable of the Project

HW/SW integrated system

Core Industry

Health

Other Industries

IT

Core Technology

Artificial Intelligence(AI)

Other Technologies

3D/4D Printing, Wearables and Implantables

Sustainable Development Goals

Good Health and Well-Being for People, Industry, Innovation and Infrastructure

Required Resources

If you need this project, please contact me on contact@adikhanofficial.com