Real time Object Detection and Recognition For Blind and Partially Blind People

To create an intelligent system which can perform almost identical function as that of human eye is our goal. We want to use NVIDIA Jetson nano development kit due to its 128 cuda cores which are highly efficient in machine learning a

Project Title

Real time Object Detection and Recognition For Blind and Partially Blind People

Project Area of Specialization

Artificial Intelligence

Project Summary

To create an intelligent system which can perform almost identical function as that of human eye is our goal. We want to use NVIDIA Jetson nano development kit due to its 128 cuda cores which are highly efficient in machine learning and neural network which we are implementing in our project. Jetson nano will work like a human brain which will process the images taken by the Raspberry Pi cameras present on the headwear using neural network algorithms. This network will analyse the images to parts in order to compare them with the most important characteristics of the object present in the database, where images for comparison will be stored. Mathematical equations programmed in Python will be used to ensure a high accuracy of approximate 90% and above. Afterwards the object that has been detected and recognised it will be conveyed to the user by means of audio output using wireless earphones. 

The device will also be able to detect and measure the distance of the detected object from the user using the Stereo Cameras in synchronization, thus further increasing the accessibility to the user.

The device will be a 3D printed portable headwear much like glasses and will be able to be used on the go. 

Project Objectives

Our objective is to create an intelligent system which will work as a human eye and transfer detected objects and images to the brain of the system. The brain in turn will analyse the images, brain here would be the jetson nano chip which is a single board computer and with help of the input provided by the camera’s analysis will begin using complex neural network algorithms. This network analyses the images to parts in order to compare them with the most important characteristics of the objects in the images related to the database, through which the images are compared. When ensuring that the characteristics match the mathematical equations programmed in the language of the Python, the objects in the image are detected.

Our main objective is to provide a cheap but accurate device to blind and partially blind people for use in real life situations with Realtime object detection and recognition.

Project Implementation Method

To implement the project, we will use TensorRT library, which is used for deep learning, TensorRT will utilize the full potential of the cuda cores present on the jetson nano and allow for a more in-depth object detection at real time while also being able to detect a large number of objects.

For setting up the stereo camera we will use OpenCV which works as real time computer vision allowing for faster object detection rate and the stereo camera will have binocular vision as identical to human eye and will be able to calculate the depth of the objects also.

Python will be used for creating the mathematical equations and implement them in the jetson nano.

Yolo or also known as you only look once algorithm will be used along with tensor flow to maximize the accuracy of the detected objects.

All the apparatus including the

1. Jetson nano
2. Jetson nano case
3. Wireless earphones
4. WIFI Bluetooth chip
5. Stereo cameras

will be encased in a 3D printed Headwear which we will design by ourselves to make the headwear more lightweight than the normal components e.g. plastic, metal

Benefits of the Project

Blind and partially blind people will be able to hear as to what object is Infront of them and how far away the object is from them with greater accuracy thus allowing them more accessibility and expanding possibilities for them to achieve their full potential.

The project is relatively cheap compared to already existing solution to the same problem.

The project will be able to not only detect the object itself, but it will also convey to the user of the subcategory if any of the detected object.

Technical Details of Final Deliverable

The final deliverable project will have the following equipment working on the different parts of the project.

Object Detection:

For object detection, two Raspberry Pi cameras will be used as stereo cameras in synchronization with each other to simulate a human binocular vision and calculate the depth of the object and capture the actual object image in Realtime.

Object Recognition:

For object recognition, the 128 cuda cores of the jetson nano will be used to work in parallel with each other and compare the object with the database stored object in Realtime.

Output:

Wireless earphones will be used to convey the detected object to the user as blind people have more sensitive ears.

Headwear:

The overall object will be encased in a lightweight 3D printed headwear which would be easy to mass produce.

Libraries being used:

1. TensorRT – Deep Learning library
2. OpenCV – Real Time Computer Vision

Python Algorithms:

1. Yolo
2. Tensor Flow

Final Deliverable of the Project

HW/SW integrated system

Core Industry

IT

Other Industries

Medical

Core Technology

Artificial Intelligence(AI)

Other Technologies

3D/4D Printing, Wearables and Implantables

Sustainable Development Goals

Good Health and Well-Being for People, Industry, Innovation and Infrastructure

Required Resources

If you need this project, please contact me on contact@adikhanofficial.com