A Prototype Model for Bionic Eye by Instance Segmentation Technique

Blindness defined as the state of being totally sightless in both eyes. Retinal degenerative diseases such as retinitis pigmentosa (RP) and age-related macular degeneration (ARMD) lead to a gradual loss of photoreceptors in the eye that may cause profound visual impairment in more than 10 million pe

Project Title

A Prototype Model for Bionic Eye by Instance Segmentation Technique

Project Area of Specialization

Artificial Intelligence

Project Summary

Blindness defined as the state of being totally sightless in both eyes. Retinal degenerative diseases such as retinitis pigmentosa (RP) and age-related macular degeneration (ARMD) lead to a gradual loss of photoreceptors in the eye that may cause profound visual impairment in more than 10 million people worldwide. Analogous to cochlear implants, retinal neuroprostheses (also known as the bionic eye), aim to restore vision to these individuals by electrically stimulating surviving retinal cells to evoke neuronal responses that are interpreted by the brain as visual percepts (phosphenes). However, the quality of current prosthetic vision is still rudimentary. The ability of object recognition and scene understanding in real environments is severely restricted for prosthetic users.

Moreover, In 2021, University of California published his research with the help of Computer vision and image processing technique we can improve the bionic vision for blind people. In this project we will make a “Bionic eye prototype by using instance segmentation technique” to improve prosthetic vision. The work that has been done previously has several limitations, they limited their work on outdoor scenes. We will apply scene simplification strategy on indoor scenes, we combine computer vision and deep learning techniques that will help blind people to recognize different indoors objects and we use an established and psychophysically validated computational model of bionic vision to generate realistic predictions of simulated prosthetic vision (SPV).

Project Objectives

1. We will make a prototype for blind people that can improve the bionic vision, by using Deep learning scene simplification strategy.

2. We will use an established and psychophysical validated computational model of bionic vision to generate a realistic prediction of simulated prosthetic vision.

Project Implementation Method

1. Python framework Jupyter IDE we will use to build Prototype Model for Bionic Eye by Instance Segmentation Technique. For image pre processing we will use OpenCV library. Deep Learning Tools we will use are TensorFlow, Numpy, Scipy,Cython, ScikitImage, Matplotlib and Joblib.
2. We will perform instance segmentation of objects using the original architecture of Mask R-CNN. We will use Pre-trained Mask R-CNN model on custom dataset. We will perform pixel-wise masks by using ResNet101 based Mask R-CNN trained on MS COCO dataset.

3. We will use Python-based Simulation framework pulse2percept to validate our software.

4. The hardware we will use to build Bionic Eye Prototype by Instance Segmentation Technique are Raspberry pi camera module v3 DataSheet, Raspberry-Pi 4 model b 4gb DataSeet, 4inch Raspberry pi HDMI LCD Display Module 800x480.

Benefits of the Project

1. A prototype model for bionic eye will be improve the bionic vision at certain extent by using the instance segmentation technique.
2. It will restore artificial vision of visually impaired people. That helps in daily life activities, such as detecting objects and people.
3. The software will be up-gradable to benefit for future innovation.
4. It will improve the quality of life o blind patients.

Technical Details of Final Deliverable

A prototype model for bionic eye by instance segmentation technique mapped in this proposal which will help blind people to see and identify shapes of objects and people. Software will be upgradeable for future enhancement like for adding new features that will blind person can cross a road by using depth technique, and will also detect room by using saliency technique. A pi camera will capture 2D images of different objects such as chair, table, bed, and people; then it will send the images to Raspberry Pi for further processing. Deep learning algorithm will burn in Raspberry Pi that can perform further task, after receiving the input image first task, it will pre process the image than detect specific objects, the model will train on that objects the second task is it will mask and silhouettes the object or subtract the background to avoid noise, it will give segmented output than the last task the Pi will be performed, It will validate the output by an open source computational model of bionic vision to generate a realistic prediction of simulated prosthetic vision. Futhermore, the raspberry pi will show the simulated results on LCD screen.

Final Deliverable of the Project

HW/SW integrated system

Core Industry

Medical

Other Industries

IT

Core Technology

Artificial Intelligence(AI)

Other Technologies

Artificial Intelligence(AI), Augmented & Virtual Reality

Sustainable Development Goals

Good Health and Well-Being for People

Required Resources

If you need this project, please contact me on contact@adikhanofficial.com