Assistive text reading device for visually impaired people using computer vision methods

Giving visually impaired people the ability to read text has been a topic of keen interest for quondam years. Many projects and prototypes embodied this topic but some sort of degrading factors that prohibit users to use them were present. So, keeping in mind all that factors we want to contrive a a

Project Title

Assistive text reading device for visually impaired people using computer vision methods

Project Area of Specialization

Wearables and Implantable

Project Summary

Giving visually impaired people the ability to read text has been a topic of keen interest for quondam years. Many projects and prototypes embodied this topic but some sort of degrading factors that prohibit users to use them were present. So, keeping in mind all that factors we want to contrive a assistive device( Finger worn device in the form of ring ) that not only help visually impaired people but also sighted person to read text, detect currency, recognize colors and in avoiding hurdles. Our main focus is on run time efficiency and typical frame processing time. User will point the finger towards the text that he/she wants to read and our device will convert that text to voice by first detecting the text using text extraction algorithm and then converting the detected text into speech using TTS (text to speech) algorithm. A mini camera is fixed on the device wearable on the Finger that is capturing the images and after processing text is detected from the image and converted into speech. Our main focus is to produce a gadget that let visually impaired people to encounter the reading material in a same way as sighted person.

Project Objectives

• We are designing assistive text reading device to solve the issues through which visually impaired or blind people have to go.
• We note two desired qualities:

• Assistive technology should be socially acceptable.
• Supports independent and portable interaction.

• Its design is motivating as it includes small case wearable on finger and mobile, which enables a more manageable operation with little setup.
• Device is autonomous, wireless and includes a single button to interaction.
• We have to overcome the troublesome for accessing text in less than ideal conditions
• For the completion of project, we have time limit of 8-9 months.

• Our purpose is to encourage the visually impaired or blind to be aware of their abilities and providing them opportunities to participate in the world.
• We have to focus on run time efficiency and typical frame processing time on our machine, which is suitable for real-time processing. Low running time is important here.

• Task of such device is to convert the text into speech to be helpful and understandable for blind.
• Includes mobile app that is used for processing.
• After achieving goal of reading, we will add other features like color detection, hurdle detection and currency detection.
• Device reads plane text and can handle non-planer and uniform surfaces.
• A picture of specified word will be taken by a mounted camera and through internal procedures of different algorithms, word to speech conversion occurs and user will listen the word aloud.
• Text to speech Conversion Includes following steps:
• Text detection
• Text recognition
• Text to speech conversion

• Besides working on software, we focus on working and detailed analysis of every component of hardware.
• Only first time Bluetooth connection with a smartphone is done manually, otherwise it will be connected automatically.
• Our purpose is to make the device less time taking i.e. it does not need all typical devices like find the phone, unlock the screen, open the app, take a picture, listen for the answer, turn off the phone and put it away. By just clicking the button, its procedure starts.

Project Implementation Method

• The idea which was selected is to implement an “Assistive text reading device for visually impaired”.
• Studied a lot of research papers to know about basic components requirement, its working and implementation.
• We focused on previously used work on such devices for visually impaired and their positive or negative sides keeping in mind the improvements which could be possible in previous work.
• We also focus on those features of device which will contain minimum drawbacks and will be more helpful.
• We focused on run time efficiency.
• We read about simplest design with easily available components.

• We made an overview on project keeping in mind all the requirements.

• We note three desired qualities:

• Assistive technology should be socially acceptable.
• Supports independent and portable interaction.
• Should be affordable.

• Our project will motivate visually impaired and blind to recognize and show their abilities.
• We divided it into two parts consisting of hardware and software. Which is further divided into mini parts.
• Software includes text detection, text recognition and text to speech conversion which is divided in all three members individually.
• We have to make a list of components and order them so they will be available as earlier as possible to start our work.

• Our hardware will be limited to following components:

• Image acquisition module (C329 UART camera)
• AVR microcontroller (ATmega32u4)
• Bluetooth module (RN42)
• Push button switch
• Polymer Lithium-ion battery

• Data sheet of each component is studied thoroughly before implementing that component.
• Initial study of each component separately and coding of algorithm on android studio.

• After availability of hardware, we have to test each separately and make a due date for compilation of each step.
• Initially, each component will be tested separately.
• Overall completion of project will take 8-9 months.
• For project implementation we will use latest tools.
• For software, android studio along with Genymotion is used.
• For checking connections of Bluetooth module blue term is used.
• For capturing and grabbing image from camera module, image capturing software (compatible with our camera) is used.
• Image captured from camera is sent to microcontroller using UART interface and microcontroller deliver that image data to Bluetooth module. Bluetooth module will share that data with mobile app which will convert recognized text to speech.

Benefits of the Project

While in a quest to find a final year project, our main focus was to work on a project that can ease people with disabilities. So finally after going through so many research papers, we find a one we are looking for. An assistive device for visually impaired that can be wearable on index finger that can perform different tasks i.e. reading, color detection, hurdle detection and currency detection. For not getting jumbled, we time bound ourselves to one task at a time.

 Our first focus is to make this device to convert text from surrounding into speech. So that the one who cannot read, can able to listen what is written. Through our first goal, we can have following benefits:

1. The brails are used for reading by blinds but not every book can be available in a brail form. So this device can help blinds and visually impaired in reading any book.
2. This device can help blinds and visually impaired to read sign boards, menu cards and food item names during shopping.
3. This device can also assist children in reading.

Second task is to use this device as a hurdle detector:

1. This device will detect hurdles while walking from a specific distance, this will discharge blinds from carrying walking stick.
2.  Not only blinds but also old people can use it while walking.

Third task is to use this as a color detector:

1. Help blinds and visually impaired while deciding about their clothing, while shopping.
2. This will help children in identifying colors.
3. Help color blinds in recognizing colors.

Fourth task is to detect currency:

1. Currency detection feature will help blinds to pay bills independently.

Basically, through this project our main effort is to bring independence in the lives of disable people, so that they can enjoy their life like other normal people. Even if not fully, this device can accomplish this task to some extent.

While doing this project, we are getting skills that are not only helping now but will also help in future. Android App Development is the major part of this project, which will basically perform the main task. So while developing app for our final year project, we have been learning java language and xml which were not a part of our engineering courses. In addition to this we are using Android studio software for developing our app which is also new to us. So this will give us advantage in future that we can able to develop Apps and can also opt it as profession.

Apart from this, we are learning skills that can only be learnt through group work i.e. how to divide a work without being biased, how to help each other to accomplish task without pointing each other shortcomings,

Furthermore, we never had an experience of having deals with trader. For our hardware of the project, we communicate with robotics.pk which is a dealer from Karachi. As the hardware was not available in Pakistan, robotics.pk brought that for us from foreign firm.

Technical Details of Final Deliverable

Final deliverable comprises mainly the following components:

1. C329 Image acquisition module
2. Microcontroller atmega32u4
3. RN-42

C329 will capture image using OV7725 CMOS sensor embedded on its chip and that image will be compressed by OV529 JPEG engine on the board. Then the picture will be sent to atmega32u4 using UART protocol. ATmega will transmit the picture to Bluetooth module using another UART and RN-42 will transmit the image to mobile app .Finally. App will recognize the text in the image and convert that text into speech.

Now technical details of components are given as

C329 UART module

The camera module utilized in the project has following technical details:

• The module will perform JPEG compression
• Will attach to host through RS232 interface
• It is small size, only 20x28mm
• Its UART interface up to 921.6kbps
• Multiple lenses can be attached with it.
• Comprise of OV7725 VGA CMOS sensor which is only ¼” inch
• Sensitive to light because of OV7725 which excels in low light
• Will operate in wide temperature range of -20 degree to 70 degree
• Best expected frame rate will be 2FPS
• Picture will be compressed by OV529 JPEG engine and UART protocol will be used by it to communicate with microcontroller.

ATmega32u4

Microcontroller used has following features and will fulfill below given requirements:

• 8 bit microcontroller
• Maximum clock frequency is 16MHz
• Program memory size is 32KB
• Operating Supply Voltage is 2.7 V to 5.5 V
• Minimum operating temperature is -40 degrees
• Maximum temperature range is +85 degrees
• Available in SMT/SMD form
• Mounted on Pro micro
• Having height of 1.05mm
• Width is 10mm
• DATA RAM type is SRAM
• 2.5KB SRAM
• Program memory type is flash  
• Only its two UART and one digital I/O will be used

    RN-42

Wireless module going to be used in final deliverable will fulfill following requirements:

• On board UART interface
• Will deliver up to 3Mbps data rate up to 20meters
• Will auto discover/ pair with host and require no software configuration ( instant cable replacement)
• Consumes low power
• Will automatically connect in master mode
• Compatible with 3.3V and 5V microcontrollers
• Will work in wide temperature range of -40 degree – 80 degrees
• Dimensions are 59.18x29.48x11.43mm
• Default baud rate is 9600 but will maximally work at 115k
• Having LEDs for status indication of power and pairing mode

Final Deliverable of the Project

HW/SW integrated system

Type of Industry

Others

Technologies

Wearables and Implantables

Sustainable Development Goals

Good Health and Well-Being for People, Industry, Innovation and Infrastructure, Reduced Inequality

Required Resources

If you need this project, please contact me on contact@adikhanofficial.com