Handwritten Text Recognition

Deep learning is a subset of Machine Learning (ML) area. Application of the multiple-layer artificial neural networks has made a significant impact in advacning pattern recognition tasks in recent years like in handwriting recognition. Handwritten Text Recognition (HTR) am

Project Title

Project Area of Specialization

Artificial Intelligence

Project Summary

Deep learning is a subset of Machine Learning (ML) area. Application of the multiple-layer artificial neural networks has made a significant impact in advacning pattern recognition tasks in recent years like in handwriting recognition.

Handwritten Text Recognition (HTR) amins to enable the computer machines to automatically transcribe documents. The hidden Markov model and Artificial Neural Networks (ANNs) are the two main techniques for HTR well explored in literature . Targeting HTR task, we explore ANNs in this project. At first, we achieve enhancement of input images using preprocessing methods making the input to the classifier clean as compared to its original form. For feature learning and classificationm, we explore Convolutional Neural Network (CNN), Recurrent Neural Network (RNN), and Connectionist Temporal Classification (CTC), demonstrate its ability and efficiency in recognizing handwritten sentences from the images. We build a Neural Network (NN) using images from the IAM dataset to train it. The layers of the Convolutional Neural Network extract the relevant features from the input images and a Recurrent Neural Network propagates the information through the image. The RNN outputs matrix is given to the CTC to decode the matrix. We detect and make predictions on handwritten sentences in English and the predicted output is passed through Google-Text-To-Speech (GTTS) which is a python library and a CLI tool to interface with Google Translate text-to-speech API, which provides the predicted output in the form of speech.

Project Objectives

The main objective of our project is to develop an hand written text image reader which works on scanned documents in the form of images. Taking the scanned documents which contains text in handwritten form, we extract that text from the image and convert it into speech.

Project Implementation Method

The implementation aims to demonstrate that the system is able to recognize handwritten sentences and transform them from analog to digital formats using a Convolutional Neural Network (CNN) machine-learning approach, making the message accessible to others.

Electronic devices can recognize handwritten data, which can be trained through programming and then generate predictions based on large amounts of data and algorithms.

The proposed approach makes use of artificial neural networks (ANNs). To extract relevant features from the input image, multiple Convolutional Neural Network (CNN) layers are trained. The Recurrent Neural Network (RNN) layers get a 1D or 2D feature map (or sequence) from these layers. Information is propagated across the sequence through the RNN. Following that, the RNN's output is mapped onto a matrix with a score for each character per sequence element. Because the ANN is trained using a specific coding scheme, the RNN output must be decoded to obtain the final text. The Connectionist Temporal Classification (CTC) technique trains and decodes this matrix.

Benefits of the Project

A main benefit of this project is to make reading easy for visual impared people because they can't see but through voice they can hear and understand.

Technical Details of Final Deliverable

The final delivarables include a programme (software system) consisting of various algorithms which works in real time on real world scenarios like any hand written document in English language and is able to recognize handwritten sentences and read them in clear speech for the visually impaired people or for other uses.