Air Writing

Project Title

Air Writing

Project Area of Specialization Artificial IntelligenceProject Summary

We are building a software that can be used for creating, recognizing and visualizing alphanumeric 
data in air. Our software uses the module of MediaPipe which detects hand and finger and we create 
writing finger by using the position of tip of index finger. This writing event is shown on canvas using 
OpenCV and frames are captured and saved as images in backend. Initially we trained the model for 
alphanumeric recognition for our air writing software. Later our focus is to predict word. Later the 
images will be tested by model trained by dataset like EMNIST or IAM for word and digit recognition 
and this predicted word/digit will be shown on screen and we also will do speech recognition so it 
speaks or reads out the text we have predicted.

Project Objectives

• To implement Air Writing Interface
• To develop accurate finger tracking
• It can be a vital component in improving the user experiences
•  Implements interface which requires minimum attention focus

Project Implementation Method

Our software uses module of Opencv , Keras, TensorFlow, MediaPipe, Pandas and Numpy
We access webcam using opencv and get hands and finger detected via mediapipe and get position of 
tip of index finger. Then we create writing event using cv2.line command
After the writing event we decide which conditions bring a break in writing event. So if hand is not in 
boundaries of screen or fingertip is not shown then break comes. Then we decide how pressing key 
helps us to take frame in order to record the writing event in form of image and converted into image 
format saved into folder. 
Model is trained using keras and tensorflow and using dataset allows us to train model to predict or 
classify labels of images for alphanumeric recognition.

Even OCR technique is applied for word recognition if writing is cursive. It uses Nano Nets model.
These frames are passed into model to get predicted text. Later we focus on Word recogniton and this 
predicted text is later shown on screen. This text is passed through speech recognition. Speech Recognition process converts the test into audio so that user gets to hear what he/she writes in air.

This tool is accessed through software- website which is built using HTML, CSS and Django.

Benefits of the Project

Air-writing is the process of writing characters or words in free space using finger or hand movements 
without the aid of any hand-held device. Compared to other non-traditional input methods such as 
typing with a virtual keyboard or similar schemes, air writing offers the advantage of “eye-free” 
execution, requiring minimum attention focus.
Air-writing digit recognition empowers users to communicate with the machine and interact naturally 
with no mechanical gadgets.

Technical Details of Final Deliverable

• ?We will have an interface where user gives an input by making finger movements in air which is known as “Air – Writing” and by choosing suitable models and algorithms our interface will be able to predict the air written characters.
• Software provides writing characters in free space by making hand or finger movements without the aid of any hand-held device.
• By using MediaPipe we detect and track finger movement
• By using OpenCV webcam is accessed which allows handtracking module
• Writing event happens
• Air-written words recognized by OCR and ML technique.
• The text predicted is shown on screen and converted into audio using 'pyttsx3'
• This tool is provided through website built using HTML, CSS,Django

Final Deliverable of the Project Software SystemCore Industry ITOther IndustriesCore Technology Artificial Intelligence(AI)Other TechnologiesSustainable Development Goals Quality EducationRequired Resources

Item Name	Type	No. of Units	Per Unit Cost (in Rs)	Total (in Rs)
			Total in (Rs)	5000
General	Miscellaneous	1	5000	5000