Gaze Controlled Desktop

This project aims at proposing such a system that can replace Keyboard and Mouse and control desktop using only eye movement. This system will be useful for both physically disabled and able persons as it is focused on enhancing the human computer interaction which it will, in both the cases. Existi

Project Title

Gaze Controlled Desktop

Project Area of Specialization

Artificial Intelligence

Project Summary

This project aims at proposing such a system that can replace Keyboard and Mouse and control desktop using only eye movement. This system will be useful for both physically disabled and able persons as it is focused on enhancing the human computer interaction which it will, in both the cases. Existing systems that use eye gaze gestures for controlling desktop involves the use of hardware device (sensors and special cameras combined). We here propose a system that can control the desktop using eye gaze gestures just as the existing systems but by using only the webcam, thus making it more feasible and affordable.

Gaze Controlled Desktop is mostly focused on people who have a physical disability and cannot make use of their hands. They’ll be able to perform basic desktop control operations (like keyboard and mouse control) using only their eye movement.

Project Objectives

• Replace the traditional keyboard with an on-screen keyboard from which the user can type by fixating gaze on keys.

• Replace the traditional mouse by using gaze to hover the mouse cursor and performing mouse click events by eye blinking.

• Adjust brightness levels and sound volume from the onscreen menu. Navigate using the arrow keys from the menu. The menu will also contain icons for selecting keyboard and mouse.

• Turn screen on/off automatically by detecting the presence of the user.

• Perform additional functions like drag and drop, scrolling, zoom in, zoom out.

Project Implementation Method

The system will be developed using Incremental model. Every module will be designed, developed and tested hand in hand so that we achieve maximum efficiency and make the system possibly error free. Moreover, a backend module will be developed using OpenCv and CNN Face detection algorithms from Dlib combined with the Eye Gaze Datasets that will be trained using deep learning on Google Colab altogether implemented on Visual Studio. A webcam will be used for the input using Image processing. A desktop application will be developed as a user interface.

Python: For developing different modules

OpenCV: Human face libraries and algorithms used for face detection and then eye movement detection.

Deep Learning (Google Colab): Using python for training data sets.

Eye Gaze data sets: To develop a trained module that can detect gaze gestures.

Visual Studio: Used for developing the front end user interface and the back end combined with OpenCv and Dlib.

Dlib: CNN algorithm for Face detection and eye movement tracking.

Web Cam: Used as an input device using image processing.

Benefits of the Project

We ought to design a system that allows a person to use the desktop with the help of eye movement and move the cursor as per the direction that is given by the person. Aim of this application is to introduce a low cost based system for the people who have the physical disability and connect them with the real world.

In order to provide more convenient and natural communication mechanism for the user to interact with the computer, we propose a system that uses eye gaze gestures to control desktop and replaces the traditional keyboard and mouse thus providing a better Human Computer Interaction for both the disabled and the able people  without using any external hardware component. All that our system needs is a webcam which is already present in most modern laptops and monitors.

Technical Details of Final Deliverable

The following modules will be the main final deliverables of our project:

Keyboard Control: Replaces the traditional keyboard with an on-screen keyboard from which the user can type by fixating gaze on keys.

Mouse Control: Replaces the traditional mouse by using gaze to hover the mouse cursor and performing mouse click events by eye blinking.

On-screen Menu: Adjust brightness levels and sound volume from the on-screen menu. Navigate using the arrow keys from the menu. The menu will also contain icons for selecting keyboard and mouse.

Presence Detection: Turn screen on/off automatically by detecting the presence of the user.

Additional Functions: Perform additional functions like drag and drop, scrolling, zoom in, zoom out.

To summarize, the basic deliverables of the project would be:

• A desktop application for the interaction of the user.
• An on-screen menu displayed after the application has started.
• A backend module that uses trained datasets of eye gaze in combination with OpenCv, Human face libraries and certain algorithms for the detection of face and eye gaze

Final Deliverable of the Project

Software System

Core Industry

IT

Other Industries

Media , Health

Core Technology

Artificial Intelligence(AI)

Other Technologies

Sustainable Development Goals

Good Health and Well-Being for People

Required Resources

If you need this project, please contact me on contact@adikhanofficial.com