Hand Gesture Assessment System based on Deep Learning within a context of Punjab Police Investigation

In this modern era, scientists are trying to improve Human-Computer-Interaction (HCI). Gesture recognition system is helpful in this aspect. Gestures include facial expressions, body posture position and hand movements. Facial expressions field was the first interesting topic but HCI using hand gest

Project Title

Hand Gesture Assessment System based on Deep Learning within a context of Punjab Police Investigation

Project Area of Specialization

Artificial Intelligence

Project Summary

In this modern era, scientists are trying to improve Human-Computer-Interaction (HCI). Gesture recognition system is helpful in this aspect. Gestures include facial expressions, body posture position and hand movements. Facial expressions field was the first interesting topic but HCI using hand gestures is the hot topic nowadays. Hand gestures play a vital role in communication. These gestures are unintentional and depends upon the situation occurring. These gestures vary from situation to situation. For example, if hands are clenched and raised in upper position then it means “Frustration” even if the person is smiling. Similarly, palms facing up is a sign of cooperative behavior and if facing down it is a gesture of confidence.

Previous cited works are related to only either sign language identification or facial expressions recognition. In our project, we are solely focusing on hands gestures to identify the mental state of a person.

Our main purpose is to identify these gestures by computer vision. The dataset is composed of frames extracted from videos collected from open sources such as YouTube. These frames are annotated with their desired labels or their ground truth values. In the end, model will be able to classify these gestures of a person.

The applications of this project lie in legal department i.e. investigating a crime scene.

Project Objectives

The objective of a project is to develop such system which can identify different gestures to conclude the mental state of a person at a specific time interval.  Gestures are unintentional and no one can control them, thus we are developing a system to identify them. This can help in human computer interaction (HCI), crime investigations etc.

• Industrial objective is to develop a system which detects mental stated based on their hand gestures.
• Since no other system has been yet developed which identify mental state based on hand gestures. Most of work being done is either on sign language or on facial expressions, so our research objective is to work on different mental states depicted by hand gestures.

Academic objective is to learn different core deep learning concepts and implements them on real life problems.

Project Implementation Method

Our main purpose is to identify hand gestures. Our approach is with deep learning method. Specifically, with image recognition by applying convolutions and getting the output. The following is the process we are following:

• Data collection
• Labeling
• Feature engineering
• Model training
• Evaluation
• Deployment

Benefits of the Project

The benefit of the project is to develop such systems which can identify mental states on basis of their hand gestures. The research objective is to open a new research are for hand gesture recognition for mental state identification.

Technical Details of Final Deliverable

The final deliverable is a deep learning model which will be able to assess different human gestures. The model will be able to generalize different classes of these mental states. The input will be an image of the gesture passed into model after some feature engineering i.e. reading image, converting it to array. Then it will go through model where the class of image will be predicted. After training, and fine tuning of the model it will then be used to detect gestures by using CCTV cameras deployed in investigation rooms and during interviews.

Final Deliverable of the Project

HW/SW integrated system

Core Industry

Legal

Other Industries

Medical , Media , Security

Core Technology

Artificial Intelligence(AI)

Other Technologies

Sustainable Development Goals

Industry, Innovation and Infrastructure, Peace and Justice Strong Institutions

Required Resources

If you need this project, please contact me on contact@adikhanofficial.com