PosGestix Recognizer

Project Title

PosGestix Recognizer

Project Area of Specialization Artificial IntelligenceProject Summary

Multi-sensory loss impairment (deafblindness) and Musculoskeletal disorder is increasing in humans due to genetic conditions and aging which is a great concern for future world. Patients suffering from these conditions, try to put over their message with hand gestures and poses.  Many of these deficiencies may be overcome by introducing a more natural human computer interaction, especially speech and gesture. In order to help these patients, so that they can live a better life, we have proposed a system which can monitor human gestures and poses, which helps these patients so that they can communicate in an effective way. We have used depth camera to detect different joint points of human body in real time and those joint points will be matched with gestures stored in database to find that what actually he or she is trying to say. Our proposed system can successfully recognize different gestures and poses in real time.

Project Objectives

• The main objective of this project is to help disable people to communicate effectively with other humans and machines as well and live their lives in easiest way.
• Aid to physically challenged people. In machine wheel chairs the movement of hand will act as a controller of speed as well as direction.
• Those people doesn't need to move their hand over keyboard or on mouse. They just need to perform a gesture and the task will perform.
• The pose and gesture recognition system can transcribe the symbols represented through poses and gestures into useful task.
• Through the use of gesture and pose recognition the remote control with the wave of hand of various devices is possible.
• For systems where the act of finding or acquiring a physical controller could require too much time, gestures and pose can be used as an alternative control mechanisms.

• To develop a multimodal interface to manipulate the low- and high level aspects of 3D hierarchical digital models so that the hands and the face of possibly separate performers will be tracked, their gestures and facial expressions will be recognized and estimated features will be mapped to digital puppets in real-time.

• The project involves tracking of hands, background segmentation, skin color filtering, particle filtering, hand pose estimation, gesture and expression and keyframe classification,

Project Implementation Method

Gestures and poses are an important means of communicating in our day-to-day life. Often we communicate by the movement of body parts like hands and head rather than speaking. So for a successful machine-human interaction consideration of these gestures is inevitable. The proposed method can recognize human gestures using a depth camera. The user just need to connect camera with laptop and open the human pose and gesture recognition application and press the start button. The camera views the subject in the front plane and generates a depth image of the subject in the plane towards the camera. This depth image is then used for background removal, followed by generation of the depth profile of the subject. In addition to this, the difference between subsequent frames gives the motion profile of the subject and is used for recognition of gestures. These allow the efficient use of depth camera to successfully recognize multiple human gestures. 

Benefits of the Project

1. Human poses and gestures recognition system serves as a key for overcoming many difficulties and providing convenience for human life, especially for patients for 24/7 monitoring.
2. Through our proposed system patients can effectively communicate with machine as well as other humans. Also, they can navigate without any human assistance which prevents them from many psychological issues like depression, anxiety and anger.
3. The body motion recognition technology has been used for physical therapy and rehabilitation exercises, monitoring patients’ activities, and many other ?elds.
4. Not only patients, normal human-beings can also get benefits through the proposed system. For instance, they can use human pose and gestures recognition techniques in virtual reality applications and augmented reality games, to assist robots, and to interact with their devices in a touchless environment just by their gestures and poses.
5. This system makes computers interact with human in more practical, natural, and responsive, as if an interaction is between humans.
6. Learning-based approach stored features that can be used later for searching matched objects in the testing environment.
7. This proposed method is highly efficient and fit for real-time applications. For instance, this system is able to monitor patient for fall prevention. Also, this system can be used to monitor COVID-19 patients by maintaining social distancing.

Technical Details of Final Deliverable

• The proposed system is capable of identifying human gestures and poses in real environment and interpreting them into useful commands.
• Gesture Memory (database) is built based on identi?ed gestures from a human study which are capable of creating every essential task.
• Moreover, user’s capability to remember and recall gestures is also evaluated.
• Gesture and pose recognition unit extracts the information of hand skeleton using a Microsoft Kinect sensor, and extracted data is sent to the Gesture Clari?er for clari?cation of gestures based on gesture features.
• Gesture identi?cation unit understand and identify the command related to the observed gesture and pose.
• The main focus of this project is to monitor patient for 24/7 in order to avoid any accident that leads to serious mishaps.
• From this, we can also make more beneficial applications in future for instance a system that understand the needs of patients and fulfill their needs by itself without human assistance.

Final Deliverable of the Project HW/SW integrated systemCore Industry ITOther Industries Medical , Health , Security Core Technology Internet of Things (IoT)Other Technologies Artificial Intelligence(AI)Sustainable Development Goals Good Health and Well-Being for PeopleRequired Resources

Item Name	Type	No. of Units	Per Unit Cost (in Rs)	Total (in Rs)
			Total in (Rs)	79534
Logitech Desktop Web Camera	Equipment	1	50534	50534
Raspberry Pi 4	Equipment	1	19200	19200
Internet	Miscellaneous	1	4000	4000
Printing	Miscellaneous	5	1000	5000
Stationary	Miscellaneous	8	100	800