EmotiBot: Human Emotions Recognizer Agent with Prototype Hardware

Project Title

EmotiBot: Human Emotions Recognizer Agent with Prototype Hardware

Project Area of Specialization Artificial IntelligenceProject Summary

Recognizing human expressions and emotions has drawn the attention of researchers, as the capability of recognizing one’s expressions helps in human-computer interaction. It also helps in the right advertising campaigns and crowned with an augmented and enhanced human communication. Recent discoveries in neurosciences, together with the extension of notions like emotional intelligence and multilevel intelligence, has led to the emergence of the new framework “Affective Computing”, according to which, the main aim is to build machines that recognize, express, model, communicate and respond to user’s emotion indicators. In the new framework, emotions hold a key role in machines which could impact positively their future decisions, bringing closer to taking part in a more sociable loop of human-machine interaction. Our project implementation will be based upon the connection between Artificially Intelligent Agents and humans that will involve an emotional feedback framework, in which Artificially Intelligent Agents can understand emotions from some cues from human speech and facial expressions.

Project Objectives

Our primary goal is to design an artificial intelligence program that will have the functionality to recognize human emotions through speech and facial expressions. Our secondary goal is to design a hardware architecture that will be able to provide all the above-described functionalities and provide the desired outcome. To achieve these goals, We must:

• Do research on the implementation of connection between Artificially Intelligent Agents and humans that will involve an emotional feedback framework.
• Explore ways to get appropriate response, broaden the horizon by capturing numerous natural responses instead of just basic ones and work to make this system robust and accurate.
• Familiarize with Affective Computing and Detection of Emotional Information
• Understand the concepts of machine learning, deep learning, Image processing, and its applicability in affective computing.
• Know about Embedded systems & hardware design
• Try to go extra mile by enlarging capabilities of our system through capturing psychological emotions/assessments and brain signals.

Project Implementation Method

The user’s speech and his facial features are taken as an input. These features are then sent to the backend server where deep learning, NLP and image processing algorithms determine the emotions of the user. Appropriate response based on his/her emotions, for example if the user is angry the response could be something like “What brings us to tears, will lead us to grace. Our pain is never wasted”. This response is then provided as speech by our agent.

Benefits of the Project

• It is a sociable agent having the ability to engage in psychological interactions with user using Human Computer Interaction (HCI).
• It especially focusses on Robot Assisted Therapy (RAT) by identifying the user’s current cognitive state and responding accordingly.
• It will serve as an assisting tool for the children suffering with autism to better their social and emotional abilities by recognizing situation-based emotions.
• This agent helps in uplifting user’s mood and provides companionship for children/users suffering from Alzheimer’s disease.
• This kind of agent contributes to the future market for social artificially intelligent agents by further enhancing interactions with the user.
• The proposed agent could be used in an organization to test the Emotional Quotient (EQ) of an employee.

Technical Details of Final Deliverable

Software:

On the software side our project consists of multiple deep learning models trained for recognzing human emotions through speech, text and facial features and another deep learning model that responds to the user based on his/her emotions and current cognitive state.

Hardware:

On the hardware side our project consists of a small robotic prototype (much like in the movie Wall - E) which listens to a user's speech, captures his/her facial features and responds to the user via a speaker and displays facial features on a screen. 

Final Deliverable of the Project HW/SW integrated systemCore Industry ITOther IndustriesCore Technology Artificial Intelligence(AI)Other TechnologiesSustainable Development Goals Good Health and Well-Being for PeopleRequired Resources

Item Name	Type	No. of Units	Per Unit Cost (in Rs)	Total (in Rs)
			Total in (Rs)	43000
Raspberry Pi 4	Equipment	1	15000	15000
7 inch screen	Equipment	1	9000	9000
USB Microphone	Equipment	1	9000	9000
Speaker	Equipment	1	5000	5000
Camera	Equipment	1	5000	5000