Recognition of Human Emotions based on Skin Response

Project Title

Recognition of Human Emotions based on Skin Response

Project Area of Specialization Artificial IntelligenceProject Summary

A communication gap barrier exists between the normal and paralyzed people. In order to address this communication barrier, a novel approach to recognize human emotions of handicapped people is presented in our work i.e. the Galvanic skin response for which we acquire the signals thus detecting the important emotions of patients. Since skin conductance and resistance vary with the emotional state. The phenomenon to translate emotions through GSR values using techniques like SVM would be applied to monitor the emotions.

The device is able to detect the human emotions like of those people who are not able to communicate, have a speech disorder apraxia caused due to damaging speech-generating neurons in brain and dysarthria caused due to the weakness of speech-generating muscles.

The variations of emotions are caused due to variations in different human physical, psychological, physiological changes.

Emotions are recognized by physical activities, for example, facial expressions, body language, and different physical activities but paralyzed people or people who are not able to express their emotions, describe their feelings and are not able to communicate using physical means thus  having a  communication barrier. 

To remove this barrier, we are making a device which detects the emotions of completely paralyzed people or people who are suffering from inability of communication  due to the variation of emotions caused as a result of different activities in a human body like brain activity, muscle activity, respiratory system, body temperature, blood pressure and activity of sweat glands. 

So, we are translating human emotions while using the activityof  sweat glands in the skin.

 IoT in this setup would make it a smarter device to coordinate amongst the network at different stages.

Project Objectives

The overview of this project is to make a cost-effective and refine a solution for emotion recognition using the galvanic skin response of people and patients. The goals of this project are recognition of strong emotions (i.e. happy, angry, excited and tired or sleepy) form mapping scheme given one positive, negative, active and passive emotions from the emotion space model shown in the figure.

Figure  Emotion space model

Figure  Emotion space model

Project Implementation Method

Emotions play a very important role in human life, which affect the physiological and psychological changes. The block diagram is shown in the figure.

Figure  Block Diagram

In general, emotion recognition methods could be classi?ed into two major categories. One is using human physical signals such as facial expression, speech, gesture, posture, etc., However, the reliability can’t be guaranteed. The other category is using the physiological signals which include the galvanic skin response (GSR) for the detection of emotions. Physiological changes that occur due to variations of emotions in the human body shown in figure.

Figure Physiological Signals

We will implement our project through Specialized teams and Doctors who deal with paralyzed and disabled patients to recommend our product, pharmacies, health care departments, and NGOs.

The major target market for our business plan is for every kind of class except lower middle class and lower class as this device is not too expensive but we are trying to give them our product through NGOs and different health care commissions. They can be of any age, religion, and place. The program will be executed in phases 1 in the areas with Doctors, pharmacies and NGOs. To check the sales and test the market. Then the 2nd phase will be initiated which is for the Hospitals and middle-class people. In the market, we are introducing a blend of new techniques and technology. The main goal is to provide convenience and comfort to our customers, by making their lives easy.

Benefits of the Project

There are a lot of benefits of this project like there is no solution for the people who are not able to communicate and there is huge market gap regarding to this problem and this device detects the strong and important emotions of patients for example being hungry, happy or sad, in state of nature’s call, hungry,  encountering his blood relatives like mother, sister, or children and thus identifying the emotional state of a patient. It removes the communication barrier of disabled patients. It adds mobility and simplicity value for the customers. This product is unique because the phenomenon is to translate emotions in an electrodermal activity of skin cells.

Technical Details of Final Deliverable

Technical details of our project include the following things:

We are using the machine learning technique of artificial intelligence.

Node MCU ESP8266 Controller for implementation of IoT

GSR sensor having 2 electrodes which will be wearable to two fingers one is placed on the middle finger and another one is placed on index finger for the collection of real-time raw data of the user, this raw data is used to make a data set.

Initially, the prototype is developed for the collection of data which includes the SD card module to save the data.

Secondly, this data file uploaded to MATLAB for further processing in MATLAB. 

Step 1: Sampling and Noise Reduction

The signal is sampled at a particular sample rate using Nyquist Criteria and designed notch filter for noise reduction.

Step 2: Signal Analyzation 

Signal Analyzation is done by using a signal analyzer app of MATLAB and then the signal is analyzed in time and frequency domain and then draws its power spectrum for extraction of different features of the signal.

Step 3: Classification into Emotions: 

Classification is done by using classifier learner app of MATLAB on which variables and responses are selected in the first step and then extracted emotions related features and then by using different classification techniques e.g. Support Vector Machine, K Nearest Neighbour, Decision tree, etc. supervised and unsupervised learning. and drawn signals ROC and in the end, we will get classified emotions then this code is then converted into C code while using MATLAB coder App and then this file is then burnt into the controller.

The work flow diagram is as follow

The controller is Node MCU, another module is designed called base module which is for the person who is in the surrounding of the patient, will have an LCD on it which shows the emotional state of the patients.

The emotion of the patient will also show on the designed app on the mobile phone.

Final Deliverable of the Project HW/SW integrated systemCore Industry MedicalOther Industries Health Core Technology Artificial Intelligence(AI)Other Technologies Internet of Things (IoT), Wearables and ImplantablesSustainable Development Goals Good Health and Well-Being for People, Reduced Inequality, Life on LandRequired Resources

Elapsed time in (days or weeks or month or quarter) since start of the project	Milestone	Deliverable
Month 1	Proposal Submission	Successfully submitted and accepted.
Month 2	Literature Review	Successfully done with the literature review
Month 3	Simulation of Sample data	Simulate the sample data set by applying machine learning by using MATLAB tool.
Month 4	Methodology	Required Hardware components and methodology was made successfully
Month 5	Industrial Analysis	Industrial Analysis of the project was conducted, and collaboration with the industry successfully
Month 6	Prototype Development	Prototype developed successfully for the collection of the data.
Month 7	Data Collection	Data of the people and patients was collected to make the data set for the further processing.
Month 8	Simulation of Collected Data Set	Collected data set further processed while using MATLAB tools of: Signal Analyzer and Classifier Learner
Month 9	App Development	Development of app was done to show the emotional state of patients or people on app.
Month 10	Development of Algorithms Machine Learning	Algorithms was developed to compare with the live recording data.
Month 11	Final Hardware, Software, & App Testing	IoT implemented in project and final testing of hardware, software and app was done successfully
Month 12	Complete Hardware	Completion of the Hardware