Extraction of human facial portrait from mind using EEG signal processing and deep learning

Electroencephalography is a method of monitoring electrical activity in the human brain. When we see something, our brain creates a mental perception, which is essentially a mental impression of that thing. We are able to capture this perception using EEG to get a direct illustration of what?s happe

Project Title

Extraction of human facial portrait from mind using EEG signal processing and deep learning

Project Area of Specialization

Electrical/Electronic Engineering

Project Summary

Electroencephalography is a method of monitoring electrical activity in the human brain. When we see something, our brain creates a mental perception, which is essentially a mental impression of that thing. We are able to capture this perception using EEG to get a direct illustration of what’s happening in the brain during this process. Deep learning-based analysis has enabled the visualization of perceptional content. Recent work showed that visual cortical activity can be decoded into hierarchical features of a deep neural network (DNN) for the same input image, providing a way to make use of the information from hierarchical visual features. We want to exract an image reconstruction method, in which the pixel values of an image will be optimized to make its DNN features similar to those which will be decoded from human brain activity at multiple layers. We expect that the classification of images resembles the stimulus images and the subjective visual content by developing the visual object classifier driven by human brain signals.

Project Objectives

the project aims to create the perception of  the human portrait . When a person see something, brain creates a mental percept, which is essentially a mental impression of that thing. This project will able to capture this percept using EEG to get a direct illustration of what's happening in the brain during image visualizing process. Our main objective is to create a system that will be used for the criminal investigation and for disable persons in future with few more improvements.

Project Implementation Method

First of all, we have collected the pictures that will be used for the training of the subjects. These are the pictures of four difffferent persons and afterwards their facial features will be extracted by using the EEG signals of our subjects. At the initial stage, we have to recorded the EEG signals from the Neurosky Mindwave Mobile 2 headset using eegID. The eegID is basically an andriod app provided by the neurosky store and build connection directly through Bluetooth to the headset, and view Electroencephalography (EEG) data, This data can be recorded and processed. At first stage, the mind reading phase can identify the 

two-dimensional(channels and time) EEG signals in order to learn the EEG feature representation

from the EEG signals that are recorded while subject looks at the images. Then from these EEG

signals an encoder network is designed through recurrent neural network(RNN) that will use to extract EEG features from raw EEG signals. LSTM recurrent neural network is employed for this purpose. The training process is supervised by the class of the images for which each input EEG sequences were recorded, and a classififier for EEG features is jointly trained in the process. 

The second stage of the method extracts the EEG features direclty from images by learning a mapping from CNN deep visual descriptors to EEG features. After that, new images can be classifified by simply estimating their EEG features through the trained CNN-based regressor and employ the stage-one classififier to predict the corresponding image class. The comparsion of both features will predict the subject's perception.

Benefits of the Project

It could provide a means of communication for people who are unable to verbally communicate i.e, the patients affected form head injury might not able to recognize his/her relative similar case with the coma patient and other memory loss patients. Not only could it produce a neural-based reconstruction of what a person is perceiving, but also of what they remember and imagine. Also the system will be used for the criminal investigation.

Technical Details of Final Deliverable

In this project the work relies on three key instincts:

• EEG signals recorded while a subject looks at an image convey feature-level and cognitive

level information about the image content, a qualitative difffference between EEG signals

evoked, on one subject, by visual stimuli of two difffferent object classes.

• A low-dimensional manifold within the multidimensional and temporally varying EEG

signals exists and can be extracted to obtain a 1D representation which refer to as EEG

features.

• EEG features are assumed to mainly encode visual data, thus it is possible to extract the

corresponding image descriptors for automated classifification.

First of all, we have collected the pictures that will be used for the training of the subjects. These are the pictures of four difffferent persons and afterwards their facial features will be extracted by using the EEG signals of our subjects. At the initial stage, we have to recorded the EEG signals from the Neurosky Mindwave Mobile 2 headset using eegID. The eegID is basically an andriod app provided by the neurosky store and build connection directly through Bluetooth to the headset, and view Electroencephalography (EEG) data, This data can be recorded and processed. At first stage, the mind reading phase can identify the 

two-dimensional(channels and time) EEG signals in order to learn the EEG feature representation

from the EEG signals that are recorded while subject looks at the images. Then from these EEG

signals an encoder network is designed through recurrent neural network(RNN) that will use to extract EEG features from raw EEG signals. LSTM recurrent neural network is employed for this purpose. The training process is supervised by the class of the images for which each input EEG sequences were recorded, and a classififier for EEG features is jointly trained in the process. 

The second stage of the method extracts the EEG features direclty from images by learning a mapping from CNN deep visual descriptors to EEG features. After that, new images can be classifified by simply estimating their EEG features through the trained CNN-based regressor and employ the stage-one classififier to predict the corresponding image class. The comparsion of both features will predict the subject's perception.

Final Deliverable of the Project

HW/SW integrated system

Core Industry

Security

Other Industries

IT

Core Technology

NeuroTech

Other Technologies

Artificial Intelligence(AI)

Sustainable Development Goals

Peace and Justice Strong Institutions

Required Resources

If you need this project, please contact me on contact@adikhanofficial.com