Personality Prediction Using Video Processing

Project Title

Personality Prediction Using Video Processing

Project Area of Specialization Artificial IntelligenceProject Summary

Predicting personality using traditional questionnaires was too time consuming with an human error that cannot be disregarded and may produce biased decisions that may result in unreliable solutions.

We are proposing a machine learning solution, which will predict personality traits using video processing. We are using facial features, ambiance, audio and transcript to predict with higher accuracy and fast results.

Project Objectives

1. To develop ML solutions which will be suitable for different personality problems.
2. To achieve maximum accuracy than existing proposed models.
3. To improve the performance of the prediction.
4. To design and develop an platform (Application) to make it affective and useful.
5. A system that automates decision making in the domain of personality prediction like HR interview.
6. The scope of our project is to make a cross-platform application that detects the personality of the person using video processing.
7.  Define and train models for facial, audio, ambience and transcript features.
8. Combine results from all above models using fusion.
9. Design a cross-platform application on which users will upload a video to be predicted.

Project Implementation Method Technical Approach

We are using 4 different features to predict personality.

1. Facial

2. Ambiance

3. Audio

4. Transcript

After working on all these four models and combining them together (fusion) we were facing issues like overfitting and  very long training and evaluation time, which makes it impractical.

So we decided to fused facial and ambiance models, as having one model for ambiance and facial features we have to only process frames containing faces and surrounding both which lead to less parameter tuning.

Also in case of audio and transcript we fused them together to make a single model for better results.

Model Defining and Training

1. Till now we have trained our dataset on multiple CNN models i.e. Resnet v2 101, VGG16, we are getting low accuracies so we are fine tuning pre-trained VGG-Face Model with some architecture changes to improve training time and higher accuracies.

2. The deep descriptors of the last convolutional layers are aggregated as a single visual feature which leads to less no of parameters to be evaluated.

Frontend Design

1. We have also started working on front-end cross platform application development on React Native framework.

2. On this application we have designed 3 pages the home page is for uploading video and getting results.

3. The second page is About us page and the third one is our proposed model description.

In last we have to deploy our save network on Tensorflow serving and create an flask API to comunicate with react native application.

Benefits of the Project

The emergence of connected AI is expected to enable ML algorithms to learn continuously based on newly available information. Such developments are anticipated to drive the market in the coming years. Personality computing benefits from methods aimed towards understanding, predicting, and synthesizing human behavior. Automated recognition of apparent personality is a part of many applications such as human-computer interaction,computer-based learning, automatic job interviews and crowd simulations.

Technical Details of Final Deliverable Technology Stack:

Front-End : React.js and HTML 

Back-End  : Python ( Flask, Tensorflow and Keras )  

Server       : Tensorflow serving

Final Deliverable of the Project Software SystemCore Industry ITOther IndustriesCore Technology Artificial Intelligence(AI)Other Technologies Cloud InfrastructureSustainable Development Goals Gender Equality, Decent Work and Economic Growth, Industry, Innovation and Infrastructure, Reduced InequalityRequired Resources

Item Name	Type	No. of Units	Per Unit Cost (in Rs)	Total (in Rs)
			Total in (Rs)	76000
GPU 1080 ti	Equipment	1	30000	30000
intel Core i7-8700	Equipment	1	20000	20000
ASRock H310CM-HDV/M.2	Equipment	1	17000	17000
Printing Expenses	Miscellaneous	1	3000	3000
Overheads	Miscellaneous	1	6000	6000