An automated personality prediction by video processing using deep learning
Personality has a strong influence on people?s lives, behaviors, and decisions. So personality prediction has been an active field of research for a long time. However, due to the increase in computation power, there has been a growing interest in recognizing the human personality and integrating th
2025-06-28 16:25:05 - Adil Khan
An automated personality prediction by video processing using deep learning
Project Area of Specialization Artificial IntelligenceProject SummaryPersonality has a strong influence on people’s lives, behaviors, and decisions. So personality prediction has been an active field of research for a long time. However, due to the increase in computation power, there has been a growing interest in recognizing the human personality and integrating them into computing, also known as “personality computing.”
Previously Predicting personality using traditional questionnaires was too time-consuming with a human error that cannot be disregarded and may produce biased decisions that may result in unreliable solutions.
The existing ML solutions are based on single features, which is inefficient or predicts higher accuracy results. So our system will use bimodal fusion to predict personality which is more accurate in comparison to previously proposed systems. The most outstanding thing about our project is that we will be generating our own dataset for Pakistani demography. This act allows us to achieve highly accurate results at a local level.
In our project, we are developing a hybrid mobile application that will run on both android and iOS. We are using Flutter to build an app. Our mobile application allows us to upload a short video to carry out predictions.
We are using deep learning algorithms such as convolutional neural network (CNN) and long short term memory (LSTM) to train our model. Our trained model will be connected to our app using Flask API.
Project ObjectivesOur objective for this FYP is that an automated personality analysis can be performed using a system by simply processing a video of 10 seconds. Moreover, it will include the development of a cross-platform application that detects the person's personality using video processing. Also, we will define and train facial, audio, ambiance, and transcript features models. To see the analysis we will combine results from all the above models using fusion. One of our most important objectives that are not achieved before is to create and upload a local dataset consisting of video interviews. We will gather data from our local hospitals and medical centers which will be labeled by a physiatrist who will be on board with us as we want to achieve accurate analysis which can be used in industry and other sectors of work. This will lead to the screening of candidates that a company is not looking for with some specific traits. One of our greatest objectives to achieve is that we will try to alter the phycology test that is conducted in our armed forces by trained professionals which consumes a lot of costs, time, and effort. This project will provide screening of non-eligible candidates that the armed forces are not looking for
Project Implementation MethodOur project is a combination of two sciences i.e.
Psychology (the science of mind and behavior), and computer science.
Our project implementation is also based on two main models. The first model is the Rule-based model in which we have defined personality traits parallel to certain facial features and audios features.
The second model is the model in which automated personality analysis will be performed. Video analysis consists of a few steps. Firstly, some image processing techniques are applied to the sample video provided by the user to extract facial features. The next step is to extract audio features. After extracting features through segmentation, the classification is performed using deep learning models. In order to improve accuracy, the repetitiveness of traits is considered and the dominant trait is fetched.
The benefits of this project are in many sectors of work including human resources, political movements, media recommendations, forensics, health care, counseling, and many more as per user and with modifications.
One of the benefits is enhanced, personal assistants. Present-day automated voice assistants such as Siri, Google Assistant, Alexa, etc. can be made to automatically detect the personality of the user and, hence, give customized responses.
Other benefits include:
- Recommendation systems People that share a particular personality type may have similar interests and hobbies. The products and services that are recommended to a person should be those that have been positively evaluated by other users with a similar personality type.
- Word polarity detection Personality detection is a subtask of sentiment analysis as it can be exploited for word polarity disambiguation in sentiment lexica (where the same concept can convey different emotions to different types of people) and for disambiguation between sarcastic and non-sarcastic content.
- Specialized health care and counseling. This is yet another area where huge practical applications of personality trait prediction exist. According to an individual’s personality, appropriate automated counseling may be given or a psychiatrist may make use of this information to give better counseling advice.
- Forensics If the police are aware of the personality traits of the people who were present at the crime scene, it may help in reducing the circle of suspects. Personality detection also helps in the field of automated deception detection and can help in building lie detectors with higher accuracy.
- Job screening In human resource management, personality traits affect one’s suitability for certain jobs. For example, a company may need to recruit someone who will motivate and lead a particular team. They can narrow down their screening by eliminating candidates who are highly nervous and sensitive, i.e., those having high values of neuroticism traits. Or they can screen the candidates with those traits the company is not looking for.
- Political forecasting Large-scale automated personality detection is being used as a guideline for politicians to come up with more effective and targeted campaigns. If an analytical firm is able to procure large-scale behavioral data about the voters, the firm can then create their psych graphical profiles. These profiles can give an insight into the kind of advertisement that would be most effective in persuading a particular person at a particular location for some political event.
These are the benefits that can be seen directly but there are many more which can be utilized by modifying the analysis and using it in a different way.
Technical Details of Final Deliverable
We are developing a deep learning-based system that will predict a person's personality without human interference for that purpose we will train a model on audio and visual features and in the end combine them to get the personality of a person based on OCEAN. For audio, we will use LSTM and for visual features, we will use CNN as our model. After training our model we will deploy it on a server using flask (python micro web framework).and then design an app using flutter for accessing our model through it and getting our predictions.
Our APP will have three screens the first screen will be a splash Screen (usually represents the brand/company name). the second screen will be a Dashboard screen in which we have the option to move to other screens that will be Predict from the gallery, Record now, About Us, and user manual.
Our system will be only for the English language only initially. And require high-speed internet with a good camera on a user device to work efficiently. And generally, it will be compatible with devices with an android version greater than seven and iOS six.
| Item Name | Type | No. of Units | Per Unit Cost (in Rs) | Total (in Rs) |
|---|---|---|---|---|
| Total in (Rs) | 77400 | |||
| Nividia GeForce GPU | Equipment | 1 | 40000 | 40000 |
| Smart Phone | Equipment | 1 | 20000 | 20000 |
| IP Camera | Equipment | 2 | 3700 | 7400 |
| Transport, Delivery and fuel | Miscellaneous | 1 | 5000 | 5000 |
| App Charges | Miscellaneous | 1 | 5000 | 5000 |