RoboEye

Face recognition has been widely a hot topic for academicians and researchers owing to its numerous applications in sophisticated intelligent systems. But existing face recognition algorithms are prone to be attacked by various face presentation attacks (face-PAs) when it comes to performing in unco

Project Title

RoboEye

Project Area of Specialization

Artificial Intelligence

Project Summary

Face recognition has been widely a hot topic for academicians and researchers owing to its numerous applications in sophisticated intelligent systems. But existing face recognition algorithms are prone to be attacked by various face presentation attacks (face-PAs) when it comes to performing in uncontrolled environments. So, despite the advent of Artificial Intelligence-based robust surveillance systems, the reliance on manual observations is still a practice. 

Apart from these aforementioned factors, even coming up with a well-optimized algorithm, giving us decent results, the next hurdle is in its run-time application. Though GPU has accelerated the research in the realm of Computer Vision but its cost is hampering its ample usage. Thus, there is a need for a cost-efficient customized tool that could be presented as a plug-and-play solution to the users without any additional circuitry. And can be employed for the implementation of a wide range of computer vision-based problems.

To optimally handle the aforementioned problems, we formulate a novel deep architecture to increase the accuracy of multi-view human face recognition. In particular, we propose “RoboNet”, a Deep Neural Network Architecture for facial recognition and identification on a Vision Processing Unit (VPU) in an uncontrolled open environment. Thus, solving the major issue mentioned above. The attempt is to gauge the efficiency of existing facial recognition algorithms and utilize their results to frame a better architecture for the neural network.

The core objective lies in building a robust and efficient face recognition & Identification system that detects faces rapidly in cluttered backgrounds and in video data where subjects (people in the video) are not static. Moreover, the notion of employing the state of the art, facial recognition & identification algorithms to give their respective outputs, which would be contributing according to some pre-defined percentage, and ultimately, giving the output would be highly useful, in eliminating the likelihood of the wrong prediction.

Furthermore, this project presents the implementation of an intelligent & mobile, Face Recognition based aerial surveillance system, using state of art facial recognition algorithms on VPU, that can be extended to a full-fledged aerial warfare tool. The proposed portable system tracks the subject or the suspect with the camera marks his location together with the timestamp and logs his presence in the database, using camera installation on the drone.

Project Objectives

Research Objectives: 

1. To research about existing facial recognition techniques
2. To identify the loopholes in existing methodologies and design such an architecture that would give better results in unconstrained open environments
3. To look for lesser data-hungry techniques in AI (curtailing the issues of data availability in AI and deep learning particularly)
4. To publish a research paper based on the outcomes of the above-defined techniques

Academic Objectives:  

1. To develop a redundant and error-free technique in the domain of facial recognition and identification
2. To design a robust neural architecture that will use fewer datasets and give better results

Commercial Objectives:  

1. To deliver the end product(warfare tool) to armed forces institutes and amplify their defense capability by including the state of art technologies in their warfare mechanisms
2. To deliver the surveillance tool, to the banking & finance sector and different  border and security-sensitive areas

Project Implementation Method

Initially, research about the state of art algorithms employed for facial recognition and identification purposes will be done. Having done that, a custom neural network termed “RoboNet” will be designed and developed , to serve the purpose but the workings of the complete system will also comprise getting individual outputs of pre-built models and taking an average of all.

For Robonet we ar using Tina face as the base model for face detection and QMagFace for the face recognition and verification purposes. These two algorithms have given the best results so far so we have used these to do some experimentation and come up with a more precise algorithm.

Redundant system or making use of more than one technique is done to ensure the eradication of any chance of error in the wrong prediction. Alongside that, RoboNet will also be giving its output, and based on a collective result it will be declared if the right person has been found or not.

Here is a broadly defined modular approach for the project:

Modules of the project:

Module 1: Research about existing, state of art solutions for facial recognition & identification

Module 2: Building custom neural network architecture and its fine-tuning

Module 3: Experimentation with various parameter spaces and data augmentations

Module 4: Implementation on edge device (VPU)

Module 5: Deployment on drone

Benefits of the Project

1. A full fledge aerial surveillance tool that could be extended to cater to the warfare regime if required weaponry is added
2. A plug and play solution which would use minimum circuitry for the deployment of a complete system on a cost-efficient edge computing device (VPU)
3. As per the surveillance tool use case, monitoring and tracking of suspects will be getting manifested via a web-based UI (Tkinter )
4. An array of facial recognition and identification algorithms from which a wide range of use cases could be derived depending upon the need and data availability
5. A research paper will be published based on the research outcomes which come under one module of this project
6. A cost-efficient edge-computing device is used, which ensures data security, fast processing and curtails latency issues
7. Additional circuitry is not required, only VPU and the attached camera are enough for the deployment of the complete system
8. Usable in unconstrained environments thus enhances surveillance system applicability
9. Works well on real-time video data to perform facial recognition and ultimately identification
10. Using multiple algorithms reduces the chance of wrong identification & ensures the utmost reliance
11. Potential customers would be defense forces institutes and law enforcement agencies for the warfare tool, banks and other security-sensitive organizations for the surveillance tool, and schools, colleges, and universities for AI-based attendance taking system. Depending upon requirement several other use cases can be derived from the robust facial recognition system which would act as a rudiment for all of the above-defined applications

Technical Details of Final Deliverable

If we talk in accordance with the modular approach for completion of RoboEye firstly it is about the research of existing, state of art solutions for facial recognition & identification.

Here we have used FaceNet, VggFace, Azure Face API, Face Recognition package, and an API of Google which has been provided as a free utility named Google teachable machines. 

If we talk of datasets we have used WiderFace and Celeb datasets for these models.  

The above algorithms or techniques are giving individual results and we have cumulated the in such a way that the final output will be an average of individual results given by all of these.

The data set has been broken down into 70-30 ratios. Where 70% of data has been used for training purposes 10% has been used as a validation dataset to prevent issues like overfitting and the remaining 20% comprises the test data set. That's the dataset distribution for testing, training, and validation purposes.

Moreover, as we are using pre-processed datasets these don't require any prior pre-processing. We can use them as they are. One thing more that these algorithms have been trained on millions of images so if we try to make them learn the facial encodings of any new person there won't be any issue and with minimal data, the model will be able to detect and recognize the face in both static and dynamic environments.    

After using a multitude of state of art facial detection and recognition algorithms the next step is of building a custom neural network. That's where the research part comes in. Here we have used the TINAFACE algorithm as a base algorithm for face detection purposes. It uses Wider  Face(hard) data set and with a  single-model and single-scale, our TinaFace achieves 92.1% average precision (AP).  With test time augmentation (TTA), our TinaFace outperforms the current state-of-the-art method and achieves 92.4% AP.

Next for face recognition and verification, we have used QMagFace. 

QMagFace approach performs especially well under challenging circumstances, such as cross-pose, cross-age, or cross-quality. Consequently, it leads to state-of-the-art performances on several face recognition benchmarks, such as 98.50% on AgeDB, 83.95% on XQLFQ, and 98.74% on CFP-FP.

One thing to be highlighted here is that our research module and still undergoing and is in experimental stages  with various parameter spaces and data augmentations

Next comes the implementation part here the above-defined techniques are deployed on an edge device. we are using a vision processing system for it. We are using UP AI Core XM 2280 (Myriad X *2). 

VPU will be attached to an aerial device which is a quadcopter in this case. We are using Modern X12 Drone.

It caters to the needs of this project giving better pictorial input to the AI components(face detection and recognition models) on run time and dynamic environments.

Final Deliverable of the Project

HW/SW integrated system

Core Industry

Security

Other Industries

IT

Core Technology

Artificial Intelligence(AI)

Other Technologies

Robotics

Sustainable Development Goals

Industry, Innovation and Infrastructure, Partnerships to achieve the Goal

Required Resources

If you need this project, please contact me on contact@adikhanofficial.com