Mask occluded face recognition using Generative Adversarial Network

Project Title

Mask occluded face recognition using Generative Adversarial Network

Project Area of Specialization Artificial IntelligenceProject Summary

Face recognition is one of the most convenient and fast techniques of identification. It has been widely used in different fields of criminal investigation, public security, etc. The face recognition technique extracts facial features for detection and recognition. It has been a trendy topic in recent years. Most people now started to wear masks in public as they want to protect themselves from pollution, germs or they want to hide their identity from the public. However, recognizing a masked face is a very challenging task due to the lack of facial features information available. The objective of our project is to recognize a masked face. For this problem first, we will detect the person in the live feed using some algorithm then, we will detect the mask occluded face, and finally, we will complete the image of the removed mask region using some technique. GANs are algorithmic architectures that are used widely in image generation, video generation, and voice generation. Learning images based on less appearance and structural variation is an easy task to be done. However, these methodologies do not fit for mask occluded face recognition due to the complex nature of an object, offering great occlusion i.e. mask, for example, Frequently people don’t uniformly wear a mask, Sometimes it covers only half area of the face and sometimes beyond the actual boundary of the face. In general, it offers great appearance variation. To overcome this problem, we proposed a GAN-based network algorithm that works on automatic removal of the mask from the face without disturbing the real structure of the face and process the affected part, resulting from the complete image of a face with more consistency and accuracy in comparison to the other algorithm. The other point that enhances the need to utilize the GAN is the current state of this art and its effectiveness in converting data into high-dimensional feature images which look naturalistic and consistent when compared with the other approaches that fail to give plausible results for images having great image variations. So basically GANs are used to generate the target data by latent variable and utilize their two neural networks Generator and Discriminator. . We developed a generator network by training a generator with mask and unmasked images then the output of a generator is feed into the discriminator. In face mask recognition we will use two discriminators i.e. local and global discriminators. The local discriminator works on the masked region and the global discriminator works on the entire face which results in the unmasked image which can be further improved by back propagation. So, for this problem, we will use a GAN-based network.

Project Objectives

1. We aim to develop a system that facilitates the early detection of criminals and eliminates the crime at an initial stage.

2. Training of the model by using computer vision and image processing.

3. Detect a person wearing a mask through a sensing device.

4. Synthesize the occluded face using the generative adversarial network.

5. To instruct our model to identify if the image or video of a dataset has the person in it or not.

6. To select the best facial feature generation technique which is fast, efficient, and accurate for the system in real-time.

7. To select the best model for the system.

Project Implementation Method

First, we will detect the people in the live feed using some algorithms then we will identify the occluded faces using face recognition algorithm for this we will train the system using masked and unmasked faces so it will be able to recognize masked faces. To recognize occluded faces, we will use a Generative adversarial network (GAN). To train the generator, both masked and unmasked images will be fed into the generator. The generator will generate an image using positive and negative images. The output of the generator and real data will be fed into the discriminator. The discriminator will decide whether the image is real or fake. If the image is real, it will be displayed on output but if the image is fake, a generator will again generate an image and this process will continue till we get our final image.

Alignment and segmentation of Dataset:

In this work we focus on the unmasking of masked faces, we detect the mask region then feed the input image and binary map of the detected mask region into GAN. We use GAN (Generative Adversarial Network) to remove an unwanted object (mask) from a large-scale dataset and fill this region with synthetic content (noise). Afterwards we work on alignment and masking of dataset as we divided this task into two categories.

1) Editing

2) Mapping

1.  Editing :This section consist of preparation of dataset which includes alignment and masking of dataset. We download a synthesized dataset of CelebA which is unaligned so we align the dataset in (512x512) images by using eye-coordinate of all images. Afterwards to fulfill the requirement of our project and to prepare dataset of our interest we masked the faces using dlib library.

2. Mapping: By using the mapping generator, first of all, we detect the non-face object (mask) and secondly we generate a binary segmentation map using an encoder or decoder network the encoder part of the generator consists of convolution layers. The decoder architecture is a mirror copy of encoder architecture except that convolution is replaced by deconvolution layer. This will gives image of clean mask.

a) Editing Generator :The editing generator then constructs new content of the given input image with the missing region along with the output of mapping generator and produce a generated image, Iedit.

b) Discriminators:  Local discriminator(Dlocal) that works on missing region and determines whether the synthesized content in missing image is real or not while global discriminator(Dglobal) work on the whole image ,it generates the face structure of covered mask.

Hardware:

In a hardware implementation, we use an image sensing device such as a camera to take an image as input and detect the person whether it’s wearing a mask or not this input passes through the microcontroller for image processing.We will use Jetson Nano as a microcontroller.

Benefits of the Project

Following are the sectors in which our project standout of great help.

1. In the education sector, it would assist to implement an attendance system.

2. In the industrial sector to plant security systems.

3. In government and private organizations to eliminate the alarming situation.

Technical Details of Final Deliverable

Our project "mask occluded face recognition using GAN" will provide a technique to extract facial features for the detection and recognition of masked faces. This project has influence over government and private organizations and it can be used for security purposes and to eliminate crime. This project allows government organizations and other sectors to enhance their security systems to prevent and reduce crimes that directly affect the performance and reputation of one organization.

Final Deliverable of the Project HW/SW integrated systemCore Industry SecurityOther IndustriesCore Technology Artificial Intelligence(AI)Other TechnologiesSustainable Development GoalsRequired Resources

Item Name	Type	No. of Units	Per Unit Cost (in Rs)	Total (in Rs)
			Total in (Rs)	62500
Jetson Nano	Equipment	1	35000	35000
Camera	Equipment	1	5500	5500
Display Screen	Equipment	1	10000	10000
SD card	Equipment	1	5000	5000
Ports	Equipment	2	1000	2000
Printing	Miscellaneous	1	5000	5000