Multi Camera Vehicle Re Identification

Project Title

Multi Camera Vehicle Re Identification

Project Area of Specialization Artificial IntelligenceProject Summary

Re-identification is the process of associating images of the same object taken from different cameras or from the same camera at different points in time.

The need of surveillance is increasing due to higher crime rate. Law enforcement agencies are trying to build safe city projects with the help of surveillance cameras. These surveillance cameras are monitored manually to avoid any crimes. But this approach requires thousands of cameras to be monitored at a time by the humans and that is impossible. Therefore, there is need to automate the surveillance systems. We are exploring the computer vision techniques and building an algorithm that will be capable of vehicle re-identification. It determines whether the given vehicle image obtained from one camera has already appeared over a camera network or not. There are many possible practical applications where the vehicle Re-Id system can be employed, such as smart vehicle parking, suspicious vehicle tracking, vehicle incident detection, vehicle counting, and automatic toll collection.

Vehicle re-identification process is divided in to three phases. In the first phase, we detect vehicles from the images to localize the vehicle coordinates. After this step the localized vehicles will be cropped from the query images and will be resized to a standard size. Then these detected vehicles are further processed with the help of computer vision techniques to detect the color, make and type of the vehicle. These features of a vehicle are reliable and consistent. Other features are liable to changes and can easily be tampered such as license plate. They proved to be successful, but they are having certain requirements to perform well such as angle of elevation of camera. They are also inefficient in conditions where license plate is occluded or tampered. Therefore, there is need for such features that can work without any constraints. By the end of this step the output will be the description of vehicle on basis of its features which can help for re-identifying the vehicle.

In the second phase, we are going to assign id to the detected vehicle corresponding to its features which will be stored in a lookup table. The lookup table used for storing the id of vehicle corresponding to its features will be shared to the neighboring nodes in the network. The unique id will be assigned to a vehicle by a designated node only and all other noes will be propagating that id to its neighboring nodes i.e. the nodes which are along the path followed by the vehicle. When the similar features already available in lookup table, are detected again by the node in the network, the vehicle is re-identified otherwise new id is assigned to the detected vehicle if the that node is designated node. Third phase is related t path inferences.

Project Objectives

The objective of this project is to develop an intelligent system which can re-identify vehicles with a reasonable confidence level at the real-time. In this project we are going to devise a system which will convert the current scattered camera surveillance mechanism into a single autonomous system capable perceiving input from cameras all over the network and then on basis of query generated by the supervisor it will re-identify that vehicle in the live video feed obtained from the cameras spread all over the city. The proposed system will be capable of performing the operation of re-identification in the real-time.

Project Implementation Method System Basic Overview

The basic concept behind this re-identification system is to use the currently available surveillance system and link all cameras in such a way that all of the cameras act as a part of single system acting autonomously and processing the input queries without any human assistance in the process of re-identification. In this system all cameras(nodes) perceive the live input from its environment and will extract the desired features from the input video and then those features will be feed into a well-trained model that will classify the features and will reidentify the vehicle.

In order to reduce the communiction overhead and optimize the response time of model we'll shift the inference model to the edge and insted of transmitting the video feed to a server all nodes will extract visula features themself and will register them in a lookup table which will be shred among other nodes for reidentification purpose.

Technical Design Overview

In order to make our system efficient we will deploy the divide and conquer methodology in which the model will first detect all possible vehicles in the input frame and then will classify the input on basis of the texture i.e. on basis of the color, make and type of the car so that the input unknowns can be reduced and confidence level could be increased then in the second phase the model will try to re identify the vehicle form the short listed inputs on basis of the information obtained from classification and lookup table. Upone successful identification system will share lookup table with other nodes in a network. Following figure-3 shows the flow diagram of this model.

                                                  

In order to design such a model, first we will design a simple model for a single camera and will divide our work into three phases which will be

• Multi-Vehicle Detection
  1. Vehicle Detection and Localization using DNNs (on Mobile GPUs) Optimized YOLO/SSD to run on Intel NCS2
  2. Lightweight Multi-Object (Tracking Stationary Object Filtering and Camera Motion Cancellation)
  3. Attribute Extraction (Visual Descriptor: color histogram, make and type )
• Vehicle Association across Cameras
  1. Visual Descriptor Association (Register attributes in a lookup table)
  2. Spatial-temporal Association (Time of detection)
• Path Inference
  1. Probabilistic trajectory estimation
  2. Invoking mobile pipeline when ambiguity happens(Multiple vehicles of same make and type appear at the same time)

Benefits of the Project Advantages and applications

By implementing a vehicle re-ID system, we can increase the reliability of surveillance of smart city projects as the processing speed of computer is way faster as compared to human and for a well-trained model it is very rarer that it will miss classify the input query thus the re-id system can reduce the human error factor in result increasing reliability of system. The re-Id system can used for the following major tasks in a smart city project: 

• Suspicious vehicle search: In most of the unlawful acts the criminals use the fast vehicles to flee of the site after their purpose is completed and it is a very stress full and time resource consuming tasks for the security officers to accurately identify the suspected vehicle and a small mistake can cause a lot of damage so in this scenario the re-id system can easily overcome the human error and can keep track of suspect’s vehicle. 
• Cross camera vehicle tracking: In some of the sports the viewer is interested in a specific vehicle and just want to be updated about the status of that specified vehicle so in this case the re-id system can be used to keep track of specific vehicle’s position and keep the viewer UpToDate whenever that specific vehicle come into the field. 
• Vehicle counting: This re-id system can be used for counting the vehicles passed through a specific path as it could uniquely distinguish between different vehicles on basis of their unique features like physical appearance and license plate.

Additional Use Cases

• Automatic toll collection: Vehicle Re-Id systems can also be applied on toll gates to automatically identify vehicle type and apply the toll rate.
• Lane access restriction management: In most parts of big city the traffic flow is controlled bt restricting the entrance of heavy vehicle in day time so in this scenario the re-id system can help surveillance team to keep track of vehicle which did not followed the rule and automatically generate a ticket for that specific vehicle and also this system can be used for allowing automatic entrance of specific vehicles to the priority lane by identifying their priority. 
• Intelligent parking: In side a city there are many private or government parking where access is provided to the specific people only by using this re-id system we can fully automate those parking access and can save a lot of human effort,
• Traffic flow analysis: To analyze the congestion on different routes at different times, such as peak hour calculations or behavior of the specific type of vehicle.
• Vehicle Speed Approximation: This re-identification system can be used for vehicle speed approximation in such a way that whenever a camera identifies a unique vehicle it record the appearance with time stamp and upon re-identification on next camera we can calculate speed by dividing distance between those two cameras by the time taken by vehicle to travel that distance

Technical Details of Final Deliverable Deliverable system

Upon the completion of the project the final deliverable will be a single module consisting the raspberry pi, a camera and Intel® Movidius Neural Compute Stick.

The re-Identification model will be trained on the NVIDIA TK1 and once it is trained the model will be deployed on the Intel® Movidius Neural Compute Stick which will be used to run that model on raspberry pi.

This module will be a standalone re-identifying unit and will be ready to go out of box. In order to connect the re-id module with the centralized network we will just need an internet connection and the modules will automatically connect to the central server through a secure connection and will start broadcasting its live feed to the server/control room.

Final Deliverable of the Project HW/SW integrated systemCore Industry SecurityOther Industries Transportation Core Technology Artificial Intelligence(AI)Other Technologies Internet of Things (IoT)Sustainable Development Goals Industry, Innovation and Infrastructure, Sustainable Cities and Communities, Peace and Justice Strong Institutions, Partnerships to achieve the GoalRequired Resources

Item Name	Type	No. of Units	Per Unit Cost (in Rs)	Total (in Rs)
			Total in (Rs)	80000
Single-board computer(Raspberry Pi) and Accessories	Equipment	2	12000	24000
Intel neural compute stick	Equipment	2	20500	41000
Camera	Equipment	2	2500	5000
Prototype case	Miscellaneous	2	3500	7000
Miscellaneous	Miscellaneous	1	3000	3000