IoT based automated surveillance system for smart cities

Due to advances in information technology (e.g., digital video cameras, ubiquitous sensors), the automatic detection of human behaviours from video is a very recent research topic. Human detection and their corresponding behaviours have been studied under distinct perspectives in a wide variety of d

Project Title

IoT based automated surveillance system for smart cities

Project Area of Specialization

Internet of Things

Project Summary

Due to advances in information technology (e.g., digital video cameras, ubiquitous sensors), the automatic detection of human behaviours from video is a very recent research topic. Human detection and their corresponding behaviours have been studied under distinct perspectives in a wide variety of disciplines, such as psychology, biomechanics and computer graphics. An action consists of action primitives and describes a, possibly cyclic, whole-body movement. Finally, activities contain a number of subsequent actions, and give an interpretation of the movement that is being performed.

The main research question underneath this work can be characterized as follows: given a succession of pictures with one or more persons performing an action, can a framework be outlined to distinguish: who is performing the action and what action was performed? The particular interest in this area has dramatically increased with the advances of information technology. In particular, depth-imaging (3D) has substantially improved in the last few years, finally reaching an affordable consumer price (e.g., Kinect for Xbox 360). As indicated by BBC news, in 2011 Microsoft Kinect was the most selling gadget on record, with its sales achieving 23 million units in 2013. One of the highlight of this sensor is the low price for consumer market and the research intensive computing. The data capturing is cheap and the data is stored in a form ready for post-processing. This is a high resolution sensor and allows data to be captured in color, grayscale, infrared and depth. Using this device, it is possible to capture multiple users at the same time. The device allows to capture gait and gesture easily.

In this project, we propose an automatic human action recognition system based on Kinect sensor. The propose system will have the ability to recognize normal and abnormal behaviours in real time. By using this system, it will be possible to identify significant joint patterns (i.e, postures) with the help of Kinect sensor. The overall system architecture can be divided into 4 phases i.e., data acquisition, pre-processing, feature extraction and classification. In this project, we also aim to develop our own dataset at Abdul Wali khan University Mardan campus and compare the results with the state-of-the-art methods. As a novel solution to automatic action recognition, challenges and application of the developed system will be explored in health care, university campuses, surveillance and activity recognition.

Project Objectives

Recently, the area of IoT based automated surveillance system for smart cities has been analysed by various researchers. Many solutions have been proposed regarding the processing of color images captures from simple traditional cameras. In majority of works, the authors are using binary images to represent the silhouette of an individual. These silhouettes are obtained by processing RGB images, which requires a vast number of computational steps e.g., background removal. Other research works use sensors worn by an individual on their body to capture body posture.

With the recent advances in 3D depth sensors such as Microsoft Kinect have created many opportunities for surveillance, security and entertainment. To maintain the security of both people and infrastructure, new technologies are contributing to the realization of more powerful systems that detect varying human behaviours . In order to detect varying human behaviours irrespective of lighting conditions in buildings or production facilities, researchers have extensively attempted to use depth information from 3D sensors.

The proposed work will use Kinect sensor for obtaining the skeleton of an individual. These cameras are widely available and provide high definition result. The vision system of the Microsoft Kinect is composed of two cameras (i.e., an RGB camera and an IR camera) with 640x480 resolution, and an IR projector that is responsible of shooting infrared rays toward the environment. The distortion degree of each ray projected against the scene is used to estimate a depth map in which each pixel value represents the distance of a specific 3D point from the Kinect.

Project Implementation Method

The IoT based automated surveillance system for smart cities proposed in this project will be divided into four phases i.e., data acquisition, pre-processing, feature extraction and classification. We will use the skeleton joint position obtained from software development kit (SDK) of the Microsoft Kinect. The project is focused on how the Kinect sensor captures the 3D information of a scene and recognizes the action being performed by the human body by retrieving the depth image information and real-time skeletal tracking.

The recorded actions will be divided into two categories i.e., normal and abnormal behaviours. To perform skeleton tracking, the Kinect sensor detects 24 joints in the human body which represent different body parts. Utilizing this 3D joint information, the Kinect identifies the gestures and actions being performed by the human body. Pre-processing will be done in order to remove noise from the data. Finally, classification will be done using deep learning neural algorithm. The implementation for the proposed system will be done using Python with OPEN CV.

Benefits of the Project

IoT based automated surveillance system for smart cities using Resberry & Kinect has a great number of applications in various fields like robotics, health care, computer science, and many commercial areas. Users can play games by utilizing their own body movements. Indeed, even in the field of medicine, the doctor can operate on a patient from a remote location by using Kinect. Kinect sensors has a wide range of applications in software development for human interaction. Action recognition using Kinect has been a great advancement in computer vision based HCI (Human Computer Interaction).

Technical Details of Final Deliverable

The main contributions of this work include:

1. a systematic literature review on automatic human action recognition. This review paper will help in identifying the underlying challenges like pose, illumination, clutter and the proposed solution in the literature. We will use these results for comparing the outcome of our project to existing methods, to bring novelty to the existing system and to cover the research gape. 
2. Developing a prototype of the proposed system that will be able to work in real time for performing automatic action recognition. The system can be deployed in universities, schools, banks, health care and other surveillance areas for abnormal action recognition.
3. Two research articles will be published from this project i.e., a systematic literature review, and another covering the technical detail carried out in this project.

Final Deliverable of the Project

HW/SW integrated system

Core Industry

Security

Other Industries

Education , Transportation

Core Technology

Internet of Things (IoT)

Other Technologies

Artificial Intelligence(AI)

Sustainable Development Goals

Industry, Innovation and Infrastructure, Sustainable Cities and Communities

Required Resources

If you need this project, please contact me on contact@adikhanofficial.com