Driver Activities Recognition and Monitoring System using Deep Learning

In this era, almost everyone has bought its own vehicle to reach on time at their destination. So, the number of personal vehicles is increasing day by day and traffic on the road is also increasing. With the increase in working in a corporate world, many people have to sacrifice their time and slee

Project Title

Driver Activities Recognition and Monitoring System using Deep Learning

Project Area of Specialization

Artificial Intelligence

Project Summary

In this era, almost everyone has bought its own vehicle to reach on time at their destination. So, the number of personal vehicles is increasing day by day and traffic on the road is also increasing. With the increase in working in a corporate world, many people have to sacrifice their time and sleep to survive in the completive world. It takes a heavy toll on the person’s life cycle as well on the society. To make sure that the company operating profitably, the personal expenses must meet the employees are sacrificing their healthy and luxury life. Sometimes, driver lost the concentration during a driving. Due to this irregularity, a lot of accidents are occurring on the highway. For resolving this problem of modern era, monitoring the activities of driver, detecting unfortunate movements, and try to return the driver to its normal state is major concern. The probability of such a kind of mishaps can be minimized up to 90 percent. Our propose system have a lot of benefits for the community by preventing the occurrences of accidents.  It will help us to minimize the number of accidents by monitoring the activities of drivers and generating the alarms before the mishaps happened. In this way, we can save lives of thousands of people. We have evaluated our system and found an average accuracy of 95.55% on suggested system that is comparable with other state of methods.

System would cover the activities of the driver and detect the state of the driver either the driver is in drowsiness state or not, if in drowsiness state then the system will measure it and force driver to retrain from the drowsiness state by generating a warning alarm. System will also detect the state of smoking, drinking, mobile use and unwanted movement of driver’s head. In every project there are some constraints. So, in our case, the major constraints are dataset collection.

Project Objectives

1. To accurately detect a face from an image
2. To detect the region of interest in this case the eyes
3. To accurately classify the state of the eye either closed or open
4. To provide a warning to the driver if drowsiness is detected
5. To detect smoking and drinking during driving 
6. To detect the mobile using state and generate an alarm

Project Implementation Method

.This section presents our proposed 3D-CNN approach to robustly detect challenging objects, i.e. eyes, cell phone in the videos and images. In 3D-CNN it works on Region of Interest (ROP) technique, like it first checks region and then detect an object.

So, we use convolutional network; as the convolution layers go deeper, each pixel in the corresponding feature map gather more and more convolutional information outside the ROI region. In order to enhance the capability of the network, we use different layers i.e. conv3 and conv4, for ROI pooling (Fig. 1). So, the network can detect lower level features which contain higher proportion of information in ROI regions.

The proposed method first makes use of deep features extracted from our 3D-CNN approach to individually detect different objects. Then, we use geometric information, namely, the joint area between the eyes. Our defined network includes five sharing convolution layers, i.e. conv1, conv2, conv3, conv4 and conv5 as the standard one. In the first two convolution layers, right after each convolution layer, there are one ReLU layer, one LRN layer and one Max-pooling layer respectively. In the next three convolution layers, right after each convolution layer, there is only one ReLU layer. Especially, in three convolution layers, i.e. 3, 4 and 5, their outputs are also used as the input to three corresponding ROI pooling layers and normalization layers as shown in Fig. 1. These L-2 normalization outputs are concatenated and shrunk to use as the input for the next two fully connected layers. In the final steps, there are both a SoftMax layer for object classification and a regression function to take care of bounding box refinement.

Tools and technologies:

            Frameworks: Numpy, Scikit-learn, Matplotlib, TensorFlow, Keras.

            Language: Python      

Database: MYSQL

            Technology: Computer Vision with Deep Learning

            Algorithm: 2D, 3D CNNs

Benefits of the Project

In the 21st century, stress driver has continued to be a major challenge contributing to a large number of accidents on different roads. Drowsiness is one of the factors of collision of different vehicles. Sometimes, driver gets involved in other activities like eating, smoking, drinking and mobile call. According to circumstances, any kind of accident can be possible just little bit distortion of driver. In Pakistan, fatigue driver especially among long distance truck drivers, public service vehicles drivers and private vehicle drivers is a major concern. It continues despite the government putting in place several measures to address the problem, also including regulation of the public vehicle travel time, increasing the number of drivers for buses that travel at night, use of alcohol blows to detect drunk drivers among many others.

Driver Activities Monitoring and Detection System among drivers have not been achieved making it difficult to enforce relevant legislations. Only few systems are available in the market. Those are so expensive to make them available for public who can afford the cost of the current vehicles fitted with latest technologies. Hence, the system has improved to great extent to make it affordable for public as many people have low monthly income. Public service vehicles to help address the many accidents associated with driver activities.

Scope

System would cover the activities of the driver and detect the state of the driver either the driver is in drowsiness state or not, if in drowsiness state then the system will measures it and force driver to retrain from the drowsiness state by generating a warning alarm. System will also detect the state of smoking, drinking, mobile use and unwanted movement of driver’s head. In every project there are some constraints. So, in our case, the major constraints are dataset collection.

Technical Details of Final Deliverable

Our final Deliverable a system which is deploy into a car with a camera and system . Camera detect the activities of the driver by using Faster RCNN and make an alarm on showing activities.

our system detect the 4 major activities of the drivers which are given below:

1) Droziness state of the Driver 

2) Drinkin during Drive

3) Smoking Detection

4) Mobile use

our system when find out that all activities it show alarmed to driver.

Final Deliverable of the Project

HW/SW integrated system

Core Industry

IT

Other Industries

Others

Core Technology

Artificial Intelligence(AI)

Other Technologies

Internet of Things (IoT)

Sustainable Development Goals

Good Health and Well-Being for People, Industry, Innovation and Infrastructure

Required Resources

If you need this project, please contact me on contact@adikhanofficial.com