Intelligent Lane Detection using Artificial Intelligence

Lane detection in driving scenes is an important module for saving human lives and preventing fatal accidents. Lane line detection has been an active field of research with many applications. However, due to the challenging scenarios such as occluded lanes in real driving environment, lane detection

Project Title

Intelligent Lane Detection using Artificial Intelligence

Project Area of Specialization

Artificial Intelligence

Project Summary

Lane detection in driving scenes is an important module for saving human lives and preventing fatal accidents. Lane line detection has been an active field of research with many applications. However, due to the challenging scenarios such as occluded lanes in real driving environment, lane detection is still a challenging task today. To address these issues, a robust method for lane detection based on computer vision and machine learning techniques is proposed. The first step of this approach is to apply Difference of Gaussian edge detection method to road image. Hence, noise in the image are filtered out and fine edges of lane lines are obtained. This road image will be warped using inverse perspective mapping (IPM) to obtain a ‘bird’s eye view’ of the road. In order to mask the region of interest, a fully convolutional network will be trained end-to-end to detect road pixels using semantic segmentation to overcome light invariance challenges. Using machine learning, a Lane Departure Warning system will be implemented to assist drivers with other functionality of steering angle and trajectory prediction. The proposed system can be applied on a wide variety of road and weather conditions.

Project Objectives

Our algorithm pipeline consists of Edge detection, Inverse Perspective Transform and line fitting to detect lane lines during driving.

To improve accuracy, our lane detection method will be trained on a convolutional neural network using datasets such as CULane, TUsimple, British CamVid. KITTI academic datasets. Semantic segmentation refers to the process of linking each pixel in an image to a class label. We can think of semantic segmentation as image classification at a pixel level. We used transfer learning and data augmentation to train on the British CamVid academic dataset using the state of the art ResNet Architecture which achieved super human performance on the Imagenet competition with an accuracy of 96.4%. It’s the only neural network architecture that has overcome the vanishing gradient problem by using skip layers. It has 152 layers enabling it to make better inferences for pattern recognition. Semantic segmentation allows the AI to perceive road scenes and correctly identify different classes such as roads, lane lines, pedestrians and cars.

A level of advanced driver assistant system will be implemented, as in a center offset prompt that will inform the driver in what percentage are they deviating from the center of the lane. This is to be referred as the “lane keeping state” [either “Good Lane Keeping” or “WARNING! OFF LANE”] that will be determined from the radius of curvature of the lane. Furthermore, a steering prediction will also be provided based on the trajectory of the lane lines which will be either “stay straight”, “left curve ahead” or “right curve ahead”

To determine the robustness and accuracy of our model, it will be tested practically on a toy car in a model environment. The toy car will be a robot chassis consisting of two back motors and a turning mechanism controlled by a servo to make turns on a increased possibilities of turning angles to simulate real life scenario.

Our algorithm will be deployed on a toy car that will identify lane lines and autonomously navigate the environment. This model will be implemented on the latest Raspberry Pi 4 (4 GB RAM) which is capable of handling the machine learning algorithms to detect lane lines as well as navigate the environment autonomously by controlling the robot chassis forward motors and steering servo via its GPIO pins.    

Project Implementation Method

Difference of Gaussian is a feature enhancement algorithm that involves the subtraction of one blurred version of an original image from another, less blurred version of the original. Blurring an image using a Gaussian kernel suppresses only high-frequency spatial information. Thus, edge detection is more applicable to a wide variety of noise scenarios.

Comparing with the current state of the art the Canny edge detector which was developed by John F. Canny in 1986. The primary disadvantage of using Canny edge detector is that it consumes a lot of time due to its complex computation. The amount of parameters leads to infinitely tweaking for getting just that little better result

Inverse Perspective transform maps the points in a given image to different, desired, image points with a new perspective. The perspective transform is a bird’s-eye view transform that let’s us view a lane from above; this is useful for calculating the lane curvature.

For line fitting, the traditional line detection method is Hough transform, which must calculate the sin cosine function and its computational accumulated in the two-dimensional histogram complexity is high. A fast line detector is proposed, which can detect lines with any orientation, location. The key advantages of this method are its simple implementation and its low computational complexity and the time our method takes is less than 1/3 Hough transform does at the same precision. At the same time the algorithm is not sensitive to noise.

Semantic segmentation refers to the process of linking each pixel in an image to a class label. We can think of semantic segmentation as image classification at a pixel level. We used transfer learning and data augmentation to train on the British CamVid academic dataset using the state of the art ResNet Architecture which achieved super human performance on the Imagenet competition with an accuracy of 96.4%. It’s the only neural network architecture that has overcome the vanishing gradient problem by using skip layers. It has 152 layers enabling it to make better inferences for pattern recognition.

Benefits of the Project

• Studies show that over 90% of road traffic collisions are caused by human error, and a 2-second early warning can prevent 99% of accidents. Advanced Driver Assistance System, or ‘ADAS’, is designed to help prevent accidents by scanning the road ahead for potential hazards and warning the driver.
• Automotive companies such as Honda, Toyota, Indus Motor Company, Hino Pak etc can deploy lane detection ADAS system on their vehicles to ensure safety of their customers and enhance their driving experience.
• Due to increased safety of ADAS lane detection, drivers can avoid fatal road accidents which result in repair costs, hospital bills, loss of human life and PTSD.
• This friendly driver assist system notifies the driver during the journey with unique alerts using sound, vibration and pictographs. The AI-powered algorithm works continuously in real time to prevent accidents and safeguard your fleet assets.
• It is proven that a 2-second warning can significantly reduce the probability of forward-collisions almost entirely; allowing for more confidence on the road and the avoidance of serious incidents.
• Development of a balanced and safe driving style, greater confidence behind the wheel.
• This system ensures a safer driving scenario even on Pakistani roads where mostly lanes are occluded.

Technical Details of Final Deliverable

Lane detection with ADAS functionality will be developed using image datasets such as KITTI, CULane, TuSimple, Cityscapes and British CamVid. Classic computer vision statistical techniques will be developed and tested on car dashcam footage to measure the merits and potential improvements that can be made to the algorithm.

ADAS functionality such as Lane Departure Warning will be introduced using mathematical feature extraction techniques such as center offset, steering angle, trajectory prediction and radius of curvature.

In order to tackle challenging scenarios, a Convolutional Neural Network will be designed to overcome light invariance challenges, occluded lanes and absence of lane lines. Semantic Segmentation or pixel by pixel masking techniques will be deployed by transfer learning on academic road datasets, utilizing the state of the art ResNet architecture. It allows the AI to perceive its environment in road scenes.

Our algorithm will be benchmarked on simulators that resemble real world scenarios. One example is the Eurotruck Simulator 2, with its immersive graphics and real world physics will allow us to test our algorithm in challenging conditions, identify points of failure and tune our neural network accordingly.

The next step is to implement our algorithm on a toy car and interface the AI with actuators which will navigate the track autonomously. The latest Raspberry Pi 4 (4 GB RAM) will run the deep learning and computer vision algorithms while the Arduino Uno will delegate commands to the actuators. The raspberry pi and the arduino will exchange information using master slave communication.

After our algorithm has been refined to be deployed on an actual car, a portable device will be developed to provide ADAS functionality on Pakistan’s roads. 

Final Deliverable of the Project

HW/SW integrated system

Core Industry

Transportation

Other Industries

Education , Security

Core Technology

Artificial Intelligence(AI)

Other Technologies

Robotics

Sustainable Development Goals

Good Health and Well-Being for People, Sustainable Cities and Communities, Life on Land

Required Resources

If you need this project, please contact me on contact@adikhanofficial.com