Multi-modal Sensor Fusion (Camera, LiDAR) for Environmental Depth Perception for Autonomous Vehicles

Our final year project is based on data acquisition, and visual perception for autonomous vehicles. The main objective is to develop sensor fusion model by gathering results from LIDAR and stereo-cameras and fusing it to estimate a depth map of short range. At the end of the project, we will be able

Project Title

Multi-modal Sensor Fusion (Camera, LiDAR) for Environmental Depth Perception for Autonomous Vehicles

Project Area of Specialization

Artificial Intelligence

Project Summary

Our final year project is based on data acquisition, and visual perception for autonomous vehicles. The main objective is to develop sensor fusion model by gathering results from LIDAR and stereo-cameras and fusing it to estimate a depth map of short range. At the end of the project, we will be able to perform LIDAR and stereo-camera calibration and create an algorithm that will apply sensor acquisition techniques utilizing LIDAR, computer vision and deep learning models to estimate depth maps obtained in real time. We will also use coding libraries such as OpenCV in our perception model. Completion of the project also includes mounting and acquiring data from sensors and testing the optimum algorithm on Jetson development kit.

Project Objectives

• Perception: Our objective is to build a perception system which can create a real time depth map of its surroundings. 
• Enhancing Skillset: Building this project will require us to gain knowledge about autonomous systems and computer vision which will be a great learning experience for us.
• Industrial Implementation: By using the learning outcomes from this project and applying it in industry will help speed up the production of autonomous cars in Pakistan.
• Research: Researching the development and use of autonomous vehicles in Pakistan. 

Project Implementation Method

  

1: Hardware & Sourcing :  

• Acquiring ATV, LIDAR And stereo-cameras.
• Designing the sensor brackets in solidworks.
• Mounting those brackets on an ATV.

2: Software & Design: 

•  Acquire data from sensors.
•  Calibrating the sensors.
•  Sensor synchronization: Synchronizing the   frame rate of both LIDAR and stereo-camera.
• Transforming sensor inputs.

3: Implementation of Sensor Fusion:

• Generating a depth map and using kalmal filtering.
• Improving depth map.

4: Depth Map Estimation 

Benefits of the Project

Technical Details of Final Deliverable

• Sensing system utilizing LIDAR and stereo cameras.
• Short range depth perception. 
• Real time depth perception in a controlled environment.

Final Deliverable of the Project

Software System

Core Industry

Transportation

Other Industries

Core Technology

Artificial Intelligence(AI)

Other Technologies

Others

Sustainable Development Goals

Good Health and Well-Being for People, Industry, Innovation and Infrastructure, Reduced Inequality, Climate Action

Required Resources

If you need this project, please contact me on contact@adikhanofficial.com