Automated-Asset Management

While objects of interest can be extracted from flat or 360? images using deep learning techniques, image data is not representative enough to also extract dimensional measurements of those objects. In this project, we aim to develop an algorithm that provides accurate measurements of certain roadwa

Project Title

Automated-Asset Management

Project Area of Specialization

Artificial Intelligence

Project Summary

While objects of interest can be extracted from flat or 360° images using deep learning techniques, image data is not representative enough to also extract dimensional measurements of those objects. In this project, we aim to develop an algorithm that provides accurate measurements of certain roadway assets using LiDAR point cloud data.

Automated-Asset Management ( AAM) is a software that will use the LIDAR point cloud and its corresponding 2D image of the roadway and then it will do the rest from identifying assets to find their desired dimension automatically.

Project Objectives

The main goals of this software are reducing the amount of time and money required to process the LIDAR point cloud with respect to their 2D images of the roadway and finally finding out the dimensions of the roadway assets. 

Project Implementation Method

Each asset in the 2d image is contained inside bounding boxes. By using the matrices in the camera calibration file, we calculate the projection matrix using this formula:-
projection_matrix=P2*R0_rect*Tr_velo_to_cam 
Here, P2 is a matrix of the camera which has captured the 2d image.
    Px matrices (3x4) project a point in the rectified referenced camera coordinate to the camera_x image. camera_0 is the reference camera coordinate.
    R0_rect R0_rect (3x3) is the rectifying rotation for reference coordinate (rectification makes images of multiple cameras lie on the same plan). It is a rotation matrix to map from object coordinate to reference coordinate
    Tr_velo_to_cam (3x4) maps a point in point cloud coordinate to reference co-ordinate. It is a euclidean transformation from lidar to reference camera cam0

With this projection matrix, we map LiDAR 3d data on the 2d image. After this mapping, we find respective 3d LiDAR points of 2d points of the bounding boxes. Now that we have bounding boxes of assets in 3d LiDAR, we apply Principal Component Analysis to find the axis where the assets lie and convert the polygon-shaped bounding boxes to rectangles to find their height and width.

Benefits of the Project

In the already available road asset management system, a lot of manpower and time is required just to annotate the assets in the photos, and if the dimensions of the assets are to be found then the same goes for that but now with AAM all that can be avoided as this system will automatically identify the assets in photos and measurements will be calculated from their respective LIDAR point cloud.

The Automated Asset Management system calculates the dimensions of the assets. It helps in Road Maintenance Process. The main benefit of this system is that it does all this process automatically rather than manually. It saves a lot of time and human resources. When this process is manually done, it takes weeks or sometimes months and a lot of human resource and a lot of costs has to be spent on it but with this automation system, a lot of time, cost, and human resource can be saved. It improves road management and therefore Transportation.

Technical Details of Final Deliverable

The final deliverable will be a python script that will be able to measure the dimensions (width and height) of the assets (currently 5 types) present in the digital image. The script also includes a usage guide which can be accessed through the help command. It will be executed from the command line and requires a path to the dataset. The path format can be acquired through the help command of the script. At least one data instance is required to produce the result. A data instance consists of the following four files; lidar file (.bin), image file (.png), calibration file (.txt) and annotation file (.json). The first three files are from Kitti 3d object detection dataset and the last JSON file contains assets’ localization bounding boxes points. All four files should have the same base name. Additionally, an optional ground truth file (.csv) can be loaded with the dataset to measure the accuracy of the system. The script will produce a result file (.csv) containing the measurements of the assets, errors if there is any problem in the files, or measuring dimensions and accuracy details if the ground truth file is provided. The formatting of the files can be accessed through the usage guide.

Final Deliverable of the Project

Software System

Core Industry

IT

Other Industries

Core Technology

Artificial Intelligence(AI)

Other Technologies

Big Data

Sustainable Development Goals

Industry, Innovation and Infrastructure, Partnerships to achieve the Goal

Required Resources

If you need this project, please contact me on contact@adikhanofficial.com