Autonomous Navigation of Customized ROS based Quadcopter using ORB SLAM with Nvidia Jetson Nano and Stereo Image Processing or LiDAR Waypoint Tracking

Over the last decade, work on Unmanned Aerial Vehicles (UAVs) and particularly Quadcopters has captured the attention of both academia and industry because of their mechanical and control simplicity, high manoeuvrability and low cost of entry point. These features of UAVs have made them popular aeri

Project Title

Autonomous Navigation of Customized ROS based Quadcopter using ORB SLAM with Nvidia Jetson Nano and Stereo Image Processing or LiDAR Waypoint Tracking

Project Area of Specialization

Robotics

Project Summary

Over the last decade, work on Unmanned Aerial Vehicles (UAVs) and particularly Quadcopters has captured the attention of both academia and industry because of their mechanical and control simplicity, high manoeuvrability and low cost of entry point. These features of UAVs have made them popular aerial robots in a large number of Robotics applications such as Defence, Transportation (package delivery), Entertainment (cinematics) and surveillance. 

However, safe navigation in unknown environments is a challenging task for most autonomous UAV systems. Most previous works generally avoid obstacles and perform navigation assuming that the environment is static and the position of the Quadcopter is known in a previously defined map. In recent years, we have seen the development of various vision-based autonomous UAVs that can perform autonomously in unknown environments - which we want to achieve in our project.

The main challenge of vision-based autonomous navigation is the UAV is required to track a set of waypoints and avoid collisions with only a set of resources with limited capability (the onboard sensors and companion computer) during the flight. To achieve this task, we generally take the UAV's odometry, estimated with the onboard visual and inertial information, as the feedback for waypoint tracking. We also require visual or ranging data (such as Depth Image, Light Detection and Ranging or LiDAR measurements, etc) to perceive the local environment. In order to avoid collisions with obstacles, both the estimated states of the UAV and obstacles are required to generate collision-free trajectories for safe, autonomous flight.

The main purpose of our Project is to develop a customized, ROS-based Quadcopter model to suit our requirements and durability standards so that it can be used for safe, autonomous navigation in unknown static environments. We will present an on-board vision-based approach for the avoidance of obstacles in such environments using visual odometry algorithms such as ORB SLAM. LiDAR Waypoint Tracking or Stereo Image Processing techniques shall be used to perceive the environment and provide estimates of the position and size of obstacles. Several Artificial Intelligence (Deep Learning) techniques can be utilized to enhance the accuracy and performance of the Image Processing and autonomous navigation techniques. This requires fast processing and computing capabilities that can only be provided by a GPU or Graphical Processor Unit such as Nvidia's Jetson Nano. Jetson Nano will also enable us to use ROS framework for controlling, communicating and interacting with our Autonomous Quadcopter.

Project Objectives

The Main Objectives of our Project are as follows:

1. Assemble and develop a suitable customized Quadcopter model capable of Autonomous Navigation and Hardware testing.

2. Be able to control the movement of our Quadcopter both manually and autonomously.

3. Use ROS framework for interfacing all on-board Sensors, Flight Controller and on-board GPU together and communicate flight data and information remotely on a Base Station (PC) via WiFi.

4. Develop an efficient Position Tracking System (using either LiDAR or Stereo Camera) for implementation of visual SLAM (ORB SLAM).

5. Study and implement Simultaneous Localization and Mapping (SLAM) and make Quadcopter capable of navigating autonomously in unknown static environments.

6. Perform SITL (Software In The Loop) Simulation in order to test Obstacle Avoidance and vision-based Navigation Algorithms.

7. Enhance performance by implementing advanced AI/Deep Learning-based Image Processing Techniques on Nvidia Jetson Nano.

Additional Objectives which we may work towards once main objectives have been fulfilled:

1. Perform HITL (Hardware In The Loop) Simulation on real Flight Controller Board.

2. Explore the possibility of expanding our Project Scope to incorporate defence or military-based applications.

3.  Explore other functionalities of Autonomous Drones (such as Positioning using AruCo Markers, QR codes or RFID tags, etc)

4. Test our Quadcopter with different payloads.

Project Implementation Method

The first stage of our Project involves the actual assembly and setup of the basic Quadcopter hardware required for an autonomous UAV. Once the hardware assembly is completed, we will have to install the PX4 Firmware into our Pixhawk Autopilot Flight Controller. The firmware setup and sensor calibration can be performed using Ground Control Station (GCS) Software called QGround Control. Once these steps are completed, we can prepare our Quadcopter for Manual Flight via Radio Telemetric Control.

For the next stage, we will introduce our Companion Computer/GPU - Nvidia's Jetson Nano - into our Quadcopter Model. We will install Ubuntu 18.0 Operating System along with ROS Melodic firmware. We will then interface our Flight Controller (Pixhawk Autopilot) with Jetson Nano via MAVROS (a ROS Package that will enable communication with Jetson Nano and Pixhawk Autopilot via MAVLink Protocol for communication). Once interfacing is successful, we will be able to receive all the information from our Pixhawk Flight Controller including all the sensor data. We will also install Ubuntu 18.0 and ROS Melodic into our PC which will act as our Base Station for accessing the Flight Data received from our Flight Controller. Our Base Station will communicate with our on-board Computer, Jetson Nano, over ROS via WiFi network.

After this, we will try to perform SITL Simulation using the Gazebo Simulator to test basic code and programming for first Autonomous Test Flight. We will find a suitable Software model similar to our Drone which we can use for our simulations. This would prove more efficient than designing our own model from scratch since it is quite a tedious and time-consuming task. We will also have to create/find a suitable environment in which we can test our Quadcopter Model. Once our code for basic Autonomous Test Flight begins to work without any obvious errors, we shall test the code on our actual hardware and analyse its performance.

Once completed, we shall work on implementing ORB SLAM. Before going forward with this, we will have to decide on a suitable Position Tracking System using either LiDAR or Stereo Cameras. We will choose the option that proves to be most efficient in terms of performance and low cost. We will setup our Computer Vision sensors on to our Quadcopter Model and interface them with Jetson Nano. Following this, we will need to find suitable Image Processing Algorithms to pre-process the LiDAR/camera outputs before they can be used further for navigation. We will then move to implementation of ORB SLAM by installing the required packages and study how this technique is used for tracking, mapping, relocalization and loop closing. We will first attempt this task on simulation. Once successful, we will test it on our hardware system.

Benefits of the Project

Our Project can provide a number of benefits to us as students and to our country which are explained as follows: 

Benefits to us as Students:

• An opportunity to understand and work with the widely popular ROS framework at Undergraduate level will further our experience and understanding in the field of advanced Robotics.
• Building this project will expose us to the problems that arise when working with actual hardware systems as well as the limitations of these systems.
• The project will further our pragmatic understanding of core practical concepts learnt throughout our BE Program.
• This Project will give us the opportunity to work on autonomous systems that can potentially enhance the way we live and work while also improving the economy of the country

Benefits to our Country:

• Our Project can be used for Defence and military-based applications such as surveillance, simple offensive maneouvers, intruder detection and others that can provide tactical advantages to our Armed Forces, hence strengthening our defences.
• It can help nature and wildlife conservationists study and protect our natural environment.
• It can be used for Border Patrol and to detect illegal activities such as Smuggling, trafficking, etc. It can also be used by law enforcement agencies for effective crime prevention in the city, hence improving the law-and-order conditions in Pakistan.
• It can be used by Disaster Mitigation and Emergency Response Agencies to inspect buildings, bridges and other infrastructure as well as disaster-stricken sites with heavy damage to save lives and provide safety.

General Benefits of Autonomous Quadcopters and other UAVs:

• They can be used to tackle very remote and dangerous tasks autonomously.
• They can be used for a wide range of applications from data collection and surveillance to delivery.
• It can be used to deliver packages from one place to another in heavily populated areas.
• They can be used to access damage, locate victims and deliver aid in remote areas affected by natural disasters. They can also assist Search & Rescue Teams and other authorities during distressing times that quick, emergency response.
• They can be used in industries for various applications such as inspection work, surveying, construction planning and management, etc.
• They can help journalists get better news coverage and also acquire aerial footages easily from areas difficult to access by humans.
• They can be used by Real Estate owners to navigate through a property autonomously, develop its map and even provide a video-feed to the owner. 

Technical Details of Final Deliverable

Our Final Project Deliverable will be a Hardware-cum-Software system along with a complete Manual/Guide explaining how to operate the System. We will try to make a working Quadcopter than be configured to work both manually (via Radio Transmitter and Receiver) and autonomously (by running a Python script). We will monitor the Quadcopter and obtain the sensors data on our Base Station (PC). We will also try to find a suitable arena where we can test our Quadcopters navigation capability by implementing SLAM.

As for the Software, we will present a working simulation of our Quadcopter while performing autonomous navigation in various environments. The Simulation will be carried out over Gazebo. 

We will also provide a Manual/Guide book (in softcopy or hardcopy format) with clear instructions on how to setup, monitor and operate the Quadcopter.

Final Deliverable of the Project

HW/SW integrated system

Core Industry

Others

Other Industries

IT , Transportation , Media , Health , Security

Core Technology

Robotics

Other Technologies

Artificial Intelligence(AI)

Sustainable Development Goals

Good Health and Well-Being for People, Decent Work and Economic Growth, Industry, Innovation and Infrastructure, Sustainable Cities and Communities, Peace and Justice Strong Institutions

Required Resources

If you need this project, please contact me on contact@adikhanofficial.com