Selfie Drone

UAVs are being used for various tasks and have implementations in various industries, photography being one of them. With the rise in popularity of selfies, specialized ?Selfie Drones? have been introduced and a ?selfie? mode can be found in almost every commercially available drone. These drones of

Project Title

Selfie Drone

Project Area of Specialization

Robotics

Project Summary

UAVs are being used for various tasks and have implementations in various industries, photography being one of them. With the rise in popularity of selfies, specialized ‘Selfie Drones’ have been introduced and a ‘selfie’ mode can be found in almost every commercially available drone. These drones offer some level of autonomy to assist the user in taking selfies. There are multiple approaches being used and developed to accomplish tasks such as tracking the target, following the target while avoiding collisions, positioning the UAV and camera for a selfie, improving selfies using Machine Learning algorithms etc. All of the commercially available drones require the user to pilot the drone to some extent. We hope to develop a framework for the DJI Phantom drone which takes a template picture as an input and captures a closely matching selfie of the user using feature extraction, pose estimation and navigation.

Project Objectives

Our objective is to make mobile application for taking selfie using drone within 10 seconds time period. Application must be robust enough to work in real world environment. User can make drone take-off and land using mobile application. User must be asked to select one of the available selfie templates. Then the drone should be wise enough to judge its final selfie location. The drone should reach its final position within 10 seconds and then takes the selfie. After taking selfie, drone should safely return to home. 

Project Implementation Method

Our main goal was to develop mobile application which is capable to take user selfie under 10 second’s time limits. Mobile applications are handier and user friendly as compared to take laptop with you every time you want to take selfie. To serve this purpose we researched and found that DJI is the leading company in Drone manufacturing Industry and they provide full SDK support for application development. DJI acquires more than 70% market of drones [101]. They have different series of drones for different purposes, but Phantom is series is the most famous one for aerial photography. They provide different SDKs such as mobile and onboard SDK. We decided to use their android mobile SDK for the development of android mobile application of selfie drone project. We used DJI Phantom 4 drone which does not require WIFI connection to connect to remote controller or mobile and has good overall performance results.

Selfie capture process through this automated model is as follows:

1. User Selects an Image from templates available on Android Application
2. User issues a capture command from Android Application
3. Drone Takes off with camera facing the human
4. Human is localized in the image frame and his pose estimated
5. Drone is navigated to a new position to match the image to be captured with the selected image template
6. Picture is captured and drone lands to home position

Our approach is to eliminate multiple test step flights from initial drone position to final desired selfie drone position.  Drone should be wise enough to calculate camera position to take selfie like the template image. To serve this purpose we had to develop an algorithm which gives us destination point based on some calculations. For this purpose, two things are very important.

1. Camera Position with respect to human body
2. Human body Posture Information

Human body pose is very important in computer vision field as it plays a vital role in development of many techniques such as for taking selfies. Camera position is very important to take selfies. This project deals with camera position estimation when the camera captures the human image. By using machine learning on human body pose, one can estimate the camera position for any random picture. This project also implements the real-time working of the algorithm as human body extraction algorithms need extensive computation. We wanted to use the image processing to build an image-based visual servoing system to first calculate the human pose parameters from the template we chose. Next the system calculates the human pose parameters in the image from the camera of the drone, then compares the difference of the parameters. Then the algorithm calculates the estimated drone position to take selfie with same effect as that of selected template. Finally, algorithm controlled the drone to fly to the specific position, so that we can take the same effect as in the selected template at the same position, same size, and same angle.

Benefits of the Project

Since this is a project in luxury category, the only benefit of this project is to make the piloting of the drone completely autonomous such that the beginneres are also able to use it.

Technical Details of Final Deliverable

The project requires the drone to take off and detect the face of user. Face recognition is done using Haar Cascade classifier. Afterwards, the pose of user has to be estimated using Pose estimation algorithms. Following are the algorithms that were implemented for detecting the pose and each of them were tested to find out their efficiencies. The output of pose algorithm was the keypoints position data for body parts. This data was used to find the body ratios. Afterwards, we had to generate Drone dataset and Pose dataset and run KRR regressor.

We used our own generated images data set from human model simulator based on OpenGL for testing of algorithm efficiency. We used two different human models with different pose. We used both real world background images and only black background images for OpenCV implementation. Later we used real images dataset taken from DJI Phantom 4’s camera for testing and implementing OpenPose method.  All training images had 720p resolution.

The experiments done on Desktop are performed on i7 2.8 GHz CPU and 16GB RAM. The latter versions mainly the OpenPose and Fritz are implemented on Samsung Galaxy S7.

Final Deliverable of the Project

HW/SW integrated system

Type of Industry

IT

Technologies

Artificial Intelligence(AI), Robotics

Sustainable Development Goals

Partnerships to achieve the Goal

Required Resources

If you need this project, please contact me on contact@adikhanofficial.com