RealityScape - Realtime Object Detection with Augmented Reality Annotations
The overall goal of this application is to make it easier to find information about a product, such as specifications, reviews, and where to buy it. With an AR display, a user merely needs to aim their camera at an object to get interactive information in real time. We dev
Project Title
RealityScape - Realtime Object Detection with Augmented Reality Annotations
Project Area of Specialization
Project Summary
The overall goal of this application is to make it easier to find information about a product, such as specifications, reviews, and where to buy it. With an AR display, a user merely needs to aim their camera at an object to get interactive information in real time.
We develop this application by combining machine learning technologies with an iOS mobile application. Our mobile application shall take data from a machine learning model using Convolutional Neural Network and will display it on the mobile application using AR whenever user performs an action.AR annotation here is done by making use of ARkit.
Project Objectives
-
Develop a product to help users make informed purchase decisions. Make it easier to lookup product details with their price.
-
Implement real-time object detection on a mobile device that is efficient and gives satisfying results.
-
Combine AR technology with Machine Learning.
Project Implementation Method
RealityScape is made up of two parts, object detection and AR annotation. Below we briefly discuss the two components of the iOS application that uses Deep Learning to detect objects.
- Object Detection: The user points the camera at the object. Then the frame is sent to the object detector which in turn returns the name and bounding box of the object. This deep learning model is stored on device which in turn allows the user to be able to detect objects without an internet connection. The modeL uses Swift 5 and CoreML.
- AR Annotation: After the object detection is done the bounding box coordinates are then passed to ARKit which draws an AR element containing the detected object’s information
Benefits of the Project
- Users can make informed purchase decisions.
- Easy to use for users when without the need of typing.
Technical Details of Final Deliverable
- iOS application able to detect object and annotate them in Augmented Reality.
- A Custom Dataset of 12 Products (3000 Images).
- Deep Learning Tiny-YOLO model.
- Augmented Reality graphic element.
Final Deliverable of the Project
Core Industry
Other Industries
Core Technology
Other Technologies
Sustainable Development Goals
Required Resources
| Item Name | Type | No. of Units | Per Unit Cost (in Rs) | Total (in Rs) |
|---|---|---|---|---|
| Mobile Phone | Equipment | 1 | 35000 | 35000 |
| Training ML Model Service | Miscellaneous | 1 | 5000 | 5000 |
| AR element Resources | Miscellaneous | 1 | 5000 | 5000 |
| Total in (Rs) | 45000 |
If you need this project, please contact me on contact@adikhanofficial.com
Event Guider
Event guider is an application which is specially designed for the purpose of guidance for...
A Solar Based Electric Car with GSM based Tracking system
Project Implementation Method (less than 2500 characters)
Smart Speaking System for Deaf and Dumb
We are creating the Smart speaking virtual gloves, this device is for deaf and dumb. In ou...
Online Fraud Detector using Cutting Edge Technology
This Project detect the frauds of online payment and it will quite beneficial for the comi...
Indigenous Development of Learning based Rotational Solar Charging Sta...
Introduction: The commercial and domestic markets energy demands have been reported to be...