Clipy Drag and Drop App

Project Title

Clipy Drag and Drop App

Project Area of Specialization Augmented and Virtual RealityProject Summary

We are creating a project in which two Apps (mobile and computer) where computer app will be used to make connections to the mobile app. Where a mobile app will be used to do core functionality. such as. Capturing an object, the captured object will be masked and shown over the camera view. now than when the user points the camera to the computer screen. and drop the object. The Object will be dropped to the exact same point on the screen. from the user can use that object for any certain task.

Project Objectives

The main objectives are as follows:

To introduce a new way of data sharing.

Let the people use the physical object for software usage.

Makes it faster than the traditional data sharing and masking approaches.

Project Implementation Method

Build Connection: we develop the medium of computer basically in this medium we create a server to connect with a mobile device the desktop app build by thePyQt5 we design it by QT Designer. In this medium run the QR code and server and through this QR code Mobile Apps connect
Detect and mask Object: In this stage, we are making our main function that is to capture and mask the object through a mobile camera we capture an object. This functionality is performed by model U^2 net when the application camera capture an object it posts that object to the server where the server uses a remove background algorithm to mask this particular object.
Drop Object: In this Stage, the capture object thrown to the computer when we click an object the computer camera is opens and then the server take screenshot of that picture as well as the application send the object to the server when the server matches both screenshot and forward picture then throw that object where the coordinated are a match

Benefits of the Project

This project can be beneficial in Educational. and Commercial domain. 

Such as most of the time, we have to remove the background of the image. whereas clipy app allows you to remove the background of the image in real-time.

Most of the publications use their research photos, whereas these photos could be used to mask, where clipy app would be the best option. Users can mask all the objects and share them with their computers in real-time. which saves a lot of time.

Clipy allows us to share data in real-time. This approach can be used to implement in different projects such as OCR or Speech to text (from mobile to computer).

Technical Details of Final Deliverable

This project is made on Python and React Native. The Drag and drop feature is designed by using python modules.

The server is made by using Flask.

System UI is created by Qt Designer.

System events functionality is made by using PyQt5, OpenCV, PySide, 

For multi-threading used threading module.

Drag and drop functionality is made by using mouse and keyboard modules in python.

Used remove.bg API.

Mobile App we made on React native.

Tested on expo CLI.

Final Deliverable of the Project Software SystemCore Industry EducationOther Industries Education Core Technology Augmented & Virtual RealityOther Technologies Artificial Intelligence(AI), Internet of Things (IoT), Big DataSustainable Development Goals Quality EducationRequired Resources