android base entity classification app

Project Title

Project Area of Specialization

Computer Science

Project Summary

Entity Recognition or NER for short is a natural language processing task used to identify important named entities in the text -- such as people, places and organizations -- they can even be dates, states, works of art and other categories depending on the libraries and notation you use. NER can be used alongside topic identification. One way is to train the model for multi-class classification using different machine learning algorithms, but it requires a lot of labelling. In addition to labelling the model also requires a deep understanding of context to deal with the ambiguity of the sentences. This makes it a challenging task for simple machine learning. ML is commonly divided into two phases namely the training and the inference phase. Training is the phase where a model, usually a neural network, is trained to behave a certain way based on given datasets. Computer programs that use deep learning go through much the same process as the toddler learning to identify the dog. Each algorithm in the hierarchy applies a nonlinear transformation to its input and uses what it learns to create a statistical model as output. Iterations continue until the output has reached an acceptable level of accuracy. The number of processing layers through which data must pass is what inspired the label deep. In deep learning, a convolutional neural network (CNN/ConvNet) is a class of deep neural networks, most commonly applied to analyze visual imagery. Now when we think of a neural network we think about matrix multiplications but that is not the case with ConvNet. It uses a special technique called Convolution. Now in mathematics convolution is a mathematical operation on two functions that produces a third function that expresses how the shape of one is modified by the other.

Project Objectives

Computational cost time during the recall phase of such a model, as it should be capable of running on a mobile device with limited computational power. As multiple objects might exist in a single frame, the frame must be divided into a grid were multiple cells, where each cell is analysed independently, which inherently increase the computational cost.
The need to distinguish between similar objects.
Project Objectives
Computational cost time during the recall phase of such a model, as it should be capable of running on a mobile device with limited computational power. As multiple objects might exist in a single frame, the frame must be divided into a grid were multiple cells, where each cell is analysed independently, which inherently increase the computational cost.
The need to distinguish between similar objects.
Identification of multiple objects in a single frame, where some objects might be only partially visible, and others are overlapping.
Gathering and pre-processing of training data.

Project Implementation Method

The tools we will use are Jupiter notebook, yolo, Deep-learning . In addition, Android Studio and geometric library such as tensor flow will be used in this project. This application allows to work with multi-class-image classification.

Benefits of the Project

The assignment entails the development of a ML model running on a mobile device capable of detecting Post-it R notes in real-time from a video feed. It will help people to decide about different decoration pieces and furniture styles for their homes by viewing their suitability using a virtual overlay on their respective places.
The need to distinguish between similar objects.
Identification of multiple objects in a single frame, where some objects might be only partially visible, and others are overlapping such as (Table,mango and more different things).

Technical Details of Final Deliverable

In this project we have presented an approach to building a Post-it R note object detection CNN for use on mobile devices. Utilising multiple base models and object detection frameworks, we successfully trained and implemented a variety of models that could be used for real-time detection of notes in an Android application. Using Transfer Learning, the network could be trained on a relatively small amount of data while still achieving high mAP scores.
Our results conclude that ML models are viable for deployment on mobile devices in terms of inference time and accuracy and can provide a heuristic based corner-detection algorithm with bounding boxes of high recall.
The results given in this report emphasises the possibility of using ML models and algorithms for object detection on mobile devices. It is shown that the trained models are feasible for use in a real-time environment on devices with limited computational power while remaining highly accurate and reliable for the end user.
In order to further develop the work presented in this report careful gathering of diversified training data could lead to increased performance, as more images from environments typical to the end use case could increase the performance of the less complex models when detecting multiple notes, as these models might have been limited by the quality of the training data. Furthermore, utilising Capsule Networks [9, 47] instead of CNNs as the base model for the network could extend the image vision capabilities of the model and add improved learning of part-whole relationships and viewpoint variation and thus increase the recall performance of the model.