Food Parts

Project Title

Food Parts

Project Area of Specialization

Artificial Intelligence

Project Summary

To the best of our knowledge, there is no such Pakistani labelled food dataset to date. We build one such dataset with 20 classes. Chrome extension was used to collect images, which were then verified manually and cleaned through a three-step verification process. Each image was checked by 3 different individuals in succession. The process involved the deletion of irrelevant and low-resolution images along with cropping irrelevant parts in images to enhance the quality of our dataset.

A subset of this dataset is labelled to demonstrate the working of Mask RCNN. The subset contains 154 images with 21 food ingredients categories. These images were labelled using VGG Image Annotator which is an image annotation tool that is used to define regions. We used a mixture of iconic and non-iconic images in our dataset so that the model may generalized better.

Project Objectives

To label Pakistani food dataset for image segmentation i.e. PFDIS-32. The goal is to use an online labelling tool like VGG image annotator to label the ingredients in Pakistani Food images.

Project Implementation Method

A pre-trained models InceptionV3, ResNet50, ResNet101, MobileNet are used to categorize the images accordingly and create a baseline for Pakistani food images. The results will then be compared with the custom-made model. This baseline model will also serve as a base for Pakistani foods. Hyperparameter and number of epochs were fixed for all models to avoid any inequity.

A label is assigned to every pixel in the image in semantic segmentation. Whereas in classification, a single label is assigned to the entire picture. Semantic segmentation deals many different objects of the same class as single entity. On the other hand, instance segmentation treats multiple objects of the same class as distinct individual objects (or instances). Therefore, instance segmentation is more difficult than semantic segmentation. We used state of the art model for instance segmentation that is Mask R-CNN and result can be seen blow.

Benefits of the Project

The work can be utilized in a large number of down-stream applications like recipe generation and calorie calculation.

Technical Details of Final Deliverable

Out come is to produce 2 dataset for image classification PFDIC-20 and segmentation (object detection) PFDIS-32. we will write two research papers and publish them in relevent venues.