VISION-BASED AFFORDABLE IT SOLUTION FOR INDUSTRIAL PRODUCTION MONITORING AND QUALITY CONTROL

The objective of the project is to develop an affordable priced real-time camera-based visual inspection system for factory environment using off-the-shelf embedded boards/GPUs. The proposed system will provide (configurable) basic vision tools for inspection of packaging and bottling defects (mispl

Project Title

VISION-BASED AFFORDABLE IT SOLUTION FOR INDUSTRIAL PRODUCTION MONITORING AND QUALITY CONTROL

Project Area of Specialization

Artificial Intelligence

Project Summary

The objective of the project is to develop an affordable priced real-time camera-based visual inspection system for factory environment using off-the-shelf embedded boards/GPUs. The proposed system will provide (configurable) basic vision tools for inspection of packaging and bottling defects (misplaced labels, poor print quality, incorrect bar codes, incorrect manufacturing and expiry dates, partial capping sealing etc.) and advanced AI (Artificial Intelligence) trainable tools for identification of visible manufacturing defects like scratches, shape deformation etc. in metal articles or stains, holes etc.  The defected product may be removed by activating a plunger or by releasing air pressure or an alarm can be raised (depending on the product). Furthermore, the data related to normal and defected articles will be wirelessly transmitted to an integrated central production database generally in Microsoft SQL software. This can be used for production planning and defect cause and source analysis.

We have done initial research, experimentation and development on this problem in our internal projects with graduate and undergraduate students. This include two senior projects leading to proof-of-concept setup for box packaging inspection, an internal research paper on metal part defect detection and a literature review on techniques and methods in image processing employed for Industrial Visual inspection.

Our commercial partner is an Industrial automation company with 20+ year experience and is pioneer in projects related to Camera-based Vision projects in national and multi-national industry. They will provide initial training, contribute and support us in development and organize real-life trial testing at the industry.

Our in-house team will initially utilize the open source code and libraries and those provided by the development environment of our selected off-the-shelf board to develop a suite of vision tools for industrial inspection tasks related to packaging. This will serve as our test-bed/platform for further useful work. In the second stage we will conduct more R&D work to add vision tools geared towards identification of manufacturing defects in articles. Trial and testing will be conducted in the University and at the Industrial sites.  

The project has the potential to provide required value addition in line with wishes of industry in a wide segment of industry related to import substitution and export by increasing high quality productivity.

Project Objectives

The objective of the project is to develop an affordable priced real-time camera-based visual inspection system for factory environment using off-the-shelf dedicated hardware (embedded boards/Graphics Processing Units (GPU). The proposed system will provide:

a- In its basic version a suite of computer vision tools for inspection of packaging and bottling defects in factory produced articles on a fast moving conveyor belt in a production/assembly line. These articles may be packaged /contained in paper-board boxes, plastic coated paper-board boxes, plastic pouches & bags, sacks and bottles etc. covering a broad range of industry. Basic defects to be detected include misplaced labels, poor print quality of text, incorrect bar codes, incorrect manufacturing and expiry dates, partial sealing of caps, shape deformations in packaging, open box flaps etc. These tools will be configurable/programmable for the target industry by the project engineer.

b- In its advanced version a suite of AI (Artificial Intelligence)-based trainable tools for identification of manufacturing defects like scratches, shape deformations etc. in metal articles or stains, holes etc. in fabric. 

The system shall connect to up to two cameras for acquiring two different views of the inspected object. It will allow trigger -capturing of images and rejection of faulty articles. Furthermore, the data related to normal and defected articles will be wirelessly transmitted to an integrated central database with current production data generally in Microsoft SQL software. This can be used for production planning and defect cause and source analysis for quality control and production efficiency. A user-friendly programming environment will be developed for the ease of the project engineer.

Project Implementation Method

Methodology

Our developed technology at university

Building on our earlier experience, our approach will be to first focus on finalizing the hardware and ordering it. With regard to hardware platform, the potential embedded boards to consider are Nvidia GPUs (like Jetson Nano), Raspberry Pi 4, ODroid, Latte-Panda or similar boards) with Camera. We have tried out Raspberry Pi and feel Nvidia GPUs will be a better choice for our case given its Computer Vision and AI libraries and Image Processing capabilities. However, this choice will be validated further in the initial phase of the project. For optimal image acquisition we may order more than one type of camera, wide-angle lenses and lighting equipment to allow some experimentation. Possible choices may include Raspberry Pi Camera Module V2, Industrial Digital Camera (Vision Unity Se) etc. that are reasonable in terms of price and performance.

We will develop the basic suite of Visual Inspection tools first by utilizing much of the open source code and libraries (outlined in Literature-review section) and those provided by the development environment of our selected board. In the second stage we will conduct more R&D work to add vision tools geared towards identification of manufacturing defects in articles. In parallel work will undertaken related to communication of collected data and its interfacing with central production databases.

After that we will be creating a User Interface using Web based application for visulizing and testing of our models. Our application will allow us to choose between model type, like if we want to test image for defects or for barcode. Tech. Stack will be React and Django.

Trial and testing will be conducted in the University and at Industrial partners’ site.  For very preliminary testing the image datasets obtained from industry will be acquired and used.  A variable speed conveyor belt assembly setup exists at the university with camera, lighting and photo-sensor installation for triggering and capturing of images. This will be helpful in integrating the Camera with the system and experimenting and finalization of image acquisition and pre-processing task to deal with non-idealities in lighting condition and motion blur. Second stage testing will be conducted at the Sectorial partner site together with his team. Testing in industry will follow this.

Our industrial collaborators

The project will be conducted in collaboration with a local automation consultant that has a 20+ years experience of Pakistani industry. They also serve as a dealer of a renowned Japanese brand PLC and Industrial vision systems. Our industrial partner will be providing us the industrial requirement, access to right equipment to facilitate development and testing at their site, and the contacts for trial testing. Letter of collaboration with Control Experts, Tapal (tealeaf brand) and CBM (Engine oil Can manufacturer) are attached.

Benefits of the Project

Our industry is insufficiently automated owing to the use of earlier generation and imported second-hand machinery, and even modern ones are bought without automation options to reduce purchasing costs. Camera-based visual inspection of product quality and production data collection in a shop-floor production line is of great significance in various industrial sectors like Agri-industry & Horticulture Industry (Pesticides, Tea, Fruit juices), Diary Industry (Milk, butter), Pharmaceutical Industry, Construction related Engineering Industry (screws, metallic parts) and not to say Textile Industry. The food industry has accounted for an estimated average of $223.5m in recent years. The pharmaceutical industry is worth $3.1b.

The development and commercialization of the proposed system is expected to have a very great impact on much of our local industry given the current scenario.

In the quest to improve country’s economy there is great focus on developing our local industry and improving its productivity. For the existing local industrial setups this means reduction of production cost, improving of quality and increase in productivity. Improving automation is the only solution: Our industry is insufficiently automated owing to use of earlier generation and/or imported second hand machineries and even modern ones are bought with limited automation to reduce purchasing cost.

Our key industrial sectors like Agri-industry & Horticulture Industry (Pesticides, Tea, Fruit juices), Diary Industry (Milk, butter), Pharmaceutical Industry, Construction related Engineering Industry (screws, metallic parts) and Textile Industry etc. are based on a conveyor-belt based production line 

or assembly line) system. The last step in all such a industry is product packaging. Though many of the processes are automated the quality inspection in most of the cases is manual and serve as a major process bottleneck. This especially applies to inspection of finished packaged product with respect to its packing defects and even with regard to manufacturing defects. Visual inspection of manufacturing defects is important in metal and textile industry.

However, the industry is reluctant to import and install international branded Camera based visual inspection systems because of its high costs (typically 17 lacs -25 lacs), difficulty in staff training and lack of required customization and compatibility with existing systems. Additional requirements of production data and defect statistics collection ask for more investment and training. Use of locally developed PC-based solutions is limited as they are not rugged for mass scale use.

Our proposed indigenous Camera-based visual inspection of production quality and production data collection for a shop-floor production line (costing below 4 lacs) has the potential to provide the required value addition.

Technical Details of Final Deliverable

Our final deliverable will be based upon software as well as hardware. If we discuss hardware first, we will have these main components working correctly in our final deliverable:

Hardware:

Conveyer Belt, Camera, DC Motor, Monitor for Display, and a Trigger Mechanism. Our camera will be placed near our conveyer belt; whenever the object passes the camera on a conveyer belt, the camera will capture the image whenever the trigger happens. The image will be sent to our web application, which runs on our Nvidia board. After processing the picture, the results will be displayed on the monitor connected to the divided board. All hardware-related things will be attached to the Nvidia board. 

Software:

As we have trained and tested our model, at this stage, our task is to integrate it on the Nvidia board using a web application, for the application's front end, we will be using REACT (an open-source JavaScript library) , and we will have four significant features of frontend:

Model Selection: 

From this dropdown menu, we will be able to select our model type; for example, if we want to barcode, we will like a barcode.

Datasets Selection:

As explained earlier, our model is trained, and weights are stored for two kinds of Datasets Coke dataset and Tapal's (Our industrial Collaborator) datasets. We have to select one dataset so that the consequences will change accordingly on the backend.

Image Type:

This feature will allow us to choose whether we want to test pre-download images or take live photos.

Results:

As a result, there will be a comparison between the original and processed images.

In our backend, we will be using DJANGO because it is a python based backend framework; the model will be integrated there; once our backend receives an image from the frontend, it will pass it to the model and, after model processing, send it back to our frontend, and it will be displayed.

Note that all images will be stored in a database.

Final Deliverable of the Project

HW/SW integrated system

Core Industry

Manufacturing

Other Industries

Core Technology

Artificial Intelligence(AI)

Other Technologies

Sustainable Development Goals

Industry, Innovation and Infrastructure

Required Resources

If you need this project, please contact me on contact@adikhanofficial.com