Automatic image-based electrical energy meter reading using deep learning

Project Title

Automatic image-based electrical energy meter reading using deep learning

Project Area of Specialization Artificial IntelligenceProject Summary

We propose to perform image-based automatic electrical energy meter reading. The major region where the display is located is the most important as it shows the reading of the energy consumption. Our aim is to extract these regions from the image of a meter and then recognize the digits and characters in the extracted regions in order to record both the readings. However, such region extraction and character recognition become challenging due to image variations caused by several factors such as partial occlusion due to dust on the meter display, orientation and scale variations caused by camera positioning, and non-uniform illumination caused by shades. To this end, we propose to use state-of-the-art deep learning algorithms such as YOLO (You Only Look Once) for text detection and extraction. This will be followed by applying optical character recognition (OCR) of the Tesseract library. Our image dataset contains 20,000 images of electrical energy meters that will further be expanded by data augmentation such as in-plane rotation and scaling in order to make the deep learning algorithms robust to such image variations. For training and evaluation, the image dataset will be annotated to produce the ground truth of all the images using the Microsoft VoTT annotation tool. The method in this project can be highly instrumental in reducing the time and effort involved in the current meter reading process where the workers visit door to door, take images of the meters, and extract the readings from these images manually.

Project Objectives Objectives

The following are the objectives of our proposed system: 

• Electrical energy meter image data collection
• Image preprocessing such as re-scaling and aspect ratio correction
• Image annotation for ground truth generation using image annotation tools
• Training deep learning models over the processed dataset using different state-of-the-art algorithms
• Testing the trained model using test and random data inputs
• Performance evaluation of deep learning models i.e different version of YOLO, R-CNN, Faster R-CNN, etc. in OpenCV and Python
• Interfacing and running Raspberry Pi with the trained model and a camera module for image capturing
• Using the Raspberry Pi to perform the data extraction from meters
• Storing the data in a record file

Project Implementation Method

The following are the steps for implementation:

1. Electrical energy meter image data collection

The images of electric meters are to be collected from residential areas of all types of meters, the project is focusing on only digital meters and analog meters are not part of the dataset collection. Pakistan has multiple manufacturers making electric meters, the different digital meters are part of the dataset and they are not further divided into type classes. The dataset has 20k images.

2. Image annotation for ground truth generation

Training a deep learning model requires the ground truth for the model to train upon, for this, the dataset is manually labeled by human labor by drawing bounding boxes around the display of the meter and the digits. The labeling has two parts, which technically splits the datasets into two. The first dataset that the model is trained upon only has the bounding boxes drawn around the display in the image of the meter. After the display is cropped out using the deep learning model, those pictures are further labeled for digits, dividing them into further 20 classes. These include 0-9 digits and then the digits that have the decimal point next to them like 1., 2., 3. etc.

3. Images pre-processing (re-scaling and aspect ratio correction)

The pre-processing of the images is necessary for model training as the model needs a normalized dataset where the images have a consistent resolution throughout and correct aspect ratios. The images in this dataset are resized to 416x416 pixels before input to the training process. Using Microsoft VoTT, the dataset is exported to a specific format which can be fed into the network.

4. Performance evaluation of deep learning models

Once the model is trained on the YOLOv3 algorithm, the same dataset is used to train other deep learning models such as the variant of  YOLO, R-CNN, Fast R-CNN, and Faster R-CNN to compare the performance of similar state of the art models for best results.

5. Raspberry Pi Implementation

The embedded part of the project involves the implementation of the trained models on Raspberry Pi, once the models are trained a pipeline of the trained models will be transferred to the Raspberry Pi which will run the models using a Python script and inquire the user for an input image, the image goes through the pipeline of display recognition, then the extraction of the reading from the display and giving a final string of numbers to the user as the reading. This reading will be stored in an excel file that is output by the Python script.

Block Diagram

Raspberry Pi Implementation Flow Chart

Benefits of the Project

The project scope targets the current meter reading process used by WAPDA, it aims at automating the process of meter reading, some of its benefits include:

• Reduced labor and effort
• Less prone to human-made errors during meter reading 
• Reduced time consumption, as this is a very time-consuming and tedious process that takes days to get completed.
• Automatic extraction of meter data
• Embedded Raspberry Pi device to perform all meter reading tasks in one place.

Technical Details of Final Deliverable

The final deliverable will include the following:

• A processed and annotated dataset of 20k images of Pakistani electric meters
• A trained deep learning model for display localization
• A trained deep learning model for digit localization and extraction
• A Raspberry Pi with trained models, interfaced camera module, and Python scripts running to perform the whole operation of the system

Final Deliverable of the Project HW/SW integrated systemCore Industry Energy Other Industries Education , IT Core Technology Artificial Intelligence(AI)Other Technologies Internet of Things (IoT), Cloud Infrastructure, Big DataSustainable Development Goals Decent Work and Economic Growth, Industry, Innovation and Infrastructure, Responsible Consumption and ProductionRequired Resources

Item Name	Type	No. of Units	Per Unit Cost (in Rs)	Total (in Rs)
			Total in (Rs)	27900
Raspberry Pi 4 - 4GB	Equipment	1	17000	17000
Camera Module for Pi	Equipment	1	700	700
SD Card 32 GB	Equipment	1	1400	1400
Card Reader	Miscellaneous	1	500	500
Heat Sinks	Miscellaneous	1	300	300
Casing + Fans	Miscellaneous	1	1000	1000
Other Expenses (Printing, Posters, Documentation)	Miscellaneous	1	7000	7000