Voicy

Problem Statement The world is growing in technology. The consumer wants ease and comfort in every aspect of life. In this world of technology, we are experiencing a new era of Internet of Things (IoT), where many elec

Project Title

Voicy

Project Area of Specialization

Internet of Things

Project Summary

Problem Statement

The world is growing in technology. The consumer wants ease and comfort in every aspect of life. In this world of technology, we are experiencing a new era of Internet of Things (IoT), where many electronic devices are interconnected by a network. The consumer wants a full control of his home in his hands.

Security has always been a major issue in conventional homes.  

Introduction

We are experiencing a new era of Internet of Things (IoT), where many electronic devices surrounding us are interconnected by a network. 

Our proposed system is a home automation system. The idea of this project is to build a home automation system based on natural language. Users will be able to control the home appliances through natural language. For example, turn ON/OFF the AC, light or fan using voice command. Google Home Mini will be used as a voice capture and voice recognizer device as well as will control the appliance. An android application will be used to remotely control the home appliances. The appliances connected to our smart home system will be able to be controlled using natural language as well as remotely via mobile application and email from any room in the home and from any location in the world through internet. 

A security system will be introduced which will open the main door when it recognizes an authorized person, using facial recognition and machine learning.

Project Objectives

Objectives:

• To provide comfort and ease to the user to control their home in a smart manner using natural language, remotely via mobile application and email.
• To save money, energy and time.
• To get robust security and safety.
• To handle regular household tasks.
• To make the home intelligent and more entertaining.

Project Implementation Method

Implementation:

Controlling: 

In our project there are three methods to control the home appliances.

 1) A mobile App.

 2) Google Assistant for voice control.

 3) Email method. 

1) Mobile App:

    In this user may use his/her mobile app to control the home appliances. All the home appliances are connected to Raspberry Pi via relay module. Raspberry Pi and our app both are connected to Google Firebase. When ever user want to ON/OFF or change the status of an appliance the command will travel through Firebase and will arrive to Raspberry Pi, then Raspberry Pi will give command to relay module and an appliance may ON/OFF.

2) Google Assistant for voice control:

    In this user may ask to Google Assistant to control the home appliances. Google Assistant is connected to Dialogflow and Dialogflow is connected to Firebase. When ever user ask Google Assistant to ON/OFF or change the status of an appliance the Dialogflow will understand the natural language using machine learning and accordingly will give the command to Firebase then command will arrive to Raspberry Pi, then Raspberry Pi will give command to relay module and an appliance may ON/OFF.

 3) Email Method:

     In this user may have to type an email to control the home appliances. There will be per-defined strings on Raspberry Pi, that will determine the status of appliances. Raspberry Pi will frequently check for new mails of home members. If there will be new mail Raspberry Pi will change the status of appliance accordingly.

Doorbell and Security: 

If a person comes to door and ring the bell Raspberry pi camera captures the image and perform facial recognition on the image. Here we are using machine learning to identify the correct person. So if the person belongs to home or is a home member the door may automatic unlocks for him/her without disturbing the other home members. So if the person is an outsider then camera will send the image of the person to host via email or host may receive a notification on his/her app. Host can also talk to the person at the door from any where in the world (Intercom). To make calls from Raspberry Pi to host's mobile we are using Asterisk.

As we know security is very important factor, so for more security we are using a random string of 4 alphabets. Each user has unique string that he/she has to speak on main door to unlock it. So the string has used it will expire and a new random string will be given to him/her via email or he/she may receive a notification on app for next entry. 

                                                                                                                                  Data Flow Arcitecture

Benefits of the Project

Benefits:

• Gives the full control of the home appliances in the user's hand.
• Provides the full monitoring of home appliances in the user's hand.
• Provides the control of home appliances to physically disabled person through natural language.
• Provides the option to control home appliances via email using any device, i.e, laptops, personal computers, etc.
• Provides home security with facial recognition using machine learning.
• Allows authorized person to enter in the house without disturbing the comfort of other home members.
• Allows authorized people to keep hands and pockets free from door keys.
• Allows to respond to any person on the door with pre-recorded voice messages.
• Allows host to communicate with any person who comes at the door by talking to him/her from anywhere in the world (Modren Intercom).

Technical Details of Final Deliverable

Technical Details:

  Technically We divided our projects into two major parts.

1) Controlling Appliances  or Automation.

2) Security and Doorbell.

1) Controlling Appliances  or Automation:

    This portion is further divided in to three parts.

    i) Mobile App.

    ii) Natural Language. 

    iii) Email.

    i) Mobile App:

        We will develop an Android app in android studio that will be connected to Firebase. So if user wanted to change the status of  an appliance he/she will use the app. To program the Raspberry Pi we will be using Python3. Using Python script                          Raspberry Pi will connect to Firebase to receive the user commands, to ON/OFF the appliances accordingly.

    ii) Natural Language:

         We will use Google Home Mini to interact with user via natural language. Google Home Mini will be connected to a Wi-Fi. If a user will say 'Ok Google' Home Mini will start listening to him/her. We will set Google Actions and connect it to Raspberry Pi          using Json code. Using Machine Learning Dialogflow will be trained on the strings or sentences that will use to control appliances. After listening to user Google Assistant will perform speech to text and pass that text to Dialogflow. Dialogflow kow check the      text using machine learning and compute results according to user commands. Webhook is connected to Dialogflow and Firebase, now Webhook will pass the computed results of Dialogflow to Firebase and changes will be done accordingly on Firebase.        Then Firebase will give a command to Raspberry Pi and appliance will ON/OFF accordingly.

    iii) Email:

         A Python3 script will be use to receive an email form authenticated user. The script will frequently check for new mails and on new mail action will be done accordingly.

                                          

                                                                                                                                                                  Fig1: Architecture of Final Deliverable 

 

2) Security and Doorbell:

 For doorbell we are using Python3 script and Asterisk.If an unknown person rings the bell Raspberry Pi camera will capture image and send to the host via email or mobile app. Host can also talk to the person at the door from anywhere in the world, for that we will use voice over IP calling.

So if the person at the door is known the door will auto unlock. For that we are using machine learning. We will take the different images of each user and train our machine on that dataset. To train our model we will use deep learning. In the training of our model we will encode face landmarks using OpenCV, face_recognition libraries in Python3. After encoding .pickle file will be generated, now our model is ready, This file will be use for face recognition.

Our system will also be able to detect spoof attacks. The system will rely on the combination of two different color spaces and will distinguish from a bona fide image and an image attack, see Fig2.

For more robustness we will use a random string of 4 alphabets, that will be unique for each user and on each entry he/she will be given a new string. User will speak the string and Google Cloud API will perform speech to text, that text will check for authentication.

                                                                                              

                                                                                                                                                                                Fig2: Spoof Detection Flow Chart

                                                                                                                                                                                                                                           

Final Deliverable of the Project

HW/SW integrated system

Type of Industry

IT , Security

Technologies

Artificial Intelligence(AI), Internet of Things (IoT)

Sustainable Development Goals

Industry, Innovation and Infrastructure

Required Resources

If you need this project, please contact me on contact@adikhanofficial.com