An AI Framework for Speech Recognition based CAMPUS Navigation Platform

Project Title

An AI Framework for Speech Recognition based CAMPUS Navigation Platform

Project Area of Specialization Artificial IntelligenceProject Summary

Speech is the best way of communication among humans. However, due to the diversity of local languages, it becomes difficult to communicate without any hesitation. The speech recognition systems modeled so far are for one or two languages speech recognition. The speech recognition system provides facilities at the places where the person faces difficulty due to the language barrier in an unknown environment. A similar hurdle of multi-language environment and unavailability of direct point of contact also occurs for the parents and other visitors during the Universities Campuses visit. They find it difficult to convey the message and find the location to whom they want to visit. The existing system has no application available to handle multi-language queries and provide the exact path user wants to visit. In this project, the aim is to translate four Pakistani local languages speech to text messages, 2-D digitize the campus indoor map for each department building, and provide a platform for the parents and visitors to enter their queries for the campus indoor navigation. They will get the route marked on the touchscreen along with the voice-based response. Also, the system will track the indoor navigation of the visitors for the security purpose.

The proposed system consists of modules: (1) Speech acquisition and recognition module that will acquire and machine learning-based recognition of the speech in 4 Pakistani local languages (English, Urdu, Punjabi, and Pashto). (2) Digitization Module that will do 2-D digitization of the indoor campus map. (3) Navigation module that will find the target location in the static map as asked by the user. (4) Route selection module, to select the route from source to destination in the digitized campus map (5) Guidance module to display the selected route on the touchscreen monitor. (6) Speech-to-text module for a voice-based response to the user, as per the local language. (7) Security module that will track the user’s navigation.

In the project, Raspberry pi is used for the system integration and to interface microphone, speaker, and touchscreen display monitor. GPS module is required to track the navigation. MATLAB and Python are used for the implementation of the speech recognition-based campus indoor navigation. The dataset of 6.4k speech samples is required to train and test the machine learning algorithm for speech recognition.

Project Objectives

• Speech Acquistion and Recognization Module: Speech to text conversion for 4 Pakistani local languages (English, Urdu, Pushto, Punjabi) using Machine Learning Algorithm
• Digitization of Local Map: Digitization of 2-D map (Campus map) and marking various places on the map.
• Speech to Text Conversion and Route Selection: Finding the Speech converted target location on the map and Route selection from source to target
• Navigation and Tracking: Display navigation on map, and guide the user using voice based navigation
• User Location Tracking: Tracking of user's navigation in the premises for security. 

       

Figure 1. Overview of the Project Objectives:An AI Framework for Speech Recognition based Campus Path Navigation 

Project Implementation Method

In the proposed project, the first step is to acquire speech as an input signal. Microphone will convert the analogue input signal into electrical signal and will be processed by ADC (Analogue to Digital Converter) to convert the analogue signal into digital signal by undergoing the process of sampling and quantization. Suitable sampling rate for human speech frequency range is 16kHz. Next stage is to apply pre-emphasis filtering on speech signal and extract important features from speech signal for analysis. These extracted features then feed up to the machine algorithms for classification from dataset. The outcome we get after classification is the target location of the visitors. This target location is searched in the 2-D digitized map. The locations in the digitized map are labeled and so the classified keywords of the speech input are matched with the labeled locations in the map. Finally, marking and displaying the route from the current location to the target location on touchscreen monitor. Along with the displayed path, voice-based guidance in the local language will be provided, so that those who could not understand the displayed guidance, can benefit from the voice response. Also, for security purpose, visitors tracking is also a part of this project.

Below block diagram and flow chart illustrates the methodology for the proposed system.

Figure 2. Overview of the proposed architecture of An AI Frame for Speech Recognition based Campus Path Navigation Platform

Figure 3. Flowchart Diagram for An AI Frame for Speech Recognition based Campus Path Navigation Platform

Figure 2. Overview of the proposed architecture of An AI Frame for Speech Recognition based Campus Path Navigation Platform

Benefits of the Project

• The Speech Recognition system facilitates indoor navigation in multi-story buildings, where there exists multi-language environment.
• Moreover, project is also applicable for security purposes. Indoor navigation in different buildings can now be monitored.
• Some of the places where this project can be applied are:

1. Universities
2. Banks
3. Shopping Malls
4. Museums 

Technical Details of Final Deliverable

• Technology development using Raspberry-PI platform for speech acquisition and recognition in 4 Pakistani local languages.
  • Sample size for analog to digital conversion (ADC): 16bits
  • Sample rate: 16k
  • Audio Channel: mono
  • Locations: 16 locations within campus
  • Dataset: 64k audios
• AI-based software technology development for speech acquisition and recognition
  • Features extraction using classical and deep learning methods
  • Classification of dataset using Neural Networks, LSTM, and CNN
  • Matching accuracy expected: >90%
• 2-D digitization of 1-D campus academic department buildings maps
  • Georeferencing
• Technological development for text to speech conversion of selected route from source to destination
  • Text to speech synthesis
• Technological development of user’s navigation tracking
  • GPS tracking
• Theoretical analysis and technical reports of implemented systems
  • Graphical and tabular representation of achieved results

Final Deliverable of the Project HW/SW integrated systemCore Industry ITOther Industries Security , Telecommunication Core Technology Artificial Intelligence(AI)Other Technologies Internet of Things (IoT), Augmented & Virtual Reality, Big DataSustainable Development Goals Industry, Innovation and Infrastructure, Sustainable Cities and Communities, Responsible Consumption and ProductionRequired Resources

Elapsed time in (days or weeks or month or quarter) since start of the project	Milestone	Deliverable
Month 1	Hardware Requirement Analysis and Hardware Finalization	Feasibility Report (Completed)
Month 2	Interfacing of Raspberry-PI with microphone, speaker and touchscreen monitor	Developed integrated system with installed OS and required libraries (Completed)
Month 3	On-board platform for speech acquisition and recognition	64k Audios samples (Completed)
Month 4	Achievement of greater than 90% accuracy for feature extraction of voice and conversion to text	A software system capable to classify audios (Completed)
Month 5	2-D digitization of campus map and path finding with greater than 90% accuracy	Software of proposed technology (In Progress)
Month 6	Matching the converted text with the location IDs in the map and selecting the route from source to target using Machine Learning Model	Software of proposed algorithm (In Progress)
Month 7	Conversion of resulted text into speech and on-board platform for voice-based response	Developed integrated system of proposed technology (In Progress)
Month 8	Development of system for tracking of user's navigation	Developed integrated system of proposed technology (In Progress)
Month 9	Complete Hardware and Software implemented system	Raspberry-PI based system for speech acquisition and path guidance along with security checks (In Progress)
Month 10	Documentation	Project thesis
Month 11	Research Paper Writing and Publishing	The Speech Recognition Part of the paper is completed. However, complete paper will be submitted by April to a Journal paper.