Smart Shoe: An Obstacle Detection Framework Using Wearable Sensors

Project Title

Smart Shoe: An Obstacle Detection Framework Using Wearable Sensors

Project Area of Specialization Artificial IntelligenceProject Summary

Smart Shoe is an android application which will help visually impaired people walk comfortably. Smart Shoe will be integrated with ultrasonic sensors attached on a pair of shoes. These sensors will be used to sense barriers coming in the way of the user wearing the shoes. The sensors after sensing the obstacle, will notify the mobile application and the mobile application will guide the user to avoid that obstacle through speech. The mobile application will also be integrated with a wearable camera which will detect the obstacles and notify the user about the nature of the object.

Project Objectives

The aims of our project include:

• Helping the visually impaired people walk
• Increasing efficiency of the project in real time environment
• Developing a user-friendly mobile application to be used by blind people
• Developing a pathway for further research in this field
• Encouraging the addition of more features in the project

Project Implementation Method

The project will be composed of the following three major modules:

1. Android Application
2. Arduino with sensors
3. Camera

The first step of the project development phase was android application development. The android application is developed on Android Studio. The main module of android application is adding audio to make it easy to use for visually impaired people. Firstly, the text to speech activity was developed to greet the user and allow him to add google maps using its API. The maps will help to locate the current location of the user and ask the user for the destination location. The location once gotten through the speech of user will be repeated after converting the speech into text and back to speech again for confirmation. Moreover, the firebase will be integrated with the application to store the updated location.

Obstacle detection through Arduino was implemented by burning code in Arduino board, the sensor that was used for detecting hurdles was ultrasonic sensor HCSR04. The other components that were used were male to male and female to male wires, a buzzer to buzz when an obstacle comes near to sensors, a light to indicate the code is running and breadboard to connect all the components firmly. The distance was set to 100cm hence any obstacle coming in 100cm would be detected by the sensors. Right now, we have used single sensors. We plan to use multiple sensors for improved efficiency and improved robustness. Bluetooth module is used to integrate Arduino with the android application.

For making our project more efficient, we integrate our mobile application with the camera and implement computer vision algorithms to allow it to detect nature of obstacles. For this purpose, Darknet framework is used. The Yolo library is used by cloning it from GitHub. The object detection code will help to make bounding boxes labeled with the tag of the object.

Benefits of the Project

Around 285 million people in the world are visually impaired and this project is being developed mainly for completely blind or visually impaired people. It will help those people to aid walking without depending on any other person. The android application will aid them to walk with ease and comfort. Moreover, the project will open new gates in the field of research. And it can be extended by adding a variety of new features, hence the growth of project in the future will be ensured. New features, for example navigation and signboard reading will make this app up for use for not only visually impaired people but also people with good sight. The long and short of this discussion is that the project does not seem to die after some years, rather it seems to grow and help more and more users differently.

Technical Details of Final Deliverable

The final deliverable of this project is a hardware/software integrated system. It is an Android application integrated with Arduino and camera. The android application developed in Android Studio notifies the user after it is turned on through a voice over. The app notifies the user through voice over about the presence of hurdle after getting informed by the Arduino. Arduino has HCSR04 Ultrasonic Sensors connected to it which will detect the hurdles by emitting frequencies which will bounce back after striking an obstacle. Arduino module will be integrated with the Android Application through Bluetooth module. The app will notify the user through voice over about the type of obstacle he is going to face after getting informed by camera capturing a video frame by frame to process the computer vision framework for object detection.

Final Deliverable of the Project HW/SW integrated systemType of Industry Medical Technologies Artificial Intelligence(AI)Sustainable Development Goals Good Health and Well-Being for PeopleRequired Resources

Item Name	Type	No. of Units	Per Unit Cost (in Rs)	Total (in Rs)
			Total in (Rs)	69820
Camera	Equipment	1	55430	55430
Arduino	Equipment	2	1400	2800
Breadboard	Equipment	2	420	840
Wires (Female to male, male to female)	Equipment	30	5	150
Buzzer	Equipment	2	140	280
Bluetooth module	Equipment	2	1120	2240
Battery	Equipment	4	560	2240
Printing	Miscellaneous	5	1000	5000
Binding of Thesis	Miscellaneous	1	840	840