VIRA Virtual Reality Telepresence Robot

A problem exists in the remote locations of our country that lack qualified medical staff. Qualified doctors cannot physically access these locations regularly and require a way to remotely consult staff and examine patients. Vira is a Telepresence Robot designed specifically for Health care workers

Project Title

VIRA Virtual Reality Telepresence Robot

Project Area of Specialization

Robotics

Project Summary

A problem exists in the remote locations of our country that lack qualified medical staff. Qualified doctors cannot physically access these locations regularly and require a way to remotely consult staff and examine patients. Vira is a Telepresence Robot designed specifically for Health care workers. The robot will be placed in a remote location and the doctor can access that location from anywhere in the world using VIRA. It will give VR experience and complete motion control to its operator (the doctor) allowing them to have a ‘virtual presence’ at that location. The robot will include different feaures to provide an immersive telepresence experience.

Project Objectives

Objectives

The objective of our project is to enable doctors to examine patients and confer with healthcare workers located in remote places, and to reduce the risk of infectious disease transmission to frontline healthcare workers. In addition, our project also aims to reduce the risk of infectious disease transmission to frontline healthcare workers and doctors. There is a lack of qualified medical staff in the remote areas of our country. VIRA will enable doctors to examine patients and confer with healthcare workers located in these areas and will reduce the risk of infectious disease transmission to frontline healthcare workers. In our country there is no such facility that enable doctors to provide treatment to those people who are located in remote places. This accessibility is especially important in rural areas where patients may have to drive hours for a relatively simple doctor's appointment. Many patients may not show up for appointments or put off making an appointment. Missed or delayed appointments can cause otherwise treatable conditions to become more severe than if they were detected early through regular visits. The ability to diagnose and treat problems before they become serious addresses the solution to reduce hospitalizations and readmissions through robot. Even if surgery is required, many of the pre- and post-operation appointments can be handled with care-at-a-distance through VIRA, reducing the patient's costs for those visits as well. Sometimes there is emergency in remote places and there is no one for them to look after because of distance. These people have to suffer a lot because of considerate able distance. Moreover, it is also not possible for rural areas people to go to hospitals for daily check-sup as it is very costly and time consuming. VIRA provide a way to cater all these problems efficiently. It provides a solution that falls into remote patient monitoring allow healthcare providers to track a patient condition from a distance. This makes it easy for doctor to watch patient’s condition and intervene in patients who are at health-risk or are recovering from recent surgery.

As there is lack of qualified medical staff in remote areas of our country and there are some old age people who have mobility issue, VIRA allow them to see the doctor from home. Doctors cannot physically access these locations regularly and require a way to remotely consult staff and examine patients. The doctors can thus discuss mode of treatment and can communicate with patients remotely. This helps to improve the efficiency of medical diagnosis and treatment plans during non-life-threatening emergencies. This project has contributed to the doctor-robot interface where the doctor/user can control the robot reliably via regular internet connection from a different location.

Project Implementation Method

The project is implemented using Raspberry pi as the main processor. The following points enlists the methodologies used to implement the various features of VIRA.

1. One-way video communication takes place between the robot and user.The video is transmitted from the raspberry pi camera to the user's cell phone. The user's cell phone is configures to display the received video in dual screen mode and placed in the VR head set which is then wore by the user.
2. Two-way audio communication takes place between the robot and user. A microphone is mounted on the robot that capture's audio from it's environment and sends it over the Wi-Fi to the user's cell phone. Similarly, audio is captured by the user's cell phone and sent to the raspberry pi and played on a speaker mounted on the robot.
3. User's Head tracking is done to enable the user to have 360 degree view and to make the VR experience highly immersive. gyroscope and accelerometer data is used to detect the direction and speed of user's head movement. This data is used to move the pi camera at the robot with the same speed and toward the same direction. The pi camera is therefore, configured to follow the user's head movement giving the user an effective telepresence experience.
4. The navigation system comprises of a gyroscope-based hand-gesture recognition glove. The arduino on the glove will send gyroscope data to the arduino on the robot. The arduino will then process this data to recognize the data as a navigation command (forward, backward, right, left) and send commands to the motor driver to control the motors and move the robot in the respective direction.
5. Lastly, the project will include a pop-up data feature. This feature is software-based and it involves the recognition of the patient in view of the robot and displaying his/her data in the form of a pop-up to the user. All patient data will be previously stored in the Raspberry pi. When a particular patient will be detected via number plate detection, the corresponding data will be displayed in VR.

Benefits of the Project

The people living in remote and rural areas of Pakistan need VIRA to gain access to proper healthcare. Our idea will benefit the people located in remoted places who do not have access to basic primary healthcare services. VIRA is designed with applications focused on the medical field:

• Doctors can treat patients who located in remote places, far away from local health facilities or in areas of with shortages of medical professionals. Rural populations are more likely to have to travel long distances to access healthcare services, particularly subspecialist services. This can be a significant burden in terms of travel time, cost, and time away from the workplace. In addition, the lack of reliable technology is a barrier to care. In urban areas, people can somehow manage to get to medical appointments; however, it is not possible for rural areas people to visit doctors daily. Rural communities often have more elderly residents who have chronic conditions requiring multiple visits to outpatient healthcare facilities and if there is emergency situation in remote location and healthcare staff who is present in that location does not know how to deal with patient , then our doctor does not have to go in that area instead he can instantly access that place using VIRA , doctor can move around his own will, he can examine patient , he can give diagnosis to patient and give consultation to doctor on how to deal with the patient.
• Reduce the risk of infectious disease transmission to frontline healthcare workers by making it possible to triage, evaluate, monitor, and treat patients from a safe distance especially during covid-19. Health-care workers are crucial to any health-care system. During the ongoing COVID-19 pandemic, health-care workers are the one that are becoming infected day by day because they have to deal with the patients with a very close distance. Due to this, they can easily get infected. VIRA prevents this from happening. Major tasks such as monitoring and post check up can perform from a safe distance through VIRA, this will reduce the number of times that frontline healthcare workers coming in contact with patient.
• Doctors can consult junior doctors through VIRA. For instance, if junior doctor is performing surgery and he requires consultation for some specific purpose, then he can consult to senior doctor from any part of world through VIRA.
• Doctors may be logged onto a robot and performing rounds in patient rooms within a hospital. In hospitals robot can move into one of the any room to be able to bring doctors in to participate in diagnosis and treatment from wherever they are. With VIRA caretakers can interact with their patients, check on their living conditions and the need for further appointments. This would help efficiency a great deal by eliminating time-consuming room visits of patient on daily basis especially in large scale hospitals.

Technical Details of Final Deliverable

1. Gyro-scope based glove: The gyroscope based glove consists of an Arduino nano ATmega328, 433 MHz LoRa SX1278 module and an MPU6050: Gyroscope + Accelerometer. The circuit on the robot body that corresponds with the glove circuit also consists of a LoRa SX1278 module and an arduino nano. The Arduino on the glove captures the data from the MPU6050 sensor and sends it to the arduino on VIRA via the LoRa module. The two LoRa modules are used for communication between the two arduinos. These modules are choosen because of their ability to transmit small data over long ranges. The data received by the Arduino on VIRA will be used to conrtol the motion of the geared motors and wheels using L298N motor driver IC.
2. Video & Sound System: The video and audio system is based on the Raspberry pi 4 Model B, the raspberry pi camera  module v2.1 8MP, A microphone and speaker. The soundwire server and mobile app are used for audio communication from VIRA to user. Shairport and tuneblade are used for audio communication from the user to VIRA. 
3. 360 degree view: The IMU application is installed in the user's cell phone to send sensor data to the raspberry pi. The Raspberry pi uses this data to control the pan and tilt motion of the pi camera via two sg90 servo motors. 
4. Chassis: The chassis of the robot is made out of aluminium. It is a 4-wheel chassis and includes a 3-feet aluminium bar on which the camera amd servos are mounted. The overall height of VIRA is approximately 4 feet so that the user does not feel a significant change in the view point and the telepresence experience is realistic. The Structure of Vira also comprises of acrylic sheets to give it strength and agility.
5. Pop-up Data: the pop-up data feature is being developed using OpenCV. All patient data will be stored in an excel sheet with a unique serial number associated with each patient. All the patients' beds will have their serial numbers displayed on a number plate. The raspberry pi will constantly capture frames and perform number plate detection, whenever a serial number is detected, it will be compared with the previously stored serial numbers. If a match is found, the data for that patient wil be included into the video being transmitted in the form of a pop-up. When that video is displayed in dual-screen mode, it wil appear in the VR environment. 

Final Deliverable of the Project

HW/SW integrated system

Core Industry

Medical

Other Industries

Others , Health

Core Technology

Robotics

Other Technologies

Internet of Things (IoT), Augmented & Virtual Reality, Wearables and Implantables

Sustainable Development Goals

Good Health and Well-Being for People, Industry, Innovation and Infrastructure

Required Resources

If you need this project, please contact me on contact@adikhanofficial.com