VIRTUAL REALITY BASED TELEPRESENCE PLATFORM

The concept of telepresence has emerged in every field for communication. The main drawback that could be found in the current technology is its non-immersive experience and person dependency. In this project, we propose a virtual reality-based 3D telepresence system that enables the user to view, p

Project Title

VIRTUAL REALITY BASED TELEPRESENCE PLATFORM

Project Area of Specialization

Augmented and Virtual Reality

Project Summary

The concept of telepresence has emerged in every field for communication. The main drawback that could be found in the current technology is its non-immersive experience and person dependency. In this project, we propose a virtual reality-based 3D telepresence system that enables the user to view, perceive and move in the remote environment virtually. The system comprises a remote module and the local module. On the local user side, a virtual reality headset is used to have a 3D view of the remote location using stereoscopic vision. An IMU sensor is attached to the headset which provides the pitch and yaw data to record the head-movement data of the local user. The head-movement data at the local user side is sent to the raspberry pi at the remote site via a network, which in turn provide these values as an input to the servomotor for the movement of stereo cameras. In that way, the local user can perceive the remote environment using VR with 360-degree view and can freely move inside it. The locomotion of the UGV at remote location is performed by the local user wearing VR device.

Project Objectives

We aim to develop a system which takes the virtual reality technology to the next level by applying the principles of 3D telepresence and to develop the sense of immersion in the communication.

To design the desired system the key objectives are stated as follows:

1-Develop stereoscopic vision using a pair of regular cameras.

2. Design and develop the mechanical assembly for the pan/tilt movement of cameras using a microcomputer and servomotors.

3-Develop python-based software applications for remote and local sides, process movement data, and render and display stereo video stream.

4. Provide connectivity between the remote module and local module to transfer the data stereo video stream between the local and remote modules.

Project Implementation Method

The project is divided into two modules, a remote module, and a local module. Both modules contain hardware and software applications. The local module consists of the VR device (Oculus or HTC VIVE) connected to the computer. A python-based application will be developed to receive the video data from the remote module and display it on the VR device in real-time. The local app will also allow the recording through the IMU sensor and sending it to the remote module. The remote side will consist of a micro-computer (raspberry pi), a pair of cameras for stereo vision, a mechanical assembly with servomotors for yaw, pitch motion and Unmanned Ground Vehicle (UGV) for movement at the remote location. The UGV, cameras, and servomotors are connected to micro-computer. The microcomputer calculates the change in the angle and sends the angle data to the servo motor which moves accordingly and which in turn moves the cameras mounted over the servo system. The cameras capture the images of the environment and send the image frames to the user after being converted into stereoscopic 3D images. The sense of immersion is achieved through these stereoscopic images.

Benefits of the Project

1. 3D Perception: will give you a real-life view as we see with our own eyes will make us feel we’re really present at that place
2. Cost Effective: The system can save travel cost by allowing the user to stay at his/her place and visit a remote place virtually without spending money every time.
3. Time-Saving: Saves the time spent in physically visiting a remote place.
4. Quality education: Can enable the students to have the exposure and visits of remote places for study purposes
5. It provides improved communication experience through immersion (enhanced version of video calling).
6. Health Safety: It is possible to visit a hazardous place virtually using the proposed system

Technical Details of Final Deliverable

The final system will be demonstrated as a hardware and software system. The local side will consist of a VR headset with an IMU sensor connected to a PC running a custom-developed software application developed in python. The remote site featuring high torque servo motors having ±90 degrees of rotational motion integrated with a pan-tilt servo assembly, which will have cameras mounted over it for providing a stereoscopic 3D vision of the environment. Both modules are connected through LAN/Internet and exchange data via the socket programming (same network).

Final Deliverable of the Project

HW/SW integrated system

Core Industry

IT

Other Industries

Telecommunication

Core Technology

Augmented & Virtual Reality

Other Technologies

Wearables and Implantables

Sustainable Development Goals

Good Health and Well-Being for People, Quality Education, Decent Work and Economic Growth, Climate Action

Required Resources

If you need this project, please contact me on contact@adikhanofficial.com