Teleoperation in the Age of Mixed Reality: VR, AR, and ROS Integration for Human-Robot Direct Interaction

Robotic systems that can be operated remotely are becoming more and more common in a variety of industries, including construction, disaster relief, space exploration, and industrial applications. Teleoperation, however, necessitates skilled operators who can handle sophisticated data processing and programming. This research introduces a unique teleoperation method that makes use of Virtual Reality (VR) and the Robot Operating System (ROS) to overcome this problem. By creating a 3D representation of the robot’s workspace in Unity and connecting it to the actual robot with ROS, this technique optimizes control. Programming is not necessary because the robot’s vision serves as the foundation for the user interface. Through ROS integration, a target point is established in Unity and sent to the robot as the end effector position. By simply setting up a new environment, reducing the requirement for knowledgeable operators, and improving situational awareness, this method may be used to any sort of robot. For even greater situational awareness, this technique may also be used with augmented reality (AR). AR can give the operator real-time information about the robot’s surroundings by superimposing digital information over the physical world. This can be achieved by including AR markers in the physical surroundings that the VR system can detect. In the operator’s field of view, the system can then superimpose information such as sensor data or robot status onto the real-world environment.