A real-time neuro-robot system for robot state control

Abstract

Embodying an in vitro biological neural network (BNN) with a robot body to achieve in vitro biological intelligence has been attracting increasing attention in the fields of neuroscience and robotics. As a step forward toward this aim, here we propose a real-time neuro-robot system based on calcium recording, which consists of a modular BNN and a simulated mobile robot. In this system, the neural signal of the BNN is recorded, analyzed, and decoded to control the motion state of the mobile robot in real-time. The sensor data of the robot is encoded and transmitted to control an electrical pump. The electrical pump is included in the system to estimate the real-time performance of the system. An obstacle avoidance task is chosen as proof-of-concept experiments. In the experiments, a calcium recording video of a BNN is replayed to emulate the real-time video stream. The video is monitored and analyzed by a custom-made graphical user interface (GUI) to control the robot motion state and the electrical pump. Experimental results demonstrate that the proposed neuro-robot system can control the robot motion state in real-time. In the future, we will connect the electrical pump to the BNN and transmit the signal from the robot to the BNN by applying local drug stimulation, therefore realizing a closed-loop neuro-robot system.