Neural network for in-sensor time series recognition based on optoelectronic memristor

Abstract

In recent years, inspired by multifunctional image sensors, in-sensor computing technology that combines sensing and computing functions has become a new research hotspot in the field of machine vision, which is an extremely promising way to break through the Von Neumann architecture by equipping the sensing unit with the computing ability and avoiding the data moving in the computation process. Whereas most existing in-sensor computing systems can only realize the processing of spatial frames in-sensor and cannot fuse the time series information. In order to solve this limitation and realize the processing of time information and spatial frames in the sensor at the same time, it is necessary to decouple and process the information in the processing unit in the sensor. In this paper, a time series recognition neural network based on optoelectronic memristor arrays is proposed. By using the optical plasticity and relaxation effects of the optoelectronic memristor arrays and based on the in-sensor computing technology, the information timing decoupling, processing and recognition in the sensor are realized. The results show that the network achieves a time series recognition accuracy of 98.4 % with two frames of image input, and the recognition rate still reaches 90 % after weight quantization and the addition of 40 % noise.