Bimodal Coupling Haptic Perceptron for Accurate Contactless Gesture Perception and Material Identification

A bimodal coupled multifunctional tactile perceptron for contactless gesture recognition and material identification is proposed to address the challenges posed by limited functionality, signal interference from multimodal collaborative work, and the high power consumption of traditional tactile sensors. This perceptron integrates a capacitive sensor and a triboelectric sensor symmetrically, employing an energy complementarity strategy to reduce power consumption and implementing symmetrical distribution of two sensors for physical isolation to prevent signal interference. The capacitive sensor detects external pressure, providing information on material properties such as hardness, softness, and deformation, with a wide linear response range of 0–745.3 kPa. The triboelectric sensor captures the electron affinity of measured object. Further, by utilising machine learning algorithms, a system for contactless gesture recognition and material identification is engineered. This system demonstrates a remarkable accuracy rate of 98.5% when recognising 5 gestures, and achieves a perfect identification (100%) of 10 different materials aided by incorporating capacitive and triboelectric response. These results greatly advance the progress of tactile perceptrons with high integration, low power consumption, and multifunctionality, enhancing their effectiveness and reliability in smart device applications.

Graphical Abstract