Standalone Assistive System to Employ Multiple Remaining Abilities in People with Tetraplegia

People with motor disabilities affecting their four limbs (e.g. tetraplegia, ALS) can use their remaining abilities such as tongue, head, and eye motion to interact with devices, such as PC, smartphone, and wheelchair. Most of the existing assistive technologies (AT) rely on a single remaining ability, which is typically insufficient when performing complex computer tasks such as “drag and drop”, typing long sentences, or selecting multiple items. In this work, a multimodal AT is presented to leverage both tongue gestures and head motion, simultaneously, to interact with target devices at latency and accuracy of 10 ms and 95.9%, respectively, measured among 15 able-bodied participants. A wearable headset transmits commands wirelessly via Bluetooth Low Energy (BLE), utilizing a human interface device (HID) protocol for seamless interfacing with various applications running on PCs and smartphones without requiring a custom driver.