An Intelligent Action Recognition System to assess Cognitive Behavior for Executive Function Disorder

Abstract

This paper proposes a novel intelligent system to monitor and assess cognitive behavior through physical tasks which are part of assessment and training for people with Executive Function Disorder. The tasks are specifically designed to fit in the theory of “embodied cognition”, where cognition can be influenced through physical activities. Usually, these assessments are performed by psychologists who manually monitor and score patients which is tiresome and time consuming. The proposed system automates this process by capturing the minute movements of the subjects, analyzing and predicting the action performed by using state of the art computer vision techniques. Detailed visualization of the user’s performance can be viewed in real time through an intelligent Graphical User Interface(GUI) which also provides support for the expert to view the performance statistics remotely. Data was collected from 5 participants with two variations to quantitatively and qualitatively increase the dataset which was combined with an existing public dataset. The action recognition, which is the core of the system, was developed using multiple algorithms, with 3D Convolutional Neural Network performing the best with a maximum of 80 percent accuracy on test set.