Qualitative spatial and temporal reasoning over diagrams for activity recognition

In quest for an efficient representation schema for activity recognition in video, we employ techniques combining diagrammatic reasoning (DR) with qualitative spatial and temporal reasoning (QSTR). QSTR allows qualitative abstraction of spatio-temporal relations among objects of interest; and is often thwart by ambiguous conclusions. 'Diagrams' influence cognitive reasoning by externalizing mental context. Hence, QSTR over diagrams holds promise. We define 'diagrams' as explicit representation of objects of interest and their spatial information on a 2D grid. A sequence of 'key diagrams' is extracted. Inter diagrammatic reasoning operators combine 'key diagrams' to obtain spatio-temporal information. The qualitative spatial and temporal information thus obtained define short-term activity (STA). Several STAs combine to form long-term activities (LTA). Sequence of STAs as a feature vector is used for LTA recognition. We evaluate our approach over six LTAs from the CAVIAR dataset.