Learning Behaviors of and Interactions Among Objects Through Spatio–Temporal Reasoning

Abstract

In this paper, we introduce an automated reasoning system for learning object behaviors and interactions through the observation of event sequences. We use an existing system to learn the models of objects and further extend it to model more complex behaviors. Furthermore, we propose a spatio-temporal reasoning based learning method for reasoning about interactions among objects. Experience gained through learning is to be used for achieving goals by these objects. We take The Incredible Machine game (TIM) as the main testbed to analyze our system. Tutorials of the game are used to train the system. We analyze the results of our reasoning system on four different input types: a knowledge base of relations; spatial information; temporal information; and spatio-temporal information from the environment. Our analysis reveals that if a knowledge base about relations is provided, most of the interactions can be learned. We have also demonstrated that our learning method which incorporates both spatial and temporal information gives close results to that of the knowledge-based approach. This is promising as gathering spatio-temporal information does not require prior knowledge about relations. Our second analysis of the spatio-temporal reasoning method in the Electric Box computer game domain verifies the success of our approach.