Reflected Reality: A Mixed Reality Knowledge Representation for Context-aware Systems

Ambient systems and agents in human-agent shared environments require a great amount of contextual knowledge to successfully handle dynamic situations. The generation, integration, and processing of various forms of knowledge, such as semantic and spatial representations, pose a challenge in intelligent systems. To address this, we integrate real world information into a virtual world using a Reflected Reality metaphor, a combination of smart spaces and intelligent virtual environments. Our system serves as a mixed reality knowledge representation and visualization for context-aware systems. It provides tools to infer knowledge at runtime, including a continuous abstraction process, as well as a semantics-based rule system. This provides a continuum of knowledge, ranging from low-level sensory data to high-level semantic facts. In this paper, we present the concept and data model behind our system and describe its implementation. This can serve as a basis for a variety of applications including serious games.