Designing experiential environments for management of personal multimedia

With the increasing ubiquity of sensors and computational resources, it is becoming easier and increasingly common for people to electronically record, photographs, text, audio, and video gathered over their lifetime. Assimilating and taking advantage of such data requires recognition of its multimedia nature, development of data models that can represent semantics across different media, representation of complex relationships in the data (such as spatio-temporal, causal, or evolutionary), and finally, development of paradigms to mediate user-media interactions. There is currently a paucity of theoretical frameworks and implementations that allow management of diverse and rich multimedia data collections in context of the aforementioned requirements. This paper presents our research in designing an experiential Multimedia Electronic Chronicle system that addresses many of these issues in the concrete context of personal multimedia information. Central to our approach is the characterization and organization of media using the concept of an "event" for unified modeling and indexing. The event-based unified multimedia model underlies the experiential user interface, which supports direct interactions with the data within a unified presentation-exploration-query environment. In this environment, explicit facilities to model space and time aid in exploration and querying as well as in representation and reasoning with dynamic relationships in the data. Experimental and comparative studies demonstrate the promise of this research.