A system of video information capture, indexing and retrieval for interpreting human activity

Abstract

This system creates a manageable information resource that enables more complete and accurate interpretation, assessment and diagnosis of human behavior in constrained physical spaces. Through activity and environmental monitoring, a continuous, voluminous audio and video record is captured. Through work in information extraction, behavior analysis and synthesis, this record is transformed into an information asset whose efficient, secure presentation empowers specialists with greater insights into problems, effectiveness of treatments, and determination of environmental and social influences. Application environments range from nursery schools to nursing homes. The foundation for this work, the informedia digital video library (H.D. Wactlar et al., 1999), has demonstrated the successful integration of speech, image, and natural language processing in automatically creating an indexed, searchable multimedia information resource for broadcast-quality video, upon which this system builds.