Semantic analysis for video contents extraction—spotting by association in news video

Spotting by Association method for video analysis is a novel metliod to detect video segments with typical semantics. Video data contains various kinds of information through continuous images, natural language, and sound. For videos to be stored and retrieved in a Digital Library, it is essential to segment the video data into meaningful pieces. To detect meaningful segments, we need to identify the segment in each modality (video, language, and sound) that corresponds to the same story. For this purpose, we propose a new method for making correspondences between image clues detected by image analysis and Iangriage clries detected by natural language analysis. As a result, relevant video segments with sufficient informat ion froni every modality are obtained. We applied OUT nietliod to closed-captioned C N N Headline News. Video segments with important events, such as a public speech, meeting, or visit. are detc-cted fairly well.