We consider the problem of learning a mapping function from low-level feature space to high-level semantic space. Under the assumption that the data lie on a submanifold embedded in a high dimensional Euclidean space, we propose a relevance feedback scheme which is naturally conducted only on the image manifold in question rather than the total ambient space. While images are typically represented by feature vectors in Rn, the natural distance is often different from the distance induced by the ambient space Rn. The geodesic distances on manifold are used to measure the similarities between images. However, when the number of data points is small, it is hard to discover the intrinsic manifold structure. Based on user interactions in a relevance feedback driven query-by-example system, the intrinsic similarities between images can be accurately estimated. We then develop an algorithmic framework to approximate the optimal mapping function by a Radial Basis Function (RBF) neural network. The semantics of a new image can be inferred by the RBF neural network. Experimental results show that our approach is effective in improving the performance of content-based image retrieval systems.
{"title":"Learning an image manifold for retrieval","authors":"Xiaofei He, Wei-Ying Ma, HongJiang Zhang","doi":"10.1145/1027527.1027532","DOIUrl":"https://doi.org/10.1145/1027527.1027532","url":null,"abstract":"We consider the problem of learning a mapping function from low-level feature space to high-level semantic space. Under the assumption that the data lie on a submanifold embedded in a high dimensional Euclidean space, we propose a relevance feedback scheme which is naturally conducted only on the image manifold in question rather than the total ambient space. While images are typically represented by feature vectors in Rn, the natural distance is often different from the distance induced by the ambient space Rn. The geodesic distances on manifold are used to measure the similarities between images. However, when the number of data points is small, it is hard to discover the intrinsic manifold structure. Based on user interactions in a relevance feedback driven query-by-example system, the intrinsic similarities between images can be accurately estimated. We then develop an algorithmic framework to approximate the optimal mapping function by a Radial Basis Function (RBF) neural network. The semantics of a new image can be inferred by the RBF neural network. Experimental results show that our approach is effective in improving the performance of content-based image retrieval systems.","PeriodicalId":292207,"journal":{"name":"MULTIMEDIA '04","volume":"119 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2004-10-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116383898","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Interactive multimedia narrative in CD/DVD-ROMs and game design offers choice and non-linearity as communication signifiers. The viewer chooses where to go in an interactive design, viewing content in a non-traditional, non-linear fashion. However the content is totally controlled by the author, not the reader. The content is pre-determined while the viewer controls the viewing path in real time. The Dawn At My Back: Memoir of a Black Texas Upbringing outlines three parallel stories of a mother, a daughter, and a society by detailing racism's impact on each element. The concept is to design a combination Book/DVD-ROM/Website that allows the reader direct interaction with the narrative. This project expands on the idea that interactivity is a dialogue between viewer and story by encouraging the book's reader to become a DVD-ROM user and website co-author.
CD/ dvd - rom中的交互式多媒体叙事和游戏设计提供了选择和非线性作为交流的符号。在交互式设计中,观看者选择去哪里,以非传统的、非线性的方式观看内容。然而,内容完全由作者控制,而不是读者。内容是预先确定的,而观看者实时控制观看路径。《我背后的黎明:德克萨斯州黑人成长回忆录》通过详细描述种族主义对每个元素的影响,概述了三个关于母亲、女儿和社会的平行故事。这个概念是设计一本书/DVD-ROM/网站的组合,让读者直接与叙事互动。这个项目通过鼓励书的读者成为DVD-ROM用户和网站的共同作者,扩展了互动性是观众和故事之间的对话的想法。
{"title":"The dawn at my back","authors":"Carroll Parrott Blue","doi":"10.1145/1027527.1027766","DOIUrl":"https://doi.org/10.1145/1027527.1027766","url":null,"abstract":"Interactive multimedia narrative in CD/DVD-ROMs and game design offers choice and non-linearity as communication signifiers. The viewer chooses where to go in an interactive design, viewing content in a non-traditional, non-linear fashion. However the content is totally controlled by the author, not the reader. The content is pre-determined while the viewer controls the viewing path in real time.\u0000 <i>The Dawn At My Back: Memoir of a Black Texas Upbringing</i> outlines three parallel stories of a mother, a daughter, and a society by detailing racism's impact on each element. The concept is to design a combination Book/DVD-ROM/Website that allows the reader direct interaction with the narrative. This project expands on the idea that interactivity is a dialogue between viewer and story by encouraging the book's reader to become a DVD-ROM user and website co-author.","PeriodicalId":292207,"journal":{"name":"MULTIMEDIA '04","volume":"116 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2004-10-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123576662","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Ye Wang, Min-Yen Kan, T. Nwe, Arun Shenoy, Jun Yin
We present a prototype that automatically aligns acoustic musical signals with their corresponding textual lyrics, in a manner similar to manually-aligned karaoke. We tackle this problem using a multimodal approach, where the appropriate pairing of audio and text processing helps create a more accurate system. Our audio processing technique uses a combination of top-down and bottom-up approaches, combining the strength of low-level audio features and high-level musical knowledge to determine the hierarchical rhythm structure, singing voice and chorus sections in the musical audio. Text processing is also employed to approximate the length of the sung passages using the textual lyrics. Results show an average error of less than one bar for per-line alignment of the lyrics on a test bed of 20 songs (sampled from CD audio and carefully selected for variety). We perform holistic and per-component testing and analysis and outline steps for further development.
{"title":"LyricAlly: automatic synchronization of acoustic musical signals and textual lyrics","authors":"Ye Wang, Min-Yen Kan, T. Nwe, Arun Shenoy, Jun Yin","doi":"10.1145/1027527.1027576","DOIUrl":"https://doi.org/10.1145/1027527.1027576","url":null,"abstract":"We present a prototype that automatically aligns acoustic musical signals with their corresponding textual lyrics, in a manner similar to manually-aligned karaoke. We tackle this problem using a multimodal approach, where the appropriate pairing of audio and text processing helps create a more accurate system. Our audio processing technique uses a combination of top-down and bottom-up approaches, combining the strength of low-level audio features and high-level musical knowledge to determine the hierarchical rhythm structure, singing voice and chorus sections in the musical audio. Text processing is also employed to approximate the length of the sung passages using the textual lyrics. Results show an average error of less than one bar for per-line alignment of the lyrics on a test bed of 20 songs (sampled from CD audio and carefully selected for variety). We perform holistic and per-component testing and analysis and outline steps for further development.","PeriodicalId":292207,"journal":{"name":"MULTIMEDIA '04","volume":"81 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2004-10-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122081125","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Jehan Wickramasuriya, M. Datt, S. Mehrotra, N. Venkatasubramanian
Around the world as both crime and technology become more prevalent, officials find themselves relying more and more on video surveillance as a cure-all in the name of public safety. Used properly, video cameras help expose wrongdoing but typically come at the cost of privacy to those not involved in any maleficent activity. What if we could design intelligent systems that are more selective in what video they capture, and focus on anomalous events while protecting the privacy of authorized personnel? This paper proposes a novel way of combining sensor technology with traditional video surveillance in building a privacy protecting framework that exploits the strengths of these modalities and complements their individual limitations. Our fully functional system utilizes off the shelf sensor hardware (i.e. RFID, motion detection) for localization, and combines this with a XML-based policy framework for access control to determine violations within the space. This information is fused with video surveillance streams in order to make decisions about how to display the individuals being surveilled. To achieve this, we have implemented several video masking techniques that correspond to varying user privacy levels. These results were achievable in real-time at acceptable frame rates, while meeting our requirements for privacy preservation.
{"title":"Privacy protecting data collection in media spaces","authors":"Jehan Wickramasuriya, M. Datt, S. Mehrotra, N. Venkatasubramanian","doi":"10.1145/1027527.1027537","DOIUrl":"https://doi.org/10.1145/1027527.1027537","url":null,"abstract":"Around the world as both crime and technology become more prevalent, officials find themselves relying more and more on video surveillance as a cure-all in the name of public safety. Used properly, video cameras help expose wrongdoing but typically come at the cost of privacy to those not involved in any maleficent activity. What if we could design intelligent systems that are more selective in what video they capture, and focus on anomalous events while protecting the privacy of authorized personnel? This paper proposes a novel way of combining sensor technology with traditional video surveillance in building a privacy protecting framework that exploits the strengths of these modalities and complements their individual limitations. Our fully functional system utilizes off the shelf sensor hardware (i.e. RFID, motion detection) for localization, and combines this with a XML-based policy framework for access control to determine violations within the space. This information is fused with video surveillance streams in order to make decisions about how to display the individuals being surveilled. To achieve this, we have implemented several video masking techniques that correspond to varying user privacy levels. These results were achievable in real-time at acceptable frame rates, while meeting our requirements for privacy preservation.","PeriodicalId":292207,"journal":{"name":"MULTIMEDIA '04","volume":"144 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2004-10-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116794797","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
This article proposes a multimodal approach for segmenting meeting recordings. This bi-modal method takes advantages of the alignment of speech transcript with documents, in the context of meetings or lectures, where documents are discussed. The method first displays the alignment results as a set of nodes in a 2D space, where the two axes represent respectively the documents content and the speech transcript. The most connected regions in this graph are detected using a clustering method. The final clusters are then projected on the speech axis. Finally, the obtained sequence of segments is considered as the thematic structure of the speech transcript. In this article, we present our bi-modal method and compare it with two other mono-modal thematic segmentation methods.
{"title":"Thematic segmentation of meetings through document/speech alignment","authors":"Dalila Mekhaldi, D. Lalanne, R. Ingold","doi":"10.1145/1027527.1027714","DOIUrl":"https://doi.org/10.1145/1027527.1027714","url":null,"abstract":"This article proposes a multimodal approach for segmenting meeting recordings. This bi-modal method takes advantages of the alignment of speech transcript with documents, in the context of meetings or lectures, where documents are discussed. The method first displays the alignment results as a set of nodes in a 2D space, where the two axes represent respectively the documents content and the speech transcript. The most connected regions in this graph are detected using a clustering method. The final clusters are then projected on the speech axis. Finally, the obtained sequence of segments is considered as the thematic structure of the speech transcript. In this article, we present our bi-modal method and compare it with two other mono-modal thematic segmentation methods.","PeriodicalId":292207,"journal":{"name":"MULTIMEDIA '04","volume":"97 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2004-10-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"117206870","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Carriers increasingly differentiate their wide-area connectivity offerings by means of customized services, such as virtual private networks (VPN) with Quality of Service (QoS) guarantees, or QVPNs. The key challenge faced by carriers is to maximize the number of QVPNs admitted by exploiting the statistical multiplexing nature of input traffic. While existing measurement-based admission control algorithms utilize statistical multiplexing along the bandwidth dimension, they do not satisfactorily exploit statistical multiplexing along the delay dimension to guarantee distinct per-QVPN delay bounds. This paper presents Delay Distribution Measurement (DDM) based admission control algorithm, the first measurement-based approach that effectively exploits statistical multiplexing along the delay dimension. In other words, DDM exploits the well known fact that the actual delay experienced by most packets of a QVPN is usually far smaller than its worst-case delay bound requirement since multiple QVPNs rarely send traffic bursts at the same time. Additionally, DDM supports QVPNs with distinct probabilistic delay guarantees -- QVPNs that can tolerate more delay violations can reserve fewer resource than those that tolerate less, even though they require the same delay bound. A comprehensive performance evaluation using Voice over IP traces shows that, when compared to deterministic admission control, DDM can potentially increase the number of admitted QVPNs (and link utilization) by up to a factor of 3.0 even when the delay violation probability is as small as 10-5.
{"title":"Probabilistic delay guarantees using delay distribution measurement","authors":"Kartik Gopalan, T. Chiueh, Yow-Jian Lin","doi":"10.1145/1027527.1027734","DOIUrl":"https://doi.org/10.1145/1027527.1027734","url":null,"abstract":"Carriers increasingly differentiate their wide-area connectivity offerings by means of customized services, such as virtual private networks (VPN) with Quality of Service (QoS) guarantees, or QVPNs. The key challenge faced by carriers is to maximize the number of QVPNs admitted by exploiting the statistical multiplexing nature of input traffic. While existing measurement-based admission control algorithms utilize statistical multiplexing along the bandwidth dimension, they do not satisfactorily exploit statistical multiplexing along the <i>delay dimension</i> to guarantee <i>distinct per-QVPN delay bounds</i>. This paper presents Delay Distribution Measurement (DDM) based admission control algorithm, the first measurement-based approach that effectively exploits statistical multiplexing along the delay dimension. In other words, DDM exploits the well known fact that the actual delay experienced by most packets of a QVPN is usually far smaller than its worst-case delay bound requirement since multiple QVPNs rarely send traffic bursts at the same time. Additionally, DDM supports QVPNs with distinct probabilistic delay guarantees -- QVPNs that can tolerate more delay violations can reserve fewer resource than those that tolerate less, even though they require the same delay bound. A comprehensive performance evaluation using Voice over IP traces shows that, when compared to deterministic admission control, DDM can potentially increase the number of admitted QVPNs (and link utilization) by up to a factor of 3.0 even when the delay violation probability is as small as 10<sup>-5</sup>.","PeriodicalId":292207,"journal":{"name":"MULTIMEDIA '04","volume":"89 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2004-10-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125079024","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
In this demonstration we present three interface designs which enable users to visually browse video data by moving a slider thumb along the timeline. In such a case, scrolling granularity is usually limited because of the fixed length of the corresponding slider. In contrast, our interaction designs enable users to skim the data at different granularity levels by providing the possibility to continuously change the slider's scale, by using a nonlinear scale, and by enabling interactive manipulation of the scrolling speed, respectively.
{"title":"Advanced user interfaces for dynamic video browsing","authors":"Wolfgang Hürst, Georg Götz, Philipp Jarvers","doi":"10.1145/1027527.1027694","DOIUrl":"https://doi.org/10.1145/1027527.1027694","url":null,"abstract":"In this demonstration we present three interface designs which enable users to visually browse video data by moving a slider thumb along the timeline. In such a case, scrolling granularity is usually limited because of the fixed length of the corresponding slider. In contrast, our interaction designs enable users to skim the data at different granularity levels by providing the possibility to continuously change the slider's scale, by using a nonlinear scale, and by enabling interactive manipulation of the scrolling speed, respectively.","PeriodicalId":292207,"journal":{"name":"MULTIMEDIA '04","volume":"43 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2004-10-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128257111","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
The rise of the internet and the increasing availability of low cost means to create digital media have created an environment and an appetite in the audience for meaningful interaction with mass media. Television news is an area that holds great potential for community based programming and can be made to allow the audience a direct role in the production of such programming. The focus of this project has been to develop a working prototype of a system to support the live production of low cost, community orientated, interactive television news programs in which the audience has direct and immediate influence over the programming.
{"title":"Interactive tele-journalism: low cost, live, interactive television news production","authors":"S. Every","doi":"10.1145/1027527.1027562","DOIUrl":"https://doi.org/10.1145/1027527.1027562","url":null,"abstract":"The rise of the internet and the increasing availability of low cost means to create digital media have created an environment and an appetite in the audience for meaningful interaction with mass media. Television news is an area that holds great potential for community based programming and can be made to allow the audience a direct role in the production of such programming.\u0000 The focus of this project has been to develop a working prototype of a system to support the live production of low cost, community orientated, interactive television news programs in which the audience has direct and immediate influence over the programming.","PeriodicalId":292207,"journal":{"name":"MULTIMEDIA '04","volume":"274 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2004-10-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130676994","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Image annotations allow users to access a large image database with textual queries. There have been several studies on automatic image annotation utilizing machine learning techniques, which automatically learn statistical models from annotated images and apply them to generate annotations for unseen images. One common problem shared by most previous learning approaches for automatic image annotation is that each annotated word is predicated for an image independently from other annotated words. In this paper, we proposed a coherent language model for automatic image annotation that takes into account the word-to-word correlation by estimating a coherent language model for an image. This new approach has two important advantages: 1) it is able to automatically determine the annotation length to improve the accuracy of retrieval results, and 2) it can be used with active learning to significantly reduce the required number of annotated image examples. Empirical studies with Corel dataset are presented to show the effectiveness of the coherent language model for automatic image annotation.
{"title":"Effective automatic image annotation via a coherent language model and active learning","authors":"Rong Jin, J. Chai, Luo Si","doi":"10.1145/1027527.1027732","DOIUrl":"https://doi.org/10.1145/1027527.1027732","url":null,"abstract":"Image annotations allow users to access a large image database with textual queries. There have been several studies on automatic image annotation utilizing machine learning techniques, which automatically learn statistical models from annotated images and apply them to generate annotations for unseen images. One common problem shared by most previous learning approaches for automatic image annotation is that each annotated word is predicated for an image independently from other annotated words. In this paper, we proposed a coherent language model for automatic image annotation that takes into account the word-to-word correlation by estimating a coherent language model for an image. This new approach has two important advantages: 1) it is able to automatically determine the annotation length to improve the accuracy of retrieval results, and 2) it can be used with active learning to significantly reduce the required number of annotated image examples. Empirical studies with Corel dataset are presented to show the effectiveness of the coherent language model for automatic image annotation.","PeriodicalId":292207,"journal":{"name":"MULTIMEDIA '04","volume":"25 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2004-10-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128874890","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
One of the goals of the Continuous Media Web project1 is to integrate digital media with the World Wide Web: media documents can hyperlink to and from other documents in the same way that HTML pages do. The dual capabilities of hyperlinking (1) to other documents while viewing a media clip, and (2) into precise time intervals in a media clip, enable greatly improved user interaction with media. We discuss the idea of a novel media browser application, which merges the concept of a traditional media player that presents video and audio to the user, with a Web browser that provides hyperlinking and navigation between networked (media) documents. The particular issue we address in this article concerns the primary navigational features: a media player relies on a playlist while a Web browser uses a browsing history for navigation. We discuss design and user interface issues that arise when integrating these two navigational features in a media browser.
{"title":"Challenges of networked media: integrating the navigational features of browsing histories and media playlists into a media browser","authors":"André T. H. Pang, C. Parker, S. Pfeiffer","doi":"10.1145/1027527.1027643","DOIUrl":"https://doi.org/10.1145/1027527.1027643","url":null,"abstract":"One of the goals of the Continuous Media Web project<sup>1</sup> is to integrate digital media with the World Wide Web: media documents can hyperlink to and from other documents in the same way that HTML pages do. The dual capabilities of hyperlinking (1) to other documents while viewing a media clip, and (2) into precise time intervals in a media clip, enable greatly improved user interaction with media. We discuss the idea of a novel <i>media browser</i> application, which merges the concept of a traditional media player that presents video and audio to the user, with a Web browser that provides hyperlinking and navigation between networked (media) documents. The particular issue we address in this article concerns the primary navigational features: a media player relies on a <i>playlist</i> while a Web browser uses a <i>browsing history</i> for navigation. We discuss design and user interface issues that arise when integrating these two navigational features in a media browser.","PeriodicalId":292207,"journal":{"name":"MULTIMEDIA '04","volume":"260 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2004-10-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123100940","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}