In this work we present a novel execution flow for the super-resolution image restoration (SRIR) non-uniform grid projection algorithm -- the macroblock-level flow. The novel flow is compared with the reference frame-level flow. The frame-level flow is characterized by the fact that transitions from one step of the algorithm to another occur only after the current step is carried out for all macro blocks (MBs) of the frame being currently processed. The novel flow carries out complete processing of one MB before the processing of another MB starts. The memory requirements of both schemes are evaluated in detail and compared. The study on the achievable memory reduction in total memory requirements was carried out for different values of the algorithm parameters: the MB size, scale factor, search area size and number of reference frames included in the sliding frame window. The results show quantitatively that the parameter that influences storage instantiation the most and has the greatest influence on the total memory size is the number of reference frames in the sliding frame window. The conducted study shows that, for a QCIF frame format, switching from frame-to macroblock-level is feasible and fully validated functionally and that the new execution flow can lead to memory reduction by a factor of 6.8 to 40, depending on the algorithm parameters values. Memory reduction greatly facilitates hardware implementations of the algorithm and this is the main result claimed. But the reduction in memory size comes at the cost of increasing the number of memory accesses and therefore communications traffic. The increase noted in memory accesses it to be quantified in future work as well as the potential impact on power consumption. The reduction in memory size might also make it fit on chip without turning to external memory, thereby reducing power consumption. This trade off in power is yet to be quantified.
{"title":"A Low Memory Requirements Execution Flow for the Non-Uniform Grid Projection Super-Resolution Algorithm","authors":"T. Szydzik, G. Callicó, A. Núñez","doi":"10.1109/ISM.2011.22","DOIUrl":"https://doi.org/10.1109/ISM.2011.22","url":null,"abstract":"In this work we present a novel execution flow for the super-resolution image restoration (SRIR) non-uniform grid projection algorithm -- the macroblock-level flow. The novel flow is compared with the reference frame-level flow. The frame-level flow is characterized by the fact that transitions from one step of the algorithm to another occur only after the current step is carried out for all macro blocks (MBs) of the frame being currently processed. The novel flow carries out complete processing of one MB before the processing of another MB starts. The memory requirements of both schemes are evaluated in detail and compared. The study on the achievable memory reduction in total memory requirements was carried out for different values of the algorithm parameters: the MB size, scale factor, search area size and number of reference frames included in the sliding frame window. The results show quantitatively that the parameter that influences storage instantiation the most and has the greatest influence on the total memory size is the number of reference frames in the sliding frame window. The conducted study shows that, for a QCIF frame format, switching from frame-to macroblock-level is feasible and fully validated functionally and that the new execution flow can lead to memory reduction by a factor of 6.8 to 40, depending on the algorithm parameters values. Memory reduction greatly facilitates hardware implementations of the algorithm and this is the main result claimed. But the reduction in memory size comes at the cost of increasing the number of memory accesses and therefore communications traffic. The increase noted in memory accesses it to be quantified in future work as well as the potential impact on power consumption. The reduction in memory size might also make it fit on chip without turning to external memory, thereby reducing power consumption. This trade off in power is yet to be quantified.","PeriodicalId":339410,"journal":{"name":"2011 IEEE International Symposium on Multimedia","volume":"6 3","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2011-12-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114122751","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
We present an intuitive user interface for the exploration of images on mobile multi-touch devices. Our interface uses a novel cylindrical 3D visualization of visually sorted images as well as touch gestures and tilting operations to support mobile users in interactive browsing of images by providing convenient navigation/interaction and intuitive visualization capabilities.
{"title":"3D Image Browsing on Mobile Devices","authors":"Klaus Schöffmann, David Ahlström, C. Beecks","doi":"10.1109/ISM.2011.60","DOIUrl":"https://doi.org/10.1109/ISM.2011.60","url":null,"abstract":"We present an intuitive user interface for the exploration of images on mobile multi-touch devices. Our interface uses a novel cylindrical 3D visualization of visually sorted images as well as touch gestures and tilting operations to support mobile users in interactive browsing of images by providing convenient navigation/interaction and intuitive visualization capabilities.","PeriodicalId":339410,"journal":{"name":"2011 IEEE International Symposium on Multimedia","volume":"152 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2011-12-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114497949","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Many state-of-the-art audio codecs operating in a transform domain provide scalability as a core function by allowing to selectively subtract bits -- usually according to a nonperceptual criterion from the full bit rate data stream. This work presents a different, or even reverse, scalability approach in which a scalable codec can selectively add perceptually significant bits to a low bit rate data stream. The scalable enhancement algorithm presented here operates in the Modified Discrete Cosine Transform domain, which is popular among perceptual audio transform encoders, but its extension on other domains is straightforward. By exploiting the information of an existing low bit rate base layer, the algorithm adds perceptually significant data to the data stream according to a psycho acoustic model, and improves the audio quality at a fraction of the bit rate that would normally be required for the encoding or transmission of the whole audio piece of the same quality. Applications of this can be found in packet retransmission schemes of compressed audio networks and in remote audio enhancement.
{"title":"Perceptually-Driven Scalable MDCT Enhancement of Compressed Audio Based on Statistical Conversion","authors":"D. Cantzos, A. Mouchtaris, C. Kyriakakis","doi":"10.1109/ISM.2011.16","DOIUrl":"https://doi.org/10.1109/ISM.2011.16","url":null,"abstract":"Many state-of-the-art audio codecs operating in a transform domain provide scalability as a core function by allowing to selectively subtract bits -- usually according to a nonperceptual criterion from the full bit rate data stream. This work presents a different, or even reverse, scalability approach in which a scalable codec can selectively add perceptually significant bits to a low bit rate data stream. The scalable enhancement algorithm presented here operates in the Modified Discrete Cosine Transform domain, which is popular among perceptual audio transform encoders, but its extension on other domains is straightforward. By exploiting the information of an existing low bit rate base layer, the algorithm adds perceptually significant data to the data stream according to a psycho acoustic model, and improves the audio quality at a fraction of the bit rate that would normally be required for the encoding or transmission of the whole audio piece of the same quality. Applications of this can be found in packet retransmission schemes of compressed audio networks and in remote audio enhancement.","PeriodicalId":339410,"journal":{"name":"2011 IEEE International Symposium on Multimedia","volume":"421 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2011-12-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115612112","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
In this paper, we present analytical information about Flickr notes and propose further directions of note based image search. Compared to a tag that is used for traditional social image search, Flickr note is a kind of text directly assigned on the image regions. Even though note has various information that may help intelligent social image sharing and search, there is no significant research that focuses on the potential and the impact of note for image search. In order to reveal the useful information and potential of Flickr notes, we have collected a number of images and analyzed them with regard to various aspects. Additionally, from the analytical results about Flickr notes, we show various possible research issues to which note information can be applied.
{"title":"Exploiting of Flickr Note and its Applications for Social Image Sharing and Search","authors":"Jin-Woo Jeong, Hyun-Ki Hong, Dong-Ho Lee","doi":"10.1109/ISM.2011.34","DOIUrl":"https://doi.org/10.1109/ISM.2011.34","url":null,"abstract":"In this paper, we present analytical information about Flickr notes and propose further directions of note based image search. Compared to a tag that is used for traditional social image search, Flickr note is a kind of text directly assigned on the image regions. Even though note has various information that may help intelligent social image sharing and search, there is no significant research that focuses on the potential and the impact of note for image search. In order to reveal the useful information and potential of Flickr notes, we have collected a number of images and analyzed them with regard to various aspects. Additionally, from the analytical results about Flickr notes, we show various possible research issues to which note information can be applied.","PeriodicalId":339410,"journal":{"name":"2011 IEEE International Symposium on Multimedia","volume":"4 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2011-12-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116152949","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Steven Luis, Fausto Fleites, Yimin Yang, Hsin-Yu Ha, Shu‐Ching Chen
We present a novel visual analytics system and multimedia enabled mobile application that allows emergency management (EM) personnel access to timely and relevant disaster situation information. The system is able to semantically integrate text-based emergency management disaster situation reports with related disaster imagery taken in the field by EM responders and community residents. In addition, through an intuitive and seamless Apple iPad application, users are able to interact with the system in diverse places and conditions and thus provide a more effective response. The system is demonstrated via its iPad application which aims at providing relevant and actionable information.
{"title":"A Visual Analytics Multimedia Mobile System for Emergency Response","authors":"Steven Luis, Fausto Fleites, Yimin Yang, Hsin-Yu Ha, Shu‐Ching Chen","doi":"10.1109/ISM.2011.61","DOIUrl":"https://doi.org/10.1109/ISM.2011.61","url":null,"abstract":"We present a novel visual analytics system and multimedia enabled mobile application that allows emergency management (EM) personnel access to timely and relevant disaster situation information. The system is able to semantically integrate text-based emergency management disaster situation reports with related disaster imagery taken in the field by EM responders and community residents. In addition, through an intuitive and seamless Apple iPad application, users are able to interact with the system in diverse places and conditions and thus provide a more effective response. The system is demonstrated via its iPad application which aims at providing relevant and actionable information.","PeriodicalId":339410,"journal":{"name":"2011 IEEE International Symposium on Multimedia","volume":"33 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2011-12-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126643940","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
In this paper, we provide background information on the EC funded Lila Project ("Library of Labs"), describe its goals and purposes, provide some insight into its software design and provide first experiences, made at the University of Stuttgart using the eLearning content deployed by the project.
{"title":"Library of Labs - A European Project on the Dissemination of Remote Experiments and Virtual Laboratories","authors":"T. Richter, Yvonne Tetour, D. Boehringer","doi":"10.1109/ISM.2011.96","DOIUrl":"https://doi.org/10.1109/ISM.2011.96","url":null,"abstract":"In this paper, we provide background information on the EC funded Lila Project (\"Library of Labs\"), describe its goals and purposes, provide some insight into its software design and provide first experiences, made at the University of Stuttgart using the eLearning content deployed by the project.","PeriodicalId":339410,"journal":{"name":"2011 IEEE International Symposium on Multimedia","volume":"48 1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2011-12-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126002687","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Gianpaolo D'Amico, A. Bimbo, Andrea Ferracani, Lea Landucci, Daniele Pezzatini, Luca Santi
In this paper we present a work-in-progress interactive exhibit for the museum of Onna, a little town near to L'Aquila (Italy), almost completely destroyed by the earthquake of April 2009. The installation will be developed as an environment in which visitors of the museum can interact with a natural interaction system and then discover the history of the disaster via rich multimedia contents. Visitors are detected through the adoption of an RFID-based technology, which allows to store their interaction history and build an interest profile used to enrich the experience. Different scenarios have been implemented and tested in order to evaluate the effectiveness of the proposed solution.
{"title":"RFID-based Solutions for User Profiling in Interactive Exhibits","authors":"Gianpaolo D'Amico, A. Bimbo, Andrea Ferracani, Lea Landucci, Daniele Pezzatini, Luca Santi","doi":"10.1109/ISM.2011.42","DOIUrl":"https://doi.org/10.1109/ISM.2011.42","url":null,"abstract":"In this paper we present a work-in-progress interactive exhibit for the museum of Onna, a little town near to L'Aquila (Italy), almost completely destroyed by the earthquake of April 2009. The installation will be developed as an environment in which visitors of the museum can interact with a natural interaction system and then discover the history of the disaster via rich multimedia contents. Visitors are detected through the adoption of an RFID-based technology, which allows to store their interaction history and build an interest profile used to enrich the experience. Different scenarios have been implemented and tested in order to evaluate the effectiveness of the proposed solution.","PeriodicalId":339410,"journal":{"name":"2011 IEEE International Symposium on Multimedia","volume":"7 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2011-12-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124683329","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pico Life is envisioned to be an augmented reality game in which 3D characters will be controlled by hand gestures on Android smart phones. Pico Life is currently powered by two mobile optimized engines: (1) The computer vision engine that runs our advanced object tracking program for hand tracking and (2) The 3D engine that runs our 3D models for the characters in the game. In the near future, we will be adding yet another mobile optimized engine, namely, the augmented reality engine. In this paper, we will present our work on object tracking and 3D modeling for Pico Life and contrast the performances of the two engines on three different mobile platforms, namely, Texas Instruments' OMAP3630 (Motorola Droid X running Android Gingerbread), Qualcomm's MSM8660 Snapdragon (HTC Evo 3D running Android Gingerbread) and the Texas Instruments' OMAP4430 (Blaze Development platform running Android Gingerbread).
Pico Life被设想为一款增强现实游戏，其中3D角色将通过Android智能手机上的手势控制。《Pico Life》目前由两个移动优化引擎提供支持:(1)运行我们用于手部追踪的高级目标追踪程序的计算机视觉引擎，以及(2)运行游戏中角色的3D模型的3D引擎。在不久的将来，我们将添加另一个移动优化引擎，即增强现实引擎。在本文中，我们将介绍我们在Pico Life的目标跟踪和3D建模方面的工作，并对比两种引擎在三个不同的移动平台上的性能，即德州仪器的OMAP3630(运行Android Gingerbread的摩托罗拉Droid X)，高通的MSM8660骁龙(运行Android Gingerbread的HTC Evo 3D)和德州仪器的OMAP4430(运行Android Gingerbread的Blaze开发平台)。
{"title":"PicoLife: A Computer Vision-based Gesture Recognition and 3D Gaming System for Android Mobile Devices","authors":"Mahesh Babu Mariappan, X. Guo, B. Prabhakaran","doi":"10.1109/ISM.2011.13","DOIUrl":"https://doi.org/10.1109/ISM.2011.13","url":null,"abstract":"Pico Life is envisioned to be an augmented reality game in which 3D characters will be controlled by hand gestures on Android smart phones. Pico Life is currently powered by two mobile optimized engines: (1) The computer vision engine that runs our advanced object tracking program for hand tracking and (2) The 3D engine that runs our 3D models for the characters in the game. In the near future, we will be adding yet another mobile optimized engine, namely, the augmented reality engine. In this paper, we will present our work on object tracking and 3D modeling for Pico Life and contrast the performances of the two engines on three different mobile platforms, namely, Texas Instruments' OMAP3630 (Motorola Droid X running Android Gingerbread), Qualcomm's MSM8660 Snapdragon (HTC Evo 3D running Android Gingerbread) and the Texas Instruments' OMAP4430 (Blaze Development platform running Android Gingerbread).","PeriodicalId":339410,"journal":{"name":"2011 IEEE International Symposium on Multimedia","volume":"11 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2011-12-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133806983","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
For research on quality of experience (QoE), mean opinion scores (MOS) are widely chosen as the results of subjective tests and the ground-truth reference for further research on objective quality modeling. Furthermore, the results of objective quality modeling are used for QoE management subsequently. Therefore, the performance of QoE management process actually depends heavily on MOS. However, the rationality of MOS for QoE management is not yet technically proven in the literature. In this paper, we first prove that subject homogeneity is implicitly assumed for obtaining MOS by modeling the arithmetic averaging process from a systematic viewpoint. However, we point out that actually subjects exhibit variability in terms of quality assessment. Then we elaborate that this mismatch may results in failures if we conduct QoE management based on MOS. Finally we propose a utility-based averaging method (uMOS) which improves the performance of QoE management.
{"title":"On the Properties of Mean Opinion Scores for Quality of Experience Management","authors":"Jie Xu, Liyuan Xing, A. Perkis, Yuming Jiang","doi":"10.1109/ISM.2011.88","DOIUrl":"https://doi.org/10.1109/ISM.2011.88","url":null,"abstract":"For research on quality of experience (QoE), mean opinion scores (MOS) are widely chosen as the results of subjective tests and the ground-truth reference for further research on objective quality modeling. Furthermore, the results of objective quality modeling are used for QoE management subsequently. Therefore, the performance of QoE management process actually depends heavily on MOS. However, the rationality of MOS for QoE management is not yet technically proven in the literature. In this paper, we first prove that subject homogeneity is implicitly assumed for obtaining MOS by modeling the arithmetic averaging process from a systematic viewpoint. However, we point out that actually subjects exhibit variability in terms of quality assessment. Then we elaborate that this mismatch may results in failures if we conduct QoE management based on MOS. Finally we propose a utility-based averaging method (uMOS) which improves the performance of QoE management.","PeriodicalId":339410,"journal":{"name":"2011 IEEE International Symposium on Multimedia","volume":"24 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2011-12-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"134350898","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Using recipes in a step-by-step format with multimedia enhancements has been found to increase confidence and enjoyment of cooking but the field lacks research with cooks on the problems they encounter, so it is unclear what granularity of recipe step and associated multimedia would best support them. The current study observed 16 cooks prepare 3 dishes using recipes in 3 different formats to understand what problems cooks have with recipes. Recipe format had a significant effect on the ratings given to the recipe for clarity and ease of use but not on time to complete the recipe. Analysis of cooking activity and cooks' feedback shows that cooks want (i) step-by-step recipes with ingredient quantities in the recipe step, (ii) pictures of the interim states of the recipe, (iii) videos of preparation of unfamiliar ingredients, and (iv) videos of preparation techniques with different types of utensils.
{"title":"What Cooks Needs from Multimedia and Textually Enhanced Recipes","authors":"Lucy Buykx, H. Petrie","doi":"10.1109/ISM.2011.70","DOIUrl":"https://doi.org/10.1109/ISM.2011.70","url":null,"abstract":"Using recipes in a step-by-step format with multimedia enhancements has been found to increase confidence and enjoyment of cooking but the field lacks research with cooks on the problems they encounter, so it is unclear what granularity of recipe step and associated multimedia would best support them. The current study observed 16 cooks prepare 3 dishes using recipes in 3 different formats to understand what problems cooks have with recipes. Recipe format had a significant effect on the ratings given to the recipe for clarity and ease of use but not on time to complete the recipe. Analysis of cooking activity and cooks' feedback shows that cooks want (i) step-by-step recipes with ingredient quantities in the recipe step, (ii) pictures of the interim states of the recipe, (iii) videos of preparation of unfamiliar ingredients, and (iv) videos of preparation techniques with different types of utensils.","PeriodicalId":339410,"journal":{"name":"2011 IEEE International Symposium on Multimedia","volume":"31 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2011-12-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133412081","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：