In this paper, we present a new system for a highly immersive virtual reality experience utilizing head-mounted displays (HMDs) with accurate indoor-position tracking abilities. The system is designed to have six degrees of freedom in the virtual world, which allows users to physically move around in the real world while wearing the wireless system with an HMD. Three-dimensional X, Y, and Z coordinate data are estimated in real time using ultrasonic sensors. In addition, the pitch, roll, and yaw values are also measured. Unlike previously developed systems which require external input devices to move in the virtual environment, our system provides a natural virtual reality experience by precisely matching the physical movements in a real world to those in a virtual environment. Results show that the system is able to estimate accurate positions of a user, and delivers a highly immersive virtual/mixed reality experience.
{"title":"Integration of a Precise Indoor Position Tracking Algorithm with an HMD-Based Virtual Reality System","authors":"Jongkyu Shin, G. An, Kyogu Lee","doi":"10.1145/2660579.2660582","DOIUrl":"https://doi.org/10.1145/2660579.2660582","url":null,"abstract":"In this paper, we present a new system for a highly immersive virtual reality experience utilizing head-mounted displays (HMDs) with accurate indoor-position tracking abilities. The system is designed to have six degrees of freedom in the virtual world, which allows users to physically move around in the real world while wearing the wireless system with an HMD. Three-dimensional X, Y, and Z coordinate data are estimated in real time using ultrasonic sensors. In addition, the pitch, roll, and yaw values are also measured. Unlike previously developed systems which require external input devices to move in the virtual environment, our system provides a natural virtual reality experience by precisely matching the physical movements in a real world to those in a virtual environment. Results show that the system is able to estimate accurate positions of a user, and delivers a highly immersive virtual/mixed reality experience.","PeriodicalId":391229,"journal":{"name":"ImmersiveMe '14","volume":"59 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2014-11-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123243519","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
The application of immersion to awaken human potential through innovative experience design is dependent upon one's ability to spark the imagination. Our keynote speaker shares his journey of exploring how the diverse application of interactive entertainment techniques can define new innovations in life transformative simulations. Using the interplay of story, play and game, his design research showcases examples of how stimulating the imagination can enhance military training for the US Army, informal education for NASA, medical imaging for ER surgeons, teacher training in Urban classrooms, experiential marketing in shopping malls as well as cognitive rehabilitation in therapy clinics. As technological advancements in simulation catch up with the science-fiction of our parents, what kind of creative leaps will our children will be making into a future where reality, virtuality and imagination work as one world? The future of immersive media will transfer from theme parks to our living rooms, and all new design paradigms will emerge to transform our homes into a school, museum, theme park, training facility, shopping center, as well as a medical and rehabilitation clinic. This talk presents a vision with all new challenges for the future.
{"title":"Immersion, Imagination & Innovation: Media Immersion Matching the power of the Imagination to Innovate the Future","authors":"Christopher B. Stapleton","doi":"10.1145/2660579.2660580","DOIUrl":"https://doi.org/10.1145/2660579.2660580","url":null,"abstract":"The application of immersion to awaken human potential through innovative experience design is dependent upon one's ability to spark the imagination. Our keynote speaker shares his journey of exploring how the diverse application of interactive entertainment techniques can define new innovations in life transformative simulations. Using the interplay of story, play and game, his design research showcases examples of how stimulating the imagination can enhance military training for the US Army, informal education for NASA, medical imaging for ER surgeons, teacher training in Urban classrooms, experiential marketing in shopping malls as well as cognitive rehabilitation in therapy clinics.\u0000 As technological advancements in simulation catch up with the science-fiction of our parents, what kind of creative leaps will our children will be making into a future where reality, virtuality and imagination work as one world? The future of immersive media will transfer from theme parks to our living rooms, and all new design paradigms will emerge to transform our homes into a school, museum, theme park, training facility, shopping center, as well as a medical and rehabilitation clinic. This talk presents a vision with all new challenges for the future.","PeriodicalId":391229,"journal":{"name":"ImmersiveMe '14","volume":"14 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2014-11-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116898542","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
We describe a spatial audio system design for multiple applications on large displays. Our system provides spatial audio based on an application window's location on the display screen. Our approach allows conveying application audio height information without special content encoding. Our approach supports spatial audio from multiple concurrent on-display AV windows. The on-display audio location moves and resizes when the AV application window is moved and resized. Our design can handle any X.Y format content and spatialize it with a limited number of discrete loudspeakers. We have implemented this spatial audio system without requiring any special hardware, by reusing existing surround sound cards and developing special software to utilize the discrete individual audio channels supported by them. Subjective listening tests confirm the spatial location perceived by listeners matches the AV window location in our system.
{"title":"On-Display Spatial Audio for Multiple Applications on Large Displays","authors":"S. Deshpande","doi":"10.1145/2660579.2660581","DOIUrl":"https://doi.org/10.1145/2660579.2660581","url":null,"abstract":"We describe a spatial audio system design for multiple applications on large displays. Our system provides spatial audio based on an application window's location on the display screen. Our approach allows conveying application audio height information without special content encoding. Our approach supports spatial audio from multiple concurrent on-display AV windows. The on-display audio location moves and resizes when the AV application window is moved and resized. Our design can handle any X.Y format content and spatialize it with a limited number of discrete loudspeakers. We have implemented this spatial audio system without requiring any special hardware, by reusing existing surround sound cards and developing special software to utilize the discrete individual audio channels supported by them. Subjective listening tests confirm the spatial location perceived by listeners matches the AV window location in our system.","PeriodicalId":391229,"journal":{"name":"ImmersiveMe '14","volume":"103 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2014-11-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123853298","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Olfaction is an important perceptual function that is often neglected in immersive media (IM) and virtual reality (VR) applications. Because the effects of olfaction have not been researched as much as those of visual, auditory, or haptic senses, the effects of olfactory stimuli on IM experiences are mainly unexplored, largely unknown, and debatable in many examples. A major factor limiting olfaction research is the lack of olfactory display options. Commercial solutions are often inadequate and expensive. Prior research on olfactory displays is helpful, but pertinent details are normally missing, and the devices are often too complex to replicate. To address this issue, we have developed a simple, low-cost olfactory display by using inexpensive components and leveraging airflow for vaporization and scent delivery. In this paper, we detail the development of our display and describe an informal study evaluating its effectiveness.
{"title":"Development of a Simple and Low-Cost Olfactory Display for Immersive Media Experiences","authors":"Nicolas S. Herrera, Ryan P. McMahan","doi":"10.1145/2660579.2660584","DOIUrl":"https://doi.org/10.1145/2660579.2660584","url":null,"abstract":"Olfaction is an important perceptual function that is often neglected in immersive media (IM) and virtual reality (VR) applications. Because the effects of olfaction have not been researched as much as those of visual, auditory, or haptic senses, the effects of olfactory stimuli on IM experiences are mainly unexplored, largely unknown, and debatable in many examples. A major factor limiting olfaction research is the lack of olfactory display options. Commercial solutions are often inadequate and expensive. Prior research on olfactory displays is helpful, but pertinent details are normally missing, and the devices are often too complex to replicate. To address this issue, we have developed a simple, low-cost olfactory display by using inexpensive components and leveraging airflow for vaporization and scent delivery. In this paper, we detail the development of our display and describe an informal study evaluating its effectiveness.","PeriodicalId":391229,"journal":{"name":"ImmersiveMe '14","volume":"42 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2014-11-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132829248","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Guy Feigenblat, Jonathan Herzig, Michal Shmueli-Scheuer, D. Konopnicki
In this work we present Mindful, a platform for defining, configuring, executing and distributing affective experiments to a large scale audience. This type of experiments measure the emotional reaction of participants to media content selected by experimenters. Furthermore, the platform manages profiles of registered users who have agreed to participate in an experiment as well as a data collection and analysis mechanisms. The analyzed data is then used to enrich users' profile and to better understand their emotional behavior. Throughout the paper we describe the platform in details and present a use case of how the platform is being used in practice.
{"title":"Mindful: A Platform for Large-Scale Affective Field Research","authors":"Guy Feigenblat, Jonathan Herzig, Michal Shmueli-Scheuer, D. Konopnicki","doi":"10.1145/2660579.2660583","DOIUrl":"https://doi.org/10.1145/2660579.2660583","url":null,"abstract":"In this work we present Mindful, a platform for defining, configuring, executing and distributing affective experiments to a large scale audience. This type of experiments measure the emotional reaction of participants to media content selected by experimenters. Furthermore, the platform manages profiles of registered users who have agreed to participate in an experiment as well as a data collection and analysis mechanisms. The analyzed data is then used to enrich users' profile and to better understand their emotional behavior. Throughout the paper we describe the platform in details and present a use case of how the platform is being used in practice.","PeriodicalId":391229,"journal":{"name":"ImmersiveMe '14","volume":"4 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2014-11-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131487367","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
This paper presents a lightweight game framework that provides real-time integration of human appearance and gesture-guided control within the game. It augments a new immersive experience since it allows game users to see their personal appearance interacting in real-time with other computer graphical characters in the game. With the goal to make the system easily realizable, we address the challenges in the whole pipeline of video processing, gesture recognition, and communication. To this end, we introduce the game framework, Gesture and Appearance Cutout Embedding (GACE), which runs the human appearance cutout algorithm and connects with game components by using memory mapped files. We also introduce the gesture-based support to enhance the immersion. Extensive experiments have shown that the proposed system runs reliably and comfortably in real-time with a commodity setting.
本文提出了一个轻量级的游戏框架，在游戏中提供了人类外观和手势引导控制的实时集成。它增强了一种新的沉浸式体验，因为它允许游戏用户看到他们的个人外观与游戏中的其他计算机图形角色实时交互。为了使系统易于实现，我们解决了视频处理、手势识别和通信等整个流程中的挑战。为此，我们引入了游戏框架GACE (Gesture and Appearance Cutout Embedding)， GACE运行人的外观切割算法，并通过内存映射文件与游戏组件连接。我们还引入了基于手势的支持来增强沉浸感。大量的实验表明，该系统在商品设置下实时运行可靠、舒适。
{"title":"GACE: Gesture and Appearance Cutout Embedding for Gaming Applications","authors":"Tam V. Nguyen, Y. H. Tan, Jose Sepulveda","doi":"10.1145/2660579.2660587","DOIUrl":"https://doi.org/10.1145/2660579.2660587","url":null,"abstract":"This paper presents a lightweight game framework that provides real-time integration of human appearance and gesture-guided control within the game. It augments a new immersive experience since it allows game users to see their personal appearance interacting in real-time with other computer graphical characters in the game. With the goal to make the system easily realizable, we address the challenges in the whole pipeline of video processing, gesture recognition, and communication. To this end, we introduce the game framework, Gesture and Appearance Cutout Embedding (GACE), which runs the human appearance cutout algorithm and connects with game components by using memory mapped files. We also introduce the gesture-based support to enhance the immersion. Extensive experiments have shown that the proposed system runs reliably and comfortably in real-time with a commodity setting.","PeriodicalId":391229,"journal":{"name":"ImmersiveMe '14","volume":"34 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2014-11-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125071670","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Hui Su, Adi Hajj-Ahmad, Chau-Wai Wong, Ravi Garg, Min Wu
Multiple videos capturing the same scene from possibly different viewing angles may be synthesized for novel immersive experience. Synchronization is an important task for such applications involving multiple pieces of audio-visual data. In this work, we exploit the electric network frequency (ENF) signal inherently embedded in the soundtrack and/or image sequence of video to temporally align video recordings. ENF is the supply frequency of power distribution networks in a power grid. Its value fluctuates slightly from its nominal value of 50 Hz or 60 Hz, and the fluctuation trends stay consistent within the same grid. Audio and video recordings that are created in areas of electric activities may capture the ENF signal due to electromagnetic interferences and other physical phenomena. We propose to synchronize video recordings by aligning the embedded ENF signals. Without major constraints on viewing angle and camera calibration as many existing methods impose, the proposed approach emerges as a new synchronization modality.
{"title":"ENF Signal Induced by Power Grid: A New Modality for Video Synchronization","authors":"Hui Su, Adi Hajj-Ahmad, Chau-Wai Wong, Ravi Garg, Min Wu","doi":"10.1145/2660579.2660588","DOIUrl":"https://doi.org/10.1145/2660579.2660588","url":null,"abstract":"Multiple videos capturing the same scene from possibly different viewing angles may be synthesized for novel immersive experience. Synchronization is an important task for such applications involving multiple pieces of audio-visual data. In this work, we exploit the electric network frequency (ENF) signal inherently embedded in the soundtrack and/or image sequence of video to temporally align video recordings. ENF is the supply frequency of power distribution networks in a power grid. Its value fluctuates slightly from its nominal value of 50 Hz or 60 Hz, and the fluctuation trends stay consistent within the same grid. Audio and video recordings that are created in areas of electric activities may capture the ENF signal due to electromagnetic interferences and other physical phenomena. We propose to synchronize video recordings by aligning the embedded ENF signals. Without major constraints on viewing angle and camera calibration as many existing methods impose, the proposed approach emerges as a new synchronization modality.","PeriodicalId":391229,"journal":{"name":"ImmersiveMe '14","volume":"37 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2014-11-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133997507","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Nimesha Ranasinghe, Kuan-Yi Lee, Gajan Suthokumar, E. Do
To create a truly immersive virtual experience, perceiving information through multiple human senses is important. Therefore, new forms of media are required that deeply involve various human senses -not only sight, sound, and touch, but also nontraditional senses like taste and smell- to create a perception of presence in a non-physical environment. However, at present, the sensation of taste is considered as one of the final frontiers of immersive media to be achieved. This paper discusses key aspects and opportunities of including the sensation of taste in the future of immersive media technologies. As a solution, we then present 'Taste+' utensils that digitally enhance the taste sensations of food and beverages without additional flavoring ingredients. Finally, we envision several future usage scenarios and challenges of the indicated technology to facilitate future immersive digital experiences.
{"title":"The Sensation of Taste in the Future of Immersive Media","authors":"Nimesha Ranasinghe, Kuan-Yi Lee, Gajan Suthokumar, E. Do","doi":"10.1145/2660579.2660586","DOIUrl":"https://doi.org/10.1145/2660579.2660586","url":null,"abstract":"To create a truly immersive virtual experience, perceiving information through multiple human senses is important. Therefore, new forms of media are required that deeply involve various human senses -not only sight, sound, and touch, but also nontraditional senses like taste and smell- to create a perception of presence in a non-physical environment. However, at present, the sensation of taste is considered as one of the final frontiers of immersive media to be achieved. This paper discusses key aspects and opportunities of including the sensation of taste in the future of immersive media technologies. As a solution, we then present 'Taste+' utensils that digitally enhance the taste sensations of food and beverages without additional flavoring ingredients. Finally, we envision several future usage scenarios and challenges of the indicated technology to facilitate future immersive digital experiences.","PeriodicalId":391229,"journal":{"name":"ImmersiveMe '14","volume":"21 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2014-11-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126876659","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
This paper is situated in the emergent field of "Design Fiction" and describes how this approach can be applied to explorations in the field of immersive media experiences. We present Sensory Fiction -- an exploration in augmenting the emotions of a reader via a modular, multi-sensory system. The science on the nature of emotions is still inconclusive and direct ways of controlling them computationally are yet to be discovered. However, this project creates a Design Fiction that highlights the opportunities and challenges that the availability of such technology might bring. We leveraged existing scientific insights to build a functional prototype that aims to induce and evoke emotions by simulating the physiological system. Used in combination with conceptual, non-functional modules (i.e. modules that do not function physically but that introduce the idea of a physical actuation), we created an artifact to spark discussion about the future of immersive emotional experiences but that can also be experienced by the audience. Lastly, we show how presenting the project in appropriate contexts and analyzing the audience's reaction is a useful strategy to evaluate Design Fiction projects.
{"title":"Sensory Fiction: A Design Fiction of Emotional Computation","authors":"Felix Heibeck, Alexis Hope, J. Legault","doi":"10.1145/2660579.2660585","DOIUrl":"https://doi.org/10.1145/2660579.2660585","url":null,"abstract":"This paper is situated in the emergent field of \"Design Fiction\" and describes how this approach can be applied to explorations in the field of immersive media experiences. We present Sensory Fiction -- an exploration in augmenting the emotions of a reader via a modular, multi-sensory system. The science on the nature of emotions is still inconclusive and direct ways of controlling them computationally are yet to be discovered. However, this project creates a Design Fiction that highlights the opportunities and challenges that the availability of such technology might bring. We leveraged existing scientific insights to build a functional prototype that aims to induce and evoke emotions by simulating the physiological system. Used in combination with conceptual, non-functional modules (i.e. modules that do not function physically but that introduce the idea of a physical actuation), we created an artifact to spark discussion about the future of immersive emotional experiences but that can also be experienced by the audience. Lastly, we show how presenting the project in appropriate contexts and analyzing the audience's reaction is a useful strategy to evaluate Design Fiction projects.","PeriodicalId":391229,"journal":{"name":"ImmersiveMe '14","volume":"9 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2014-11-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129385130","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Christopher B. Stapleton, J. Whiteside, J. Davies, Dana S. Mott, Jennifer Vick
Aphasia is a disease that renders its victims unable to effectively use language. Evidence supports the efficacy of treatment for aphasia yet the effectiveness or transferability of learned communicative abilities to everyday conversation continues to be investigated. In this paper we explore an alternative approach to aphasia treatment based on the art and science of storytelling. Inherent in storytelling are the motivations to share an experience, the cognitive abilities to organize story, and the language system to convey the experience. This approach is based on decades of research in aphasia therapy and immersive storytelling (in other fields) and has been used to engage a subject's creativity and emotions to produce transformative results in real life. We report on early, promising results that could radically innovate the rehabilitative practice of aphasia.
{"title":"Transforming Lives Through Story Immersion: Innovation of Aphasia Rehabilitation Therapy through Storytelling Learning Landscapes","authors":"Christopher B. Stapleton, J. Whiteside, J. Davies, Dana S. Mott, Jennifer Vick","doi":"10.1145/2660579.2660590","DOIUrl":"https://doi.org/10.1145/2660579.2660590","url":null,"abstract":"Aphasia is a disease that renders its victims unable to effectively use language. Evidence supports the efficacy of treatment for aphasia yet the effectiveness or transferability of learned communicative abilities to everyday conversation continues to be investigated. In this paper we explore an alternative approach to aphasia treatment based on the art and science of storytelling. Inherent in storytelling are the motivations to share an experience, the cognitive abilities to organize story, and the language system to convey the experience. This approach is based on decades of research in aphasia therapy and immersive storytelling (in other fields) and has been used to engage a subject's creativity and emotions to produce transformative results in real life. We report on early, promising results that could radically innovate the rehabilitative practice of aphasia.","PeriodicalId":391229,"journal":{"name":"ImmersiveMe '14","volume":"223 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2014-11-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126999847","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：