Pub Date : 2019-12-01DOI: 10.1109/AIVR46125.2019.00024
Zhenjie Zhao, Feng Han, Xiaojuan Ma
Virtual reality (VR) is a promising new medium for immersive storytelling. While previous research works on VR narrative have tried to engage audiences through nice scenes and interactivity, the emerging live streaming shows the role of a presenter, especially the conveyance of emotion, for promoting audience involvement and enjoyment. In this paper, to lower the requirement of emotion embodiment, we borrow experience from cartoon animation and comics, and propose a novel cartoon-style hybrid emotion embodiment model to increase a storyteller's presence during live performance, which contains an avatar with six basic emotions and auxiliary multimodal display to enhance emotion expressing. We further design and implement a system to teleoperate the embodiment model in VR for live storytelling. In particular, 1) we design a novel visual programming tool that allows users to customize emotional effects based on the emotion embodiment model; 2) we design a novel face tracking module to map presenters' emotional states to the avatar in VR. Our lightweight web-based implementation also makes the application very easy to use. We conduct two preliminary qualitative studies to explore the potential of the hybrid model and the storytelling system, including interviews with three experts and a workshop study with local secondary school students. Results show the potential of the VR storytelling system for education.
{"title":"A Live Storytelling Virtual Reality System with Programmable Cartoon-Style Emotion Embodiment","authors":"Zhenjie Zhao, Feng Han, Xiaojuan Ma","doi":"10.1109/AIVR46125.2019.00024","DOIUrl":"https://doi.org/10.1109/AIVR46125.2019.00024","url":null,"abstract":"Virtual reality (VR) is a promising new medium for immersive storytelling. While previous research works on VR narrative have tried to engage audiences through nice scenes and interactivity, the emerging live streaming shows the role of a presenter, especially the conveyance of emotion, for promoting audience involvement and enjoyment. In this paper, to lower the requirement of emotion embodiment, we borrow experience from cartoon animation and comics, and propose a novel cartoon-style hybrid emotion embodiment model to increase a storyteller's presence during live performance, which contains an avatar with six basic emotions and auxiliary multimodal display to enhance emotion expressing. We further design and implement a system to teleoperate the embodiment model in VR for live storytelling. In particular, 1) we design a novel visual programming tool that allows users to customize emotional effects based on the emotion embodiment model; 2) we design a novel face tracking module to map presenters' emotional states to the avatar in VR. Our lightweight web-based implementation also makes the application very easy to use. We conduct two preliminary qualitative studies to explore the potential of the hybrid model and the storytelling system, including interviews with three experts and a workshop study with local secondary school students. Results show the potential of the VR storytelling system for education.","PeriodicalId":274566,"journal":{"name":"2019 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR)","volume":"3 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125499447","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2019-12-01DOI: 10.1109/AIVR46125.2019.00038
Xianglong Feng, Zeyang Bao, Sheng Wei
Live virtual reality streaming (a.k.a., 360-degree video streaming) is gaining popularity recently with its rapid growth in the consumer market. However, the huge bandwidth required by delivering the 360-degree frames becomes the bottleneck, keeping this application from a wider range of deployment. Research efforts have been carried out to solve the bandwidth problem by predicting the user's viewport of interest and selectively streaming a part of the whole frame. However, currently most of the viewport prediction approaches cannot address the unique challenges in the live streaming scenario, where there is no historical user or video traces to build the prediction model. In this paper, we explore the opportunity of leveraging convolutional neural network (CNN) to predict the user's viewport in live streaming by modifying the workflow of the CNN application and the training/testing process. The evaluation results reveal that the CNN-based method could achieve a high prediction accuracy with low bandwidth usage and low timing overhead.
{"title":"Exploring CNN-Based Viewport Prediction for Live Virtual Reality Streaming","authors":"Xianglong Feng, Zeyang Bao, Sheng Wei","doi":"10.1109/AIVR46125.2019.00038","DOIUrl":"https://doi.org/10.1109/AIVR46125.2019.00038","url":null,"abstract":"Live virtual reality streaming (a.k.a., 360-degree video streaming) is gaining popularity recently with its rapid growth in the consumer market. However, the huge bandwidth required by delivering the 360-degree frames becomes the bottleneck, keeping this application from a wider range of deployment. Research efforts have been carried out to solve the bandwidth problem by predicting the user's viewport of interest and selectively streaming a part of the whole frame. However, currently most of the viewport prediction approaches cannot address the unique challenges in the live streaming scenario, where there is no historical user or video traces to build the prediction model. In this paper, we explore the opportunity of leveraging convolutional neural network (CNN) to predict the user's viewport in live streaming by modifying the workflow of the CNN application and the training/testing process. The evaluation results reveal that the CNN-based method could achieve a high prediction accuracy with low bandwidth usage and low timing overhead.","PeriodicalId":274566,"journal":{"name":"2019 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR)","volume":"48 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127608102","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2019-12-01DOI: 10.1109/AIVR46125.2019.00013
Kangsoo Kim, Nahal Norouzi, Tiffany Losekamp, G. Bruder, Mindi Anderson, G. Welch
Providers of patient care environments are facing an increasing demand for technological solutions that can facilitate increased patient satisfaction while being cost effective and practically feasible. Recent developments with respect to smart hospital room setups and smart home care environments have an immense potential to leverage advances in technologies such as Intelligent Virtual Agents, Internet of Things devices, and Augmented Reality to enable novel forms of patient interaction with caregivers and their environment. In this paper, we present a human-subjects study in which we compared four types of simulated patient care environments for a range of typical tasks. In particular, we tested two forms of caregiver mediation with a real person or a virtual agent, and we compared two forms of caregiver embodiment with disembodied verbal or embodied interaction. Our results show that, as expected, a real caregiver provides the optimal user experience but an embodied virtual assistant is also a viable option for patient care environments, providing significantly higher social presence and engagement than voice-only interaction. We discuss the implications in the field of patient care and digital assistant.
{"title":"Effects of Patient Care Assistant Embodiment and Computer Mediation on User Experience","authors":"Kangsoo Kim, Nahal Norouzi, Tiffany Losekamp, G. Bruder, Mindi Anderson, G. Welch","doi":"10.1109/AIVR46125.2019.00013","DOIUrl":"https://doi.org/10.1109/AIVR46125.2019.00013","url":null,"abstract":"Providers of patient care environments are facing an increasing demand for technological solutions that can facilitate increased patient satisfaction while being cost effective and practically feasible. Recent developments with respect to smart hospital room setups and smart home care environments have an immense potential to leverage advances in technologies such as Intelligent Virtual Agents, Internet of Things devices, and Augmented Reality to enable novel forms of patient interaction with caregivers and their environment. In this paper, we present a human-subjects study in which we compared four types of simulated patient care environments for a range of typical tasks. In particular, we tested two forms of caregiver mediation with a real person or a virtual agent, and we compared two forms of caregiver embodiment with disembodied verbal or embodied interaction. Our results show that, as expected, a real caregiver provides the optimal user experience but an embodied virtual assistant is also a viable option for patient care environments, providing significantly higher social presence and engagement than voice-only interaction. We discuss the implications in the field of patient care and digital assistant.","PeriodicalId":274566,"journal":{"name":"2019 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR)","volume":"17 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129876552","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2019-12-01DOI: 10.1109/AIVR46125.2019.00022
Irene Viola, J. Mulder, F. D. Simone, Pablo César
In recent years, there has been an increased interest in point cloud representation for visualizing digital humans in cross reality. However, due to their voluminous size, point clouds require high bandwidth to be transmitted. In this paper, we propose a temporal interpolation architecture capable of increasing the temporal resolution of dynamic digital humans, represented using point clouds. With this technique, bandwidth savings can be achieved by transmitting dynamic point clouds in a lower temporal resolution, and recreating a higher temporal resolution on the receiving side. Our interpolation architecture works by first downsampling the point clouds to a lower spatial resolution, then estimating scene flow using a newly designed neural network architecture, and finally upsampling the result back to the original spatial resolution. To improve the smoothness of the results, we additionally apply a novel technique called neighbour snapping. To be able to train and test our newly designed network, we created a synthetic point cloud data set of animated human bodies. Results from the evaluation of our architecture through a small-scale user study show the benefits of our method with respect to the state of the art in scene flow estimation for point clouds. Moreover, correlation between our user study and existing objective quality metrics confirm the need for new metrics to accurately predict the visual quality of point cloud contents.
{"title":"Temporal Interpolation of Dynamic Digital Humans using Convolutional Neural Networks","authors":"Irene Viola, J. Mulder, F. D. Simone, Pablo César","doi":"10.1109/AIVR46125.2019.00022","DOIUrl":"https://doi.org/10.1109/AIVR46125.2019.00022","url":null,"abstract":"In recent years, there has been an increased interest in point cloud representation for visualizing digital humans in cross reality. However, due to their voluminous size, point clouds require high bandwidth to be transmitted. In this paper, we propose a temporal interpolation architecture capable of increasing the temporal resolution of dynamic digital humans, represented using point clouds. With this technique, bandwidth savings can be achieved by transmitting dynamic point clouds in a lower temporal resolution, and recreating a higher temporal resolution on the receiving side. Our interpolation architecture works by first downsampling the point clouds to a lower spatial resolution, then estimating scene flow using a newly designed neural network architecture, and finally upsampling the result back to the original spatial resolution. To improve the smoothness of the results, we additionally apply a novel technique called neighbour snapping. To be able to train and test our newly designed network, we created a synthetic point cloud data set of animated human bodies. Results from the evaluation of our architecture through a small-scale user study show the benefits of our method with respect to the state of the art in scene flow estimation for point clouds. Moreover, correlation between our user study and existing objective quality metrics confirm the need for new metrics to accurately predict the visual quality of point cloud contents.","PeriodicalId":274566,"journal":{"name":"2019 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR)","volume":"37 ","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133621781","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2019-12-01DOI: 10.1109/AIVR46125.2019.00046
Mengyi Zhao, S. Dai
The haptic feedback system (HFS) in the virtual cockpit system (VCS) can definitely enhance the sense of immersion. Most HFSs in prior works sacrificed the native advantages of VCSs to achieve haptic interaction. This paper addresses the problem by proposing a novel framework for the HFS, which can predict the most likely interacting target of the human hand in advance. We introduce a HFS with a non-contact visual tracking sensor and a probability inference method based on Bayesian statistics, the features extracted by this HFS could be low-cost, high generality and flexibility. Simulations show that human intent inference can be computed in real-time and the results can meet the requirements of the HFM, which provides an important basis for haptic interactions in VCSs.
{"title":"Intent Inference of Human Hand Motion for Haptic Feedback Systems","authors":"Mengyi Zhao, S. Dai","doi":"10.1109/AIVR46125.2019.00046","DOIUrl":"https://doi.org/10.1109/AIVR46125.2019.00046","url":null,"abstract":"The haptic feedback system (HFS) in the virtual cockpit system (VCS) can definitely enhance the sense of immersion. Most HFSs in prior works sacrificed the native advantages of VCSs to achieve haptic interaction. This paper addresses the problem by proposing a novel framework for the HFS, which can predict the most likely interacting target of the human hand in advance. We introduce a HFS with a non-contact visual tracking sensor and a probability inference method based on Bayesian statistics, the features extracted by this HFS could be low-cost, high generality and flexibility. Simulations show that human intent inference can be computed in real-time and the results can meet the requirements of the HFM, which provides an important basis for haptic interactions in VCSs.","PeriodicalId":274566,"journal":{"name":"2019 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR)","volume":"82 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133317359","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2019-12-01DOI: 10.1109/AIVR46125.2019.00036
Amit L. Ahire, Dennis Basgier
Mobile platform tools are desirable when it comes to practical augmented reality applications. With the convenience and portability that the form factor has to offer, it lays an ideal basic foundation for a feasible use case in industry and commercial applications. Here, we present a novel approach of using the monocular Simultaneous Localization and Mapping (SLAM) information provided by a Cross-Reality (XR) device to augment the linked 3D CAD models. The main objective is to use the tracking technology for an augmented and mixed reality experience by tracking a 3D model and superimposing its respective 3D CAD model data over the images we receive from the camera feed of the XR device without any scene preparation (e.g markers or feature maps). The intent is to conduct a visual analysis and evaluations based on the intrinsic and extrinsic of the model in the visualization system that instant3Dhub has to offer. To achieve this we make use of the Apple's ARKit to obtain the images, sensor data and SLAM heuristic of client XR device, remote marker-less model based 3D object tracking from monocular RGB image data and hybrid client server architecture. Our approach is agnostic of any SLAM system or Augmented Reality (AR) framework. We make use of the Apple's ARKit because of the its ease of use, affordability, stability and maturity as a platform and as an integrated system.
{"title":"AR Tracking with Hybrid, Agnostic And Browser Based Approach","authors":"Amit L. Ahire, Dennis Basgier","doi":"10.1109/AIVR46125.2019.00036","DOIUrl":"https://doi.org/10.1109/AIVR46125.2019.00036","url":null,"abstract":"Mobile platform tools are desirable when it comes to practical augmented reality applications. With the convenience and portability that the form factor has to offer, it lays an ideal basic foundation for a feasible use case in industry and commercial applications. Here, we present a novel approach of using the monocular Simultaneous Localization and Mapping (SLAM) information provided by a Cross-Reality (XR) device to augment the linked 3D CAD models. The main objective is to use the tracking technology for an augmented and mixed reality experience by tracking a 3D model and superimposing its respective 3D CAD model data over the images we receive from the camera feed of the XR device without any scene preparation (e.g markers or feature maps). The intent is to conduct a visual analysis and evaluations based on the intrinsic and extrinsic of the model in the visualization system that instant3Dhub has to offer. To achieve this we make use of the Apple's ARKit to obtain the images, sensor data and SLAM heuristic of client XR device, remote marker-less model based 3D object tracking from monocular RGB image data and hybrid client server architecture. Our approach is agnostic of any SLAM system or Augmented Reality (AR) framework. We make use of the Apple's ARKit because of the its ease of use, affordability, stability and maturity as a platform and as an integrated system.","PeriodicalId":274566,"journal":{"name":"2019 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR)","volume":"34 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121054993","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2019-12-01DOI: 10.1109/AIVR46125.2019.00055
Vuthea Chheang, P. Saalfeld, T. Huber, F. Huettl, W. Kneist, B. Preim, C. Hansen
We introduce a collaborative virtual reality (VR) system for planning and simulation in laparoscopic liver surgery training. Patient image data is used for surgical model visualization and simulation. We developed two modes for training in laparoscopic procedures: exploration and surgery mode. Surgical joysticks are used in surgery mode to provide training for psychomotor skills and cooperation between a camera assistant and an experienced surgeon. Continuous feedback from our clinical partner comprised an important part of the development. Our evaluation showed that surgeons were positive about the usability and usefulness of the developed system. For further details, please refer to our full article and additional materials.
{"title":"An Interactive Demonstration of Collaborative VR for Laparoscopic Liver Surgery Training","authors":"Vuthea Chheang, P. Saalfeld, T. Huber, F. Huettl, W. Kneist, B. Preim, C. Hansen","doi":"10.1109/AIVR46125.2019.00055","DOIUrl":"https://doi.org/10.1109/AIVR46125.2019.00055","url":null,"abstract":"We introduce a collaborative virtual reality (VR) system for planning and simulation in laparoscopic liver surgery training. Patient image data is used for surgical model visualization and simulation. We developed two modes for training in laparoscopic procedures: exploration and surgery mode. Surgical joysticks are used in surgery mode to provide training for psychomotor skills and cooperation between a camera assistant and an experienced surgeon. Continuous feedback from our clinical partner comprised an important part of the development. Our evaluation showed that surgeons were positive about the usability and usefulness of the developed system. For further details, please refer to our full article and additional materials.","PeriodicalId":274566,"journal":{"name":"2019 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR)","volume":"139 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127386088","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2019-12-01DOI: 10.1109/AIVR46125.2019.00053
A. Bahremand, Linda D. Nguyen, Tanya N. Harrison, R. Likamwa
We envision that in the future, presentations for business, education, and scientific dissemination can invoke 3D spatial content to immersively display and discuss animated 3-dimensional models and spatial data visualizations to large audiences. At the moment, current frameworks have targeted a highly technical user base, prohibiting the widespread curation of immersive presentations. Furthermore, solutions for real-time multi-user interactions have focused on multiplayer gaming, rather than large format immersive presentation. However, modern mobile devices (smartphones, tablets, headsets) have the capability of rendering virtual models over the physical environment through visual anchors for Augmented Reality (AR). Our ongoing research thrust is to leverage contemporary AR infrastructure to develop an easy-to use tool for users to curate and spatially present augmented presentations to large audiences. In this demo, we have built an Augmented Reality framework that allows users to curate mixed reality presentations. Our framework allows users to prepare a sequential state of animations. At the time of presentation, presenters can invoke the animations to simultaneously occur on HMDs and mobile devices.
{"title":"HoloLucination: A Framework for Live Augmented Reality Presentations Across Mobile Devices","authors":"A. Bahremand, Linda D. Nguyen, Tanya N. Harrison, R. Likamwa","doi":"10.1109/AIVR46125.2019.00053","DOIUrl":"https://doi.org/10.1109/AIVR46125.2019.00053","url":null,"abstract":"We envision that in the future, presentations for business, education, and scientific dissemination can invoke 3D spatial content to immersively display and discuss animated 3-dimensional models and spatial data visualizations to large audiences. At the moment, current frameworks have targeted a highly technical user base, prohibiting the widespread curation of immersive presentations. Furthermore, solutions for real-time multi-user interactions have focused on multiplayer gaming, rather than large format immersive presentation. However, modern mobile devices (smartphones, tablets, headsets) have the capability of rendering virtual models over the physical environment through visual anchors for Augmented Reality (AR). Our ongoing research thrust is to leverage contemporary AR infrastructure to develop an easy-to use tool for users to curate and spatially present augmented presentations to large audiences. In this demo, we have built an Augmented Reality framework that allows users to curate mixed reality presentations. Our framework allows users to prepare a sequential state of animations. At the time of presentation, presenters can invoke the animations to simultaneously occur on HMDs and mobile devices.","PeriodicalId":274566,"journal":{"name":"2019 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR)","volume":"12 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133177997","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
In this study, we describe research carried out to develop a context aware augmented reality storyteller. The system allows users to listen to short stories from books and novels as they move around in the physical world based on the spatial characteristics of their physical location. Large-Scale Open web data from sources such as Open Street Map, Google Street View and Project Gutenberg were collected and used to implement our system. Word embedding based modeling and Natural Language Processing techniques were used to match the stories with the physical locations. As an example, we show how our proposed approach could be implemented to map stories from Aesop's Fables to different locations in two cities, Kyoto and San Francisco.
在本研究中，我们描述了为开发上下文感知增强现实故事讲述者而进行的研究。该系统允许用户在物理世界中根据其物理位置的空间特征四处走动时，听书籍和小说中的短篇故事。我们收集了来自Open Street Map、Google Street View和Project Gutenberg等来源的大规模开放网络数据，并将其用于实现我们的系统。使用基于词嵌入的建模和自然语言处理技术将故事与物理位置进行匹配。作为一个例子，我们展示了如何实现我们提出的方法，将《伊索寓言》中的故事映射到两个城市(京都和旧金山)的不同位置。
{"title":"A Book Without Pages: Developing an Augmented Reality Storyteller Based on Open Web Data","authors":"Panote Siriaraya, Takumi Kiriu, Yukiko Kawai, Shinsuke Nakajima","doi":"10.1109/AIVR46125.2019.00023","DOIUrl":"https://doi.org/10.1109/AIVR46125.2019.00023","url":null,"abstract":"In this study, we describe research carried out to develop a context aware augmented reality storyteller. The system allows users to listen to short stories from books and novels as they move around in the physical world based on the spatial characteristics of their physical location. Large-Scale Open web data from sources such as Open Street Map, Google Street View and Project Gutenberg were collected and used to implement our system. Word embedding based modeling and Natural Language Processing techniques were used to match the stories with the physical locations. As an example, we show how our proposed approach could be implemented to map stories from Aesop's Fables to different locations in two cities, Kyoto and San Francisco.","PeriodicalId":274566,"journal":{"name":"2019 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR)","volume":"236 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132153701","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2019-12-01DOI: 10.1109/AIVR46125.2019.00031
Nadine Meissler, Annika Wohlan, N. Hochgeschwender, A. Schreiber
Software systems and components are increasingly based on machine learning methods, such as Convolutional Neural Networks (CNNs). Thus, there is a growing need for common programmers and machine learning newcomers to understand the general functioning of these algorithms. However, as neural networks are complex in nature, novel presentation means are required to enable rapid access to the functionality. For that purpose, we examine how CNNs can be visualized in Virtual Reality (VR), as it offers the opportunity to focus users on content through effects such as immersion and presence. With a first exploratory study, we confirmed that our visualization approach is both intuitive to use and conductive to learning. Moreover, users indicated an increased motivation to learning due to the unusual virtual environment. Based on our findings, we propose a follow-up study that specifically compares the benefits of a virtual visualization approach to a traditional desktop visualization.
{"title":"Using Visualization of Convolutional Neural Networks in Virtual Reality for Machine Learning Newcomers","authors":"Nadine Meissler, Annika Wohlan, N. Hochgeschwender, A. Schreiber","doi":"10.1109/AIVR46125.2019.00031","DOIUrl":"https://doi.org/10.1109/AIVR46125.2019.00031","url":null,"abstract":"Software systems and components are increasingly based on machine learning methods, such as Convolutional Neural Networks (CNNs). Thus, there is a growing need for common programmers and machine learning newcomers to understand the general functioning of these algorithms. However, as neural networks are complex in nature, novel presentation means are required to enable rapid access to the functionality. For that purpose, we examine how CNNs can be visualized in Virtual Reality (VR), as it offers the opportunity to focus users on content through effects such as immersion and presence. With a first exploratory study, we confirmed that our visualization approach is both intuitive to use and conductive to learning. Moreover, users indicated an increased motivation to learning due to the unusual virtual environment. Based on our findings, we propose a follow-up study that specifically compares the benefits of a virtual visualization approach to a traditional desktop visualization.","PeriodicalId":274566,"journal":{"name":"2019 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR)","volume":"286 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132104157","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：