We present a VR-based simulator built for training surgical skills in the procedure of catheter ablation. Based on multi-body dynamics, we proposed a novel method to simulate the interactive behavior of the surgical devices and the human vascular system. An estimation based optimization technique and a track based motion control strategy are proposed to make the simulation efficient enough to achieve high level performance. The beating of human heart is also simulated in real time with our method within the position based dynamics framework. Results demonstrate that our simulator provides a realistic, effective, and stable environment for trainees to acquire essential surgical skills.
{"title":"A Virtual Reality Based Simulator for Training Surgical Skills in Procedure of Catheter Ablation","authors":"Haoyu Wang, Sheng Jiang, Jianhuang Wu","doi":"10.1109/AIVR.2018.00057","DOIUrl":"https://doi.org/10.1109/AIVR.2018.00057","url":null,"abstract":"We present a VR-based simulator built for training surgical skills in the procedure of catheter ablation. Based on multi-body dynamics, we proposed a novel method to simulate the interactive behavior of the surgical devices and the human vascular system. An estimation based optimization technique and a track based motion control strategy are proposed to make the simulation efficient enough to achieve high level performance. The beating of human heart is also simulated in real time with our method within the position based dynamics framework. Results demonstrate that our simulator provides a realistic, effective, and stable environment for trainees to acquire essential surgical skills.","PeriodicalId":371868,"journal":{"name":"2018 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR)","volume":"20 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133793588","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
We are developing a system to support remote audiences of live music shows. At live shows of rock or popular music, audiences take actions responding to the music. This is a style of nonverbal communication between audiences to share emotion. The "sense of unity", which is often mentioned among music players and audiences, is a key of successful live performance. This research tries to enable remote audiences to exchange nonverbal communication with body actions between them in a VR environment. We have developed a prototype system and conducted evaluation sessions. Most of the participants of the sessions felt the sense of unity.
{"title":"Supporting the Sense of Unity between Remote Audiences in VR-Based Remote Live Music Support System KSA2","authors":"Tatsuyoshi Kaneko, H. Tarumi, Keiya Kataoka, Yuki Kubochi, Daiki Yamashita, Tomoki Nakai, Ryota Yamaguchi","doi":"10.1109/AIVR.2018.00025","DOIUrl":"https://doi.org/10.1109/AIVR.2018.00025","url":null,"abstract":"We are developing a system to support remote audiences of live music shows. At live shows of rock or popular music, audiences take actions responding to the music. This is a style of nonverbal communication between audiences to share emotion. The \"sense of unity\", which is often mentioned among music players and audiences, is a key of successful live performance. This research tries to enable remote audiences to exchange nonverbal communication with body actions between them in a VR environment. We have developed a prototype system and conducted evaluation sessions. Most of the participants of the sessions felt the sense of unity.","PeriodicalId":371868,"journal":{"name":"2018 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR)","volume":"4 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130778862","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Yuhao Ma, Hao Guo, Hong Chen, Mengxiao Tian, Xin Huo, Chengjiang Long, Shiye Tang, Xiaoyu Song, Qing Wang
Convolutional neural networks (CNN) have shown to be useful for camera pose regression, and They have robust effects against some challenging scenarios such as lighting changes, motion blur, and scenes with lots of textureless surfaces. Additionally, PoseNet shows that the deep learning system can interpolate the camera pose in space between training images. In this paper, we explore how different strategies for processing datasets will affect the pose regression and propose a method for building multi-scene datasets for training such neural networks. We demonstrate that the location of several scenes can be remembered using only one neural network. By combining multiple scenes, we found that the position errors of the neural network do not decrease significantly as the distance between the cameras increases, which means that we do not need to train several models for the increase number of scenes. We also explore the impact factors that influence the accuracy of models for multi-scene camera pose regression, which can help us merge several scenes into one dataset in a better way. We opened our code and datasets to the public for better researches.
{"title":"A Method to Build Multi-Scene Datasets for CNN for Camera Pose Regression","authors":"Yuhao Ma, Hao Guo, Hong Chen, Mengxiao Tian, Xin Huo, Chengjiang Long, Shiye Tang, Xiaoyu Song, Qing Wang","doi":"10.1109/AIVR.2018.00022","DOIUrl":"https://doi.org/10.1109/AIVR.2018.00022","url":null,"abstract":"Convolutional neural networks (CNN) have shown to be useful for camera pose regression, and They have robust effects against some challenging scenarios such as lighting changes, motion blur, and scenes with lots of textureless surfaces. Additionally, PoseNet shows that the deep learning system can interpolate the camera pose in space between training images. In this paper, we explore how different strategies for processing datasets will affect the pose regression and propose a method for building multi-scene datasets for training such neural networks. We demonstrate that the location of several scenes can be remembered using only one neural network. By combining multiple scenes, we found that the position errors of the neural network do not decrease significantly as the distance between the cameras increases, which means that we do not need to train several models for the increase number of scenes. We also explore the impact factors that influence the accuracy of models for multi-scene camera pose regression, which can help us merge several scenes into one dataset in a better way. We opened our code and datasets to the public for better researches.","PeriodicalId":371868,"journal":{"name":"2018 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR)","volume":"38 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128826178","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Young novice drivers are the group of drivers most likely to crash. Young novice drivers who tend to misidentify potential hazards in the traffic environment. There are a number of factors that contribute to the high crash risk experienced by these drivers. Age and lack of driving experience are the main factors behind young drivers having an increased risk of being involved in a road traffic collision. In Taiwan when novice drivers pass the driving test, they need to attend classroom instruction to learn safe driving from fatal crashes. But the classroom instruction has limited direct beneficial effects on the safety of new drivers. In this paper, the researchers use Unity game engine to develop an Android mobile game to enhance learning in driving classroom instruction. The game has to be played at three levels: the learning, the examination and the free drive level. The researchers also develop VR game and simulator based on mobile game which can be used to enhance learning in driving education.
{"title":"Motorcycle Riding Safety Education with Virtual Reality","authors":"Chun-Chia Hsu, Y. Chen, Wen Ching Chou, Shih-Hsuan Huang, Kai-Kuo Chang","doi":"10.1109/AIVR.2018.00050","DOIUrl":"https://doi.org/10.1109/AIVR.2018.00050","url":null,"abstract":"Young novice drivers are the group of drivers most likely to crash. Young novice drivers who tend to misidentify potential hazards in the traffic environment. There are a number of factors that contribute to the high crash risk experienced by these drivers. Age and lack of driving experience are the main factors behind young drivers having an increased risk of being involved in a road traffic collision. In Taiwan when novice drivers pass the driving test, they need to attend classroom instruction to learn safe driving from fatal crashes. But the classroom instruction has limited direct beneficial effects on the safety of new drivers. In this paper, the researchers use Unity game engine to develop an Android mobile game to enhance learning in driving classroom instruction. The game has to be played at three levels: the learning, the examination and the free drive level. The researchers also develop VR game and simulator based on mobile game which can be used to enhance learning in driving education.","PeriodicalId":371868,"journal":{"name":"2018 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR)","volume":"29 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122692953","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Cardiac circulation is traditionally difficult for students from both high school and colleges to learn biology due its complexity and dynamic process. Cadavers can be used as a good tool for students to learn cardiovascular system. Unfortunately, this approach comes with some major concerns because of the collapsed anatomic structures, and no blood flowing. Also cadavers are normally available only in medical schools. Human subjects especially cardiac patients are ideal for students to learn human circulation from the perspectives of both structures and functions. This, however, is infeasible due to ethical and other constraints. As such most students learn cardiac circulation by reading text books, attending to lectures, viewing images and perhaps manipulating heart models. In this paper, we will present our efforts developing augmented reality (AR) technology to enhance learning of cardiac circulation. More specifically, a book based AR app APPLearn (Heart) is designed to allow each and every student learning the cardiac structure and function by interactive playing.
{"title":"Augmented Reality Simulation of Cardiac Circulation Using APPLearn (Heart)","authors":"R. Ba, Yiyu Cai, Yunqing Guan","doi":"10.1109/AIVR.2018.00055","DOIUrl":"https://doi.org/10.1109/AIVR.2018.00055","url":null,"abstract":"Cardiac circulation is traditionally difficult for students from both high school and colleges to learn biology due its complexity and dynamic process. Cadavers can be used as a good tool for students to learn cardiovascular system. Unfortunately, this approach comes with some major concerns because of the collapsed anatomic structures, and no blood flowing. Also cadavers are normally available only in medical schools. Human subjects especially cardiac patients are ideal for students to learn human circulation from the perspectives of both structures and functions. This, however, is infeasible due to ethical and other constraints. As such most students learn cardiac circulation by reading text books, attending to lectures, viewing images and perhaps manipulating heart models. In this paper, we will present our efforts developing augmented reality (AR) technology to enhance learning of cardiac circulation. More specifically, a book based AR app APPLearn (Heart) is designed to allow each and every student learning the cardiac structure and function by interactive playing.","PeriodicalId":371868,"journal":{"name":"2018 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR)","volume":"19 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125683260","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Shu Naritomi, Ryosuke Tanno, Takumi Ege, Keiji Yanai
In this demonstration, we implemented food category transformation in mixed reality using both image generation and HoloLens. Our system overlays transformed food images to food objects in the AR space, so that it is possible to convert in consideration of real shape. This system has the potential to make meals more enjoyable. In this work, we use the Conditional CycleGAN trained with a large-scale food image data collected from the Twitter Stream for food category transformation which can transform among ten kinds of foods mutually keeping the shape of a given food. We show the virtual meal experience which is food category transformation among ten kinds of typical Japanese foods: ramen noodle, curry rice, fried rice, beef rice bowl, chilled noodle, spaghetti with meat source, white rice, eel bowl, and fried noodle. Note that additional results including demo videos can be see at https://negi111111.github.io/FoodChangeLensProjectHP/
{"title":"FoodChangeLens: CNN-Based Food Transformation on HoloLens","authors":"Shu Naritomi, Ryosuke Tanno, Takumi Ege, Keiji Yanai","doi":"10.1109/AIVR.2018.00046","DOIUrl":"https://doi.org/10.1109/AIVR.2018.00046","url":null,"abstract":"In this demonstration, we implemented food category transformation in mixed reality using both image generation and HoloLens. Our system overlays transformed food images to food objects in the AR space, so that it is possible to convert in consideration of real shape. This system has the potential to make meals more enjoyable. In this work, we use the Conditional CycleGAN trained with a large-scale food image data collected from the Twitter Stream for food category transformation which can transform among ten kinds of foods mutually keeping the shape of a given food. We show the virtual meal experience which is food category transformation among ten kinds of typical Japanese foods: ramen noodle, curry rice, fried rice, beef rice bowl, chilled noodle, spaghetti with meat source, white rice, eel bowl, and fried noodle. Note that additional results including demo videos can be see at https://negi111111.github.io/FoodChangeLensProjectHP/","PeriodicalId":371868,"journal":{"name":"2018 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR)","volume":"60 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116943416","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
In this research we explore the effect of a virtual avatar that is non-human like and can express basic distinguishable emotions on users' level of engagement and interest. Virtual reality (VR) environments are able to render realistic representations. However, not all virtual environments require life-like representations of their characters-in our research a 'life-like' human character means that it resembles very closely to an actual person in real life. It is very common for games to use simple non-human characters. Cartoon-like characters can actually have a greater impact on users' affinity towards these games. The aim of this research is to examine if interactions with a cartoon-like character that has the capacity to express simple but common emotional expressions is sufficient to bring forth a change in the behavior and level of engagement of users with the character. This research seeks to find out if adding simple emotions to virtual characters is beneficial to increasing users' interest. To explore these questions, we have conducted a study with a human-like cartoon character in a VR environment that can express simple, basic human emotions based on users' input. The results of our experiment show that a cartoon-like character can benefit from displaying emotional traits or responses when interacting with humans in a VR environment.
{"title":"Evaluating the Effects of a Cartoon-Like Character with Emotions on Users' Behaviour within Virtual Reality Environments","authors":"D. Monteiro, Hai-Ning Liang, Jialin Wang, Luhan Wang, Xian Wang, Yong Yue","doi":"10.1109/AIVR.2018.00053","DOIUrl":"https://doi.org/10.1109/AIVR.2018.00053","url":null,"abstract":"In this research we explore the effect of a virtual avatar that is non-human like and can express basic distinguishable emotions on users' level of engagement and interest. Virtual reality (VR) environments are able to render realistic representations. However, not all virtual environments require life-like representations of their characters-in our research a 'life-like' human character means that it resembles very closely to an actual person in real life. It is very common for games to use simple non-human characters. Cartoon-like characters can actually have a greater impact on users' affinity towards these games. The aim of this research is to examine if interactions with a cartoon-like character that has the capacity to express simple but common emotional expressions is sufficient to bring forth a change in the behavior and level of engagement of users with the character. This research seeks to find out if adding simple emotions to virtual characters is beneficial to increasing users' interest. To explore these questions, we have conducted a study with a human-like cartoon character in a VR environment that can express simple, basic human emotions based on users' input. The results of our experiment show that a cartoon-like character can benefit from displaying emotional traits or responses when interacting with humans in a VR environment.","PeriodicalId":371868,"journal":{"name":"2018 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR)","volume":"34 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127180165","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
In this paper, we propose a novel pipeline for semi-supervised behavioral coding of videos of users testing a device or interface, with an eye toward human-computer interaction evaluation for virtual reality. Our system applies existing statistical techniques for time-series classification, including e-divisive change point detection and "Symbolic Aggregate approXimation" (SAX) with agglomerative hierarchical clustering, to 3D pose telemetry data. These techniques create classes of short segments of single-person video data–short actions of potential interest called "micro-gestures." A long short-term memory (LSTM) layer then learns these micro-gestures from pose features generated purely from video via a pre-trained OpenPose convolutional neural network (CNN) to predict their occurrence in unlabeled test videos. We present and discuss the results from testing our system on the single user pose videos of the CMU Panoptic Dataset.
{"title":"Gesture and Action Discovery for Evaluating Virtual Environments with Semi-Supervised Segmentation of Telemetry Records","authors":"A. Batch, Kyungjun Lee, H. Maddali, N. Elmqvist","doi":"10.1109/AIVR.2018.00009","DOIUrl":"https://doi.org/10.1109/AIVR.2018.00009","url":null,"abstract":"In this paper, we propose a novel pipeline for semi-supervised behavioral coding of videos of users testing a device or interface, with an eye toward human-computer interaction evaluation for virtual reality. Our system applies existing statistical techniques for time-series classification, including e-divisive change point detection and \"Symbolic Aggregate approXimation\" (SAX) with agglomerative hierarchical clustering, to 3D pose telemetry data. These techniques create classes of short segments of single-person video data–short actions of potential interest called \"micro-gestures.\" A long short-term memory (LSTM) layer then learns these micro-gestures from pose features generated purely from video via a pre-trained OpenPose convolutional neural network (CNN) to predict their occurrence in unlabeled test videos. We present and discuss the results from testing our system on the single user pose videos of the CMU Panoptic Dataset.","PeriodicalId":371868,"journal":{"name":"2018 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR)","volume":"2 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127917165","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Object detection when viewing Head Mounted Display (HMD) imagery for maritime Search and Rescue (SAR) detection tasks poses many challenges, for example, objects are difficult to distinguish due to low contrast or low observability. We survey existing Artificial Intelligence (AI) image processing algorithms that improve object detection performance. We also examine central and peripheral vision (HVS) and their relation to Field of View (FOV) within the Human Visual System when viewing such images using HMDs. We present results from our user-study which simulates different maritime scenes used in object detection tasks. Users are tested viewing sample images with different visual features over different FOVs, to inform the development of an AI algorithm for object detection.
{"title":"Understanding Head-Mounted Display FOV in Maritime Search and Rescue Object Detection","authors":"Susannah Soon, A. Lugmayr, A. Woods, T. Tan","doi":"10.1109/AIVR.2018.00023","DOIUrl":"https://doi.org/10.1109/AIVR.2018.00023","url":null,"abstract":"Object detection when viewing Head Mounted Display (HMD) imagery for maritime Search and Rescue (SAR) detection tasks poses many challenges, for example, objects are difficult to distinguish due to low contrast or low observability. We survey existing Artificial Intelligence (AI) image processing algorithms that improve object detection performance. We also examine central and peripheral vision (HVS) and their relation to Field of View (FOV) within the Human Visual System when viewing such images using HMDs. We present results from our user-study which simulates different maritime scenes used in object detection tasks. Users are tested viewing sample images with different visual features over different FOVs, to inform the development of an AI algorithm for object detection.","PeriodicalId":371868,"journal":{"name":"2018 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR)","volume":"40 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114333118","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：