Pub Date : 2020-12-01DOI: 10.1109/AIVR50618.2020.00014
Juan Izquierdo-Domenech, Jordi Linares-Pellicer, Jorge Orta-Lopez
Even though Augmented Reality (AR) is far from its maturity, we already have solutions and devices that give us an efficient technological frame in different industrial environments. Widely used mobile devices, such as tablets, or more specific ones, such as the current AR glasses available, are enough to offer solutions that improve many industrial processes; repairing, maintenance, plant control, product line reconfiguration are some examples of these tasks. Many industrial applications already utilise AR-based applications to solve those problems. In this work, we aim to go a little bit further beyond current possibilities that only focus on providing visual guidance. Our main goal is to add a semantic layer for existing AR-based applications, that visually validate worker’s actions based on visual interpretation of switches, potentiometers, analog needles or buttons, among others. This semantic layer allows a new level of interaction by adding automatic interpretation of the context that affects the actions of the operator and the display of information of interest in the AR system. We propose and explain the architecture and training of the Convolutional Neural Networks (CNN) used for the semantic layer and its integration in the AR technology.
{"title":"Supporting interaction in augmented reality assisted industrial processes using a CNN-based semantic layer","authors":"Juan Izquierdo-Domenech, Jordi Linares-Pellicer, Jorge Orta-Lopez","doi":"10.1109/AIVR50618.2020.00014","DOIUrl":"https://doi.org/10.1109/AIVR50618.2020.00014","url":null,"abstract":"Even though Augmented Reality (AR) is far from its maturity, we already have solutions and devices that give us an efficient technological frame in different industrial environments. Widely used mobile devices, such as tablets, or more specific ones, such as the current AR glasses available, are enough to offer solutions that improve many industrial processes; repairing, maintenance, plant control, product line reconfiguration are some examples of these tasks. Many industrial applications already utilise AR-based applications to solve those problems. In this work, we aim to go a little bit further beyond current possibilities that only focus on providing visual guidance. Our main goal is to add a semantic layer for existing AR-based applications, that visually validate worker’s actions based on visual interpretation of switches, potentiometers, analog needles or buttons, among others. This semantic layer allows a new level of interaction by adding automatic interpretation of the context that affects the actions of the operator and the display of information of interest in the AR system. We propose and explain the architecture and training of the Convolutional Neural Networks (CNN) used for the semantic layer and its integration in the AR technology.","PeriodicalId":348199,"journal":{"name":"2020 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR)","volume":"29 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123307028","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2020-12-01DOI: 10.1109/AIVR50618.2020.00066
Daeyeol Chang, James Hopfenblatt, P. Edara, Bimal Balakrishnan
Construction and maintenance work on roads pose safety risks to both drivers and workers. The responsible agencies regularly inspect work zones for compliance with traffic control and signage standards. The current training practice is to review documents related to temporary traffic control and reports from previous inspections, typically Power Point files with pictures. It would be beneficial if a new mechanism for training could be developed that is as effective as field visits but without the amount of time and effort required to visit multiple field sites. This study developed an immersive training module for transportation agency staff that inspect flagger operations in road construction and maintenance work zones. Human flaggers are commonly used to control traffic at work zones on two lane highways (one lane in each direction). The main objective of the proposed training is to deliver a realistic experience to trainees in an immersive virtual environment using the current traffic control protocols and standards. The module creation consisted of three steps. First, the roadway geometrics, work zone signage, traffic control devices, and the natural environment was created. Second, motion capture technology was used to replicate the actual movement of a human flagger directing traffic in a work zone. The environment and flagger avatar created in the first two steps were integrated and implemented in a simulation in the third step. The module was demonstrated to inspection staff at one state department of transportation (DOT) and revised based on their feedback. The state DOT staff were highly receptive to the use of virtual reality for training and commented on the benefits of the immersive experience that is lacking in their current training practices.
{"title":"Immersive Virtual Reality Training for Inspecting Flagger Work zones","authors":"Daeyeol Chang, James Hopfenblatt, P. Edara, Bimal Balakrishnan","doi":"10.1109/AIVR50618.2020.00066","DOIUrl":"https://doi.org/10.1109/AIVR50618.2020.00066","url":null,"abstract":"Construction and maintenance work on roads pose safety risks to both drivers and workers. The responsible agencies regularly inspect work zones for compliance with traffic control and signage standards. The current training practice is to review documents related to temporary traffic control and reports from previous inspections, typically Power Point files with pictures. It would be beneficial if a new mechanism for training could be developed that is as effective as field visits but without the amount of time and effort required to visit multiple field sites. This study developed an immersive training module for transportation agency staff that inspect flagger operations in road construction and maintenance work zones. Human flaggers are commonly used to control traffic at work zones on two lane highways (one lane in each direction). The main objective of the proposed training is to deliver a realistic experience to trainees in an immersive virtual environment using the current traffic control protocols and standards. The module creation consisted of three steps. First, the roadway geometrics, work zone signage, traffic control devices, and the natural environment was created. Second, motion capture technology was used to replicate the actual movement of a human flagger directing traffic in a work zone. The environment and flagger avatar created in the first two steps were integrated and implemented in a simulation in the third step. The module was demonstrated to inspection staff at one state department of transportation (DOT) and revised based on their feedback. The state DOT staff were highly receptive to the use of virtual reality for training and commented on the benefits of the immersive experience that is lacking in their current training practices.","PeriodicalId":348199,"journal":{"name":"2020 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR)","volume":"13 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131278134","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2020-12-01DOI: 10.1109/AIVR50618.2020.00076
Tong Xue, Abdallah El Ali, Ding Gangyi, Pablo Santiago César Garcia
We demonstrate an HMD-based annotation tool for collecting precise emotion ground truth labels while users are watching 360° videos in Virtual Reality (VR). Our tool uses an HTC VIVE Pro Eye HMD for displaying 360° videos, a Joy-Con controller for inputting emotion annotations, and an Empatica E4 wristband for capturing physiological signals. Timestamps of these devices are synchronized via an NTP server. Following dimensional emotion models, users can report their emotion in terms of valence and arousal as they watch a video in VR. Annotation feedback is provided through two peripheral visualization techniques: HaloLight and DotSize. Our annotation tool provides a starting point for researchers to design momentary and continuous self-reports in virtual environments to enable fine-grained emotion recognition.
我们展示了一个基于hmd的注释工具，用于在用户在虚拟现实(VR)中观看360°视频时收集精确的情感基础真相标签。我们的工具使用HTC VIVE Pro Eye HMD来显示360°视频，Joy-Con控制器用于输入情感注释，Empatica E4腕带用于捕获生理信号。这些设备的时间戳通过NTP服务器同步。根据维度情绪模型，用户可以在观看VR视频时报告他们的情绪，包括效价和唤醒。注释反馈通过两种外围可视化技术提供:HaloLight和DotSize。我们的注释工具为研究人员在虚拟环境中设计瞬间和连续的自我报告提供了一个起点，以实现细粒度的情感识别。
{"title":"Annotation Tool for Precise Emotion Ground Truth Label Acquisition while Watching 360° VR Videos","authors":"Tong Xue, Abdallah El Ali, Ding Gangyi, Pablo Santiago César Garcia","doi":"10.1109/AIVR50618.2020.00076","DOIUrl":"https://doi.org/10.1109/AIVR50618.2020.00076","url":null,"abstract":"We demonstrate an HMD-based annotation tool for collecting precise emotion ground truth labels while users are watching 360° videos in Virtual Reality (VR). Our tool uses an HTC VIVE Pro Eye HMD for displaying 360° videos, a Joy-Con controller for inputting emotion annotations, and an Empatica E4 wristband for capturing physiological signals. Timestamps of these devices are synchronized via an NTP server. Following dimensional emotion models, users can report their emotion in terms of valence and arousal as they watch a video in VR. Annotation feedback is provided through two peripheral visualization techniques: HaloLight and DotSize. Our annotation tool provides a starting point for researchers to design momentary and continuous self-reports in virtual environments to enable fine-grained emotion recognition.","PeriodicalId":348199,"journal":{"name":"2020 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR)","volume":"57 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132935860","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2020-12-01DOI: 10.1109/AIVR50618.2020.00043
Michael G. Nelson, Christos Mousas
In this paper we developed a generic framework for authoring virtual crowds with minimal effort. Our intention is to providing to the virtual reality community a framework that allows easy to author virtual crowd scenarios that can be used for human-crowd interaction studies. From previous studies we have conducted, we realized the need of such a framework as it facilitates quicker setup and testing as well as standardizes the measurements and the interaction with virtual crowds. The framework includes assets with realistic human models, and configurations for crowd behavior composition.
{"title":"A Virtual Reality Framework for Human-Virtual Crowd Interaction Studies","authors":"Michael G. Nelson, Christos Mousas","doi":"10.1109/AIVR50618.2020.00043","DOIUrl":"https://doi.org/10.1109/AIVR50618.2020.00043","url":null,"abstract":"In this paper we developed a generic framework for authoring virtual crowds with minimal effort. Our intention is to providing to the virtual reality community a framework that allows easy to author virtual crowd scenarios that can be used for human-crowd interaction studies. From previous studies we have conducted, we realized the need of such a framework as it facilitates quicker setup and testing as well as standardizes the measurements and the interaction with virtual crowds. The framework includes assets with realistic human models, and configurations for crowd behavior composition.","PeriodicalId":348199,"journal":{"name":"2020 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR)","volume":"39 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133887822","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2020-12-01DOI: 10.1109/AIVR50618.2020.00080
Aaron Gluck, Jessica Chen, Ratnadeep Paul
Present military forces need to prepare for increasingly varied and complex situations and maintain the readiness of their warfighters for several different scenarios. There is a constant challenge of working with limited resources people, time, and funding. Military leaders are seeking new training technologies that can meet their goals better, faster, and cheaper. Virtual Reality (VR) and Augmented Reality (AR) have emerged as extremely useful technologies for developing new training tools since they allow for easy creation and maintenance of multiple scenarios and environments. Using AR/VR for training lowers the travel and living costs associated with bringing the trainees to a central training facility, while still completely immersing the trainees in the environment. Another advantage of using AR/VR compared to traditional tools is that by using AR/VR, trainees are completely immersed in the environment, which makes it much more natural for them to consume the training material. At GE Research, we are exploring technologies to incorporate Artificial Intelligence (AI) methodologies in an immersive VR based training environment for warfighters. We have developed an AI assisted VR system for ground soldier training. In this VR training environment, the soldier (Blue Team/Blue Force) is teamed up with an AI assistant that will help them navigate an urban setting and successfully reach their goal in a stealth mode while escaping the gaze of enemy soldiers (Red Team/Red Force). We have used AI enabled virtual humans for the enemy soldiers. In addition, we have created an AI enabled, VR drone simulation to assist the dismounted soldier to move undetected through the environment.
{"title":"Artificial Intelligence Assisted Virtual Reality Warfighter Training System","authors":"Aaron Gluck, Jessica Chen, Ratnadeep Paul","doi":"10.1109/AIVR50618.2020.00080","DOIUrl":"https://doi.org/10.1109/AIVR50618.2020.00080","url":null,"abstract":"Present military forces need to prepare for increasingly varied and complex situations and maintain the readiness of their warfighters for several different scenarios. There is a constant challenge of working with limited resources people, time, and funding. Military leaders are seeking new training technologies that can meet their goals better, faster, and cheaper. Virtual Reality (VR) and Augmented Reality (AR) have emerged as extremely useful technologies for developing new training tools since they allow for easy creation and maintenance of multiple scenarios and environments. Using AR/VR for training lowers the travel and living costs associated with bringing the trainees to a central training facility, while still completely immersing the trainees in the environment. Another advantage of using AR/VR compared to traditional tools is that by using AR/VR, trainees are completely immersed in the environment, which makes it much more natural for them to consume the training material. At GE Research, we are exploring technologies to incorporate Artificial Intelligence (AI) methodologies in an immersive VR based training environment for warfighters. We have developed an AI assisted VR system for ground soldier training. In this VR training environment, the soldier (Blue Team/Blue Force) is teamed up with an AI assistant that will help them navigate an urban setting and successfully reach their goal in a stealth mode while escaping the gaze of enemy soldiers (Red Team/Red Force). We have used AI enabled virtual humans for the enemy soldiers. In addition, we have created an AI enabled, VR drone simulation to assist the dismounted soldier to move undetected through the environment.","PeriodicalId":348199,"journal":{"name":"2020 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR)","volume":"6 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116524879","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2020-12-01DOI: 10.1109/AIVR50618.2020.00059
Conrad Fifelski-von Böhlen, Anna Brinkmann, Stephan Mävers, S. Hellmers, A. Hein
Telepresence and telemanipulation robotics are suitable solutions to relieve humans from direct health risks and repetitive or unhealthy work. Through demographic changes in western countries and the COVID-19 pandemic, this relief is also considered for healthcare workers, especially caregivers, as the demands for them rises. The requirements are intuitively usable telerobotic and telepresence systems for remote assistance, to cut the high physical strain in manual patient transfers and the reduction of contact with infected patients. To ensure this, key technologies like 3D imaging and perception systems are essential. In this work, we present a novel, lightweight telepresence and telemanipulation system, specialized for caregiving. It allows an operator, wearing a virtual reality headset, to immerse into a sensor system captured scene on a distant location in real-time, with low latency of 250 ms and up to 30 fps refresh rate. Extensive measurement shows that 97.1% of the relevant point cloud data is below 1 cm error and 99.5 % is below 1.6 cm, making the system suitable for the application.
远程呈现和远程操作机器人技术是减轻人类直接健康风险和重复性或不健康工作的合适解决方案。通过西方国家的人口变化和COVID-19大流行，这种缓解也被考虑到医护人员，特别是护理人员，因为对他们的需求增加了。这些要求是直观可用的远程机器人和远程呈现系统，用于远程援助，以减少人工转移患者时的高体力负荷，并减少与感染患者的接触。为了确保这一点，关键技术，如3D成像和感知系统是必不可少的。在这项工作中，我们提出了一个新的，轻量级的远程呈现和远程操作系统，专门用于护理。它允许操作员戴着虚拟现实耳机，沉浸在传感器系统中，实时捕捉到远处的场景，延迟低至250毫秒，刷新率高达30 fps。大量测量表明，97.1%的相关点云数据误差小于1 cm, 99.5%的点云数据误差小于1.6 cm，系统适合应用。
{"title":"Virtual Reality Integrated Multi-Depth-Camera-System for Real-Time Telepresence and Telemanipulation in Caregiving","authors":"Conrad Fifelski-von Böhlen, Anna Brinkmann, Stephan Mävers, S. Hellmers, A. Hein","doi":"10.1109/AIVR50618.2020.00059","DOIUrl":"https://doi.org/10.1109/AIVR50618.2020.00059","url":null,"abstract":"Telepresence and telemanipulation robotics are suitable solutions to relieve humans from direct health risks and repetitive or unhealthy work. Through demographic changes in western countries and the COVID-19 pandemic, this relief is also considered for healthcare workers, especially caregivers, as the demands for them rises. The requirements are intuitively usable telerobotic and telepresence systems for remote assistance, to cut the high physical strain in manual patient transfers and the reduction of contact with infected patients. To ensure this, key technologies like 3D imaging and perception systems are essential. In this work, we present a novel, lightweight telepresence and telemanipulation system, specialized for caregiving. It allows an operator, wearing a virtual reality headset, to immerse into a sensor system captured scene on a distant location in real-time, with low latency of 250 ms and up to 30 fps refresh rate. Extensive measurement shows that 97.1% of the relevant point cloud data is below 1 cm error and 99.5 % is below 1.6 cm, making the system suitable for the application.","PeriodicalId":348199,"journal":{"name":"2020 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR)","volume":"93 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129542915","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2020-12-01DOI: 10.1109/AIVR50618.2020.00075
S. Palmieri, Alessio Righi, M. Bisson, A. Ianniello
Recent and continuous innovations in the field of extended reality and, in particular, augmented reality, are able to revolutionize different aspects of the reference market sectors. At the same time, a constant evolution in the area of artificial intelligence, machine learning and deep learning, if combined with the aforementioned innovations, allows to conceive solutions able to shape new ways to inform, to improve skills and to spend time. The ability to simulate contexts, environments, actions and emotions and the possibility to use the data generated by the simulations in a disruptive way permit to imagine and create learning and strengthening paths.This developing research has been carried out within the Interdepartmental Laboratory EDME (Environmental Design Multisensory Experience), which belongs to the Design Department of Politecnico di Milano. It has been conducted by investigating the state of the art of augmented reality and artificial intelligence technologies, highlighting interesting and highly innovative case studies; from this first phase we moved on to analyze the sport sector in which an important potential for future development was recognized. The last part of the first phase of this research project consisted in the elaboration of a concept for an enabling technological system and a business model with a high innovation coefficient, whose realization is hypothesized for the year 2030.It is intended to demonstrate how a design operation, which started from emerging technologies and a sector of high interest and assumed a scenario of use over ten years, is not only extremely interesting but also, and above all, useful to consciously predict and accompany the aforementioned technological development.
{"title":"Training with a world champion: augmented reality applications in sport Design-led research","authors":"S. Palmieri, Alessio Righi, M. Bisson, A. Ianniello","doi":"10.1109/AIVR50618.2020.00075","DOIUrl":"https://doi.org/10.1109/AIVR50618.2020.00075","url":null,"abstract":"Recent and continuous innovations in the field of extended reality and, in particular, augmented reality, are able to revolutionize different aspects of the reference market sectors. At the same time, a constant evolution in the area of artificial intelligence, machine learning and deep learning, if combined with the aforementioned innovations, allows to conceive solutions able to shape new ways to inform, to improve skills and to spend time. The ability to simulate contexts, environments, actions and emotions and the possibility to use the data generated by the simulations in a disruptive way permit to imagine and create learning and strengthening paths.This developing research has been carried out within the Interdepartmental Laboratory EDME (Environmental Design Multisensory Experience), which belongs to the Design Department of Politecnico di Milano. It has been conducted by investigating the state of the art of augmented reality and artificial intelligence technologies, highlighting interesting and highly innovative case studies; from this first phase we moved on to analyze the sport sector in which an important potential for future development was recognized. The last part of the first phase of this research project consisted in the elaboration of a concept for an enabling technological system and a business model with a high innovation coefficient, whose realization is hypothesized for the year 2030.It is intended to demonstrate how a design operation, which started from emerging technologies and a sector of high interest and assumed a scenario of use over ten years, is not only extremely interesting but also, and above all, useful to consciously predict and accompany the aforementioned technological development.","PeriodicalId":348199,"journal":{"name":"2020 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR)","volume":"28 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125428707","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2020-12-01DOI: 10.1109/AIVR50618.2020.00042
G. Vizzari
Despite the significance of pedestrian simulation in the design of the built environment and in the definition of crowd management procedures, we still lack data about several relevant human decision making processes. To tackle this issue, we propose a novel Virtual Reality system which allows to design controlled experiments in virtual settings, by tracking multiple users that wear Head Mounted Displays (HMD). We employed the system to experimentally evaluate the signage of a building of the University of of Milano–Bicocca and its implications on wayfinding decisions. We present here the results of a preliminary test made with the system, aiming at evaluating its usability and feeling of sickness due to the VR itself, as well as preliminary results on human wayfinding decisions.
{"title":"Virtual Reality to Study Pedestrian Wayfinding: Motivations and an Experiment on Usability","authors":"G. Vizzari","doi":"10.1109/AIVR50618.2020.00042","DOIUrl":"https://doi.org/10.1109/AIVR50618.2020.00042","url":null,"abstract":"Despite the significance of pedestrian simulation in the design of the built environment and in the definition of crowd management procedures, we still lack data about several relevant human decision making processes. To tackle this issue, we propose a novel Virtual Reality system which allows to design controlled experiments in virtual settings, by tracking multiple users that wear Head Mounted Displays (HMD). We employed the system to experimentally evaluate the signage of a building of the University of of Milano–Bicocca and its implications on wayfinding decisions. We present here the results of a preliminary test made with the system, aiming at evaluating its usability and feeling of sickness due to the VR itself, as well as preliminary results on human wayfinding decisions.","PeriodicalId":348199,"journal":{"name":"2020 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR)","volume":"33 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122960085","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2020-12-01DOI: 10.1109/AIVR50618.2020.00011
Marcel Tiator, Anna Maria Kerkmann, C. Geiger, P. Grimm
The creation of interactive VR applications from 3D scanned content usually includes a lot of manual and repetitive work. Our research aim is to develop a real-world, cross-domain, automatic, semantic segmentation system that enhances the creation of interactive VR applications. We trained segmentation agents in a superpoint growing environment that we extended with an expert function. This expert function solves the sparse reward signal problem of the previous approaches and enables to use a variant of imitation learning and deep reinforcement learning with dense feedback. Additionally, the function allows to calculate a performance metric for the degree of imitation for different segmentations. We trained our agents with 1182 scenes of the ScanNet data set. More specifically, we trained different neural network architectures with 1170 scenes and tested their performance with 12 scenes. Our intermediate results are promising such that our segmentation system might be able to assist the VR application development from 3D scanned content in near future.
{"title":"Using Semantic Segmentation to Assist the Creation of Interactive VR Applications","authors":"Marcel Tiator, Anna Maria Kerkmann, C. Geiger, P. Grimm","doi":"10.1109/AIVR50618.2020.00011","DOIUrl":"https://doi.org/10.1109/AIVR50618.2020.00011","url":null,"abstract":"The creation of interactive VR applications from 3D scanned content usually includes a lot of manual and repetitive work. Our research aim is to develop a real-world, cross-domain, automatic, semantic segmentation system that enhances the creation of interactive VR applications. We trained segmentation agents in a superpoint growing environment that we extended with an expert function. This expert function solves the sparse reward signal problem of the previous approaches and enables to use a variant of imitation learning and deep reinforcement learning with dense feedback. Additionally, the function allows to calculate a performance metric for the degree of imitation for different segmentations. We trained our agents with 1182 scenes of the ScanNet data set. More specifically, we trained different neural network architectures with 1170 scenes and tested their performance with 12 scenes. Our intermediate results are promising such that our segmentation system might be able to assist the VR application development from 3D scanned content in near future.","PeriodicalId":348199,"journal":{"name":"2020 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR)","volume":"26 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123363288","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2020-12-01DOI: 10.1109/AIVR50618.2020.00052
Benjamin Allison, Xujiong Ye, Faraz Janan
Adoption of Virtual Reality (VR), Augmented Reality (AR) and Mixed Reality (MR) - known collectively as Extended Reality (XR) devices has been rapidly increasing over recent years. However, the focus of XR research has shown a lack of diversity in solutions to the problems within medicine, with it being predominantly focused in augmenting surgical procedures. Whilst important, XR applied to aiding medical diagnosis and surgical planning is relatively unexplored. In this paper we present a fully functional mammographic image analysis system, Breast3D, that can reconstruct MRI and CT scan data in XR. With breast cancer Breast Imaging-Reporting and Data System (BI-RADS) risk lexicon, early detection and clinical workflow such as Multi-disciplinary team (MDT) meetings for cancer in mind, our new mammography visualization system reconstructs CT and MRI volumes in a real 3D space. Breast3D is built upon the past literature and inspired from research for diagnosis and surgical planning. In addition to visualising the models in MR using the Microsoft HoloLens, Breast3D is versatile and portable to different XR head-mounted displays such as HTC Vive. Breast3D demonstrates the early potential for XR within diagnostics of 3D mammographic modalities, an application that has been proposed but until now has not been implemented.
{"title":"Breast3D: An Augmented Reality System for Breast CT and MRI","authors":"Benjamin Allison, Xujiong Ye, Faraz Janan","doi":"10.1109/AIVR50618.2020.00052","DOIUrl":"https://doi.org/10.1109/AIVR50618.2020.00052","url":null,"abstract":"Adoption of Virtual Reality (VR), Augmented Reality (AR) and Mixed Reality (MR) - known collectively as Extended Reality (XR) devices has been rapidly increasing over recent years. However, the focus of XR research has shown a lack of diversity in solutions to the problems within medicine, with it being predominantly focused in augmenting surgical procedures. Whilst important, XR applied to aiding medical diagnosis and surgical planning is relatively unexplored. In this paper we present a fully functional mammographic image analysis system, Breast3D, that can reconstruct MRI and CT scan data in XR. With breast cancer Breast Imaging-Reporting and Data System (BI-RADS) risk lexicon, early detection and clinical workflow such as Multi-disciplinary team (MDT) meetings for cancer in mind, our new mammography visualization system reconstructs CT and MRI volumes in a real 3D space. Breast3D is built upon the past literature and inspired from research for diagnosis and surgical planning. In addition to visualising the models in MR using the Microsoft HoloLens, Breast3D is versatile and portable to different XR head-mounted displays such as HTC Vive. Breast3D demonstrates the early potential for XR within diagnostics of 3D mammographic modalities, an application that has been proposed but until now has not been implemented.","PeriodicalId":348199,"journal":{"name":"2020 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR)","volume":"42 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"134083048","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：