In order to realize a character that can talk with voice, it is necessary to be able to control the transition of poses across multiple utterances and the expression of emotions according to the dialogue. In this paper, we propose a method for generating customer actors' performances that can represent various situations. We extract labels related to the conversation scenario and synthesize body motions, gestures, and head/eye motions of the customer actor according to different situations corresponding to different labels. We discuss the results of generating acting motions using a dialogue scene between a staff member and a customer in a customer service situation as a concrete example.
{"title":"Motion Generation Of Conversational Character From Labeled Script","authors":"Zhihao Song, Aohan Chen, Jeongsu Shin, Junichi Hoshino","doi":"10.1109/NicoInt55861.2022.00029","DOIUrl":"https://doi.org/10.1109/NicoInt55861.2022.00029","url":null,"abstract":"In order to realize a character that can talk with voice, it is necessary to be able to control the transition of poses across multiple utterances and the expression of emotions according to the dialogue. In this paper, we propose a method for generating customer actors' performances that can represent various situations. We extract labels related to the conversation scenario and synthesize body motions, gestures, and head/eye motions of the customer actor according to different situations corresponding to different labels. We discuss the results of generating acting motions using a dialogue scene between a staff member and a customer in a customer service situation as a concrete example.","PeriodicalId":328114,"journal":{"name":"2022 Nicograph International (NicoInt)","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2022-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128316073","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-06-01DOI: 10.1109/NicoInt55861.2022.00018
Xuemei Feng, Qing Fang, K. Konno, Katsutsugu Matsuyama
We propose a rounder deformation method combining as-rigid-as-possible and spherical feature. Focusing on the linear relationship between point normal vectors and position vectors, we construct an optimization model that can be solved numerically. We have applied our proposed method to basic shapes and confirmed that it successfully converges. We also discuss the limitations and problems of our method based on the experimental results.
{"title":"An Examination of Rounder Deformation Optimization Combining As-Rigid-As-Possible and Spherical Feature","authors":"Xuemei Feng, Qing Fang, K. Konno, Katsutsugu Matsuyama","doi":"10.1109/NicoInt55861.2022.00018","DOIUrl":"https://doi.org/10.1109/NicoInt55861.2022.00018","url":null,"abstract":"We propose a rounder deformation method combining as-rigid-as-possible and spherical feature. Focusing on the linear relationship between point normal vectors and position vectors, we construct an optimization model that can be solved numerically. We have applied our proposed method to basic shapes and confirmed that it successfully converges. We also discuss the limitations and problems of our method based on the experimental results.","PeriodicalId":328114,"journal":{"name":"2022 Nicograph International (NicoInt)","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2022-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114021945","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Spatial augmented reality techniques using projection mapping have been utilized extensively in our daily lives to augment the real world with virtual content, such as restaurant decorations that are intended to increase appetite. Although previous works have indicated that the projected visual effects on the food might change humans' taste impressions, it is still challenging to provide perceptual control of food taste, which allows apparent taste modifications along with specified projected colors. In this study, we focused on shaved ice for a simplified verification of controlling the food taste through the projection mapping technique. Based on our user study, we verified that the proposed perceptual control is suitable for the taste control of shaved ice with the projected visual effects of different colors.
{"title":"Perceptual Control of Food Taste with Projection Mapping","authors":"Masanobu Ohyori, Kotaro Okino, Ryota Takigami, Shogo Yoshida, Yichen Peng, Haoran Xie, Toshiki Sato, K. Miyata","doi":"10.1109/NicoInt55861.2022.00011","DOIUrl":"https://doi.org/10.1109/NicoInt55861.2022.00011","url":null,"abstract":"Spatial augmented reality techniques using projection mapping have been utilized extensively in our daily lives to augment the real world with virtual content, such as restaurant decorations that are intended to increase appetite. Although previous works have indicated that the projected visual effects on the food might change humans' taste impressions, it is still challenging to provide perceptual control of food taste, which allows apparent taste modifications along with specified projected colors. In this study, we focused on shaved ice for a simplified verification of controlling the food taste through the projection mapping technique. Based on our user study, we verified that the proposed perceptual control is suitable for the taste control of shaved ice with the projected visual effects of different colors.","PeriodicalId":328114,"journal":{"name":"2022 Nicograph International (NicoInt)","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2022-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132154425","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-06-01DOI: 10.1109/NicoInt55861.2022.00031
Jason Chan, A. Ito, Koji Mikami
In digital games, music that dynamically changes in real time in response to interaction is becoming a widely known technique in the game industry and academic side. In this study, we use Cohn's Voice-Leading Zones to analyze chord progressions and dynamic music pieces from the music with special modulation methods and create an interactive demo using Wwise.
{"title":"Interactive Demonstration of Dynamic Music Composition Based on Cohn's “Voice-Leading Zones” Theory","authors":"Jason Chan, A. Ito, Koji Mikami","doi":"10.1109/NicoInt55861.2022.00031","DOIUrl":"https://doi.org/10.1109/NicoInt55861.2022.00031","url":null,"abstract":"In digital games, music that dynamically changes in real time in response to interaction is becoming a widely known technique in the game industry and academic side. In this study, we use Cohn's Voice-Leading Zones to analyze chord progressions and dynamic music pieces from the music with special modulation methods and create an interactive demo using Wwise.","PeriodicalId":328114,"journal":{"name":"2022 Nicograph International (NicoInt)","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2022-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114545212","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-06-01DOI: 10.1109/NicoInt55861.2022.00009
Tomoya Bansho, Mitsuyo Hashida, H. Katayose
In the field of storytelling and filmmaking, we are often aware of the viewer's subjective state, such as emotional state and so on. A similar concept is direction, which is often accumulated by the heuristic findings of creators. The issue is that the emotional state of the viewer is subjective, making it hard for computational approach. Our approach is utilizing appraisal trajectory. The appraisal trajectory is a line chart drawn the viewer's own emotional state from the beginning to the end of the content. We think that the appraisal trajectory enables us to handle the viewer's emotional state with a computer. We think to need accumulate basic findings about subjective trajectories; therefore, we develop a tool to collect subjective trajectories: emonotate. This tool provides some features: input and store the various emotional states of viewers and viewers intuitively agreeable input by visual feedback of arbitrary appraisal trajectories. In order to evaluate the validity, we conduct the experiment of 104 participants drawing subjective trajectories based on protagonist emotional state about 2 animation contents and the subjective quantity is the protagonist emotional state. Both works yielded intuitively correct results, demonstrating the realism of our tool. We think our experimental facts indicate the possibility of our tool collecting subjective trajectories and the issues.
{"title":"Emonotate: Development a Tool for Collecting Appraisal Trajectories for Time-Series Media Contents","authors":"Tomoya Bansho, Mitsuyo Hashida, H. Katayose","doi":"10.1109/NicoInt55861.2022.00009","DOIUrl":"https://doi.org/10.1109/NicoInt55861.2022.00009","url":null,"abstract":"In the field of storytelling and filmmaking, we are often aware of the viewer's subjective state, such as emotional state and so on. A similar concept is direction, which is often accumulated by the heuristic findings of creators. The issue is that the emotional state of the viewer is subjective, making it hard for computational approach. Our approach is utilizing appraisal trajectory. The appraisal trajectory is a line chart drawn the viewer's own emotional state from the beginning to the end of the content. We think that the appraisal trajectory enables us to handle the viewer's emotional state with a computer. We think to need accumulate basic findings about subjective trajectories; therefore, we develop a tool to collect subjective trajectories: emonotate. This tool provides some features: input and store the various emotional states of viewers and viewers intuitively agreeable input by visual feedback of arbitrary appraisal trajectories. In order to evaluate the validity, we conduct the experiment of 104 participants drawing subjective trajectories based on protagonist emotional state about 2 animation contents and the subjective quantity is the protagonist emotional state. Both works yielded intuitively correct results, demonstrating the realism of our tool. We think our experimental facts indicate the possibility of our tool collecting subjective trajectories and the issues.","PeriodicalId":328114,"journal":{"name":"2022 Nicograph International (NicoInt)","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2022-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116822405","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-06-01DOI: 10.1109/NicoInt55861.2022.00024
Mizuki Yamano, Zhihao Song, Junichi Hoshino
Recently, in addition to games, interaction with real virtual agents has been used for job training. Furthermore, multimodal dialog using both voice and gesture recognition is used. However, there is a challenge with conventional batch processing speech recognition: it is difficult to make the virtual agent respond while the user is speaking. Therefore, in this study, we confirm the possibility of enhancing social presence by generating responses using real-time speech recognition in spoken dialog with characters. While the user speaks, by extracting keywords from the speech content in real-time, we generate the emotional response of the virtual agent. The experimental results demonstrate that our approach improves social presence compared with conventional spoken dialog.
{"title":"Augmenting the Social Presence of Interactive Characters Using Real-time Speech Recognition","authors":"Mizuki Yamano, Zhihao Song, Junichi Hoshino","doi":"10.1109/NicoInt55861.2022.00024","DOIUrl":"https://doi.org/10.1109/NicoInt55861.2022.00024","url":null,"abstract":"Recently, in addition to games, interaction with real virtual agents has been used for job training. Furthermore, multimodal dialog using both voice and gesture recognition is used. However, there is a challenge with conventional batch processing speech recognition: it is difficult to make the virtual agent respond while the user is speaking. Therefore, in this study, we confirm the possibility of enhancing social presence by generating responses using real-time speech recognition in spoken dialog with characters. While the user speaks, by extracting keywords from the speech content in real-time, we generate the emotional response of the virtual agent. The experimental results demonstrate that our approach improves social presence compared with conventional spoken dialog.","PeriodicalId":328114,"journal":{"name":"2022 Nicograph International (NicoInt)","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2022-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130236718","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-06-01DOI: 10.1109/NicoInt55861.2022.00010
Taiyo Taguchi, Tomokazu Ishikawa
In recent years, Exergaming which allows people to exercise easily indoors has been gaining attention. A lot of Exergaming is often played using VR devices or dedicated devices, however, since Exergaming is played indoors, there is a risk of colliding with objects in the room or people around the player. We hypothesized that Exergaming using MR devices would solve the problems of VR devices. Therefore, we conducted a comparison of Exergaming using MR and VR devices. Subjects were asked to play Exergaming with the same level using an MR device (HoloLens2) and a VR device (Oculus Quest 2), and then compared in three categories: “Breathability”, “Op-erability”, and “Safety”. Additionally, the difference in gaming sickness and heart rate between the two devices were investigated. An MR device was superior in terms of “Breathability” and “Safety”, and a VR device was superior in terms of “Operability”. For the difference in gaming sickness, VR Exergaming resulted in no problem in all items, however, MR gave serious symptoms in “Oculomotor disturbance”, “Disorientation” and “Total”. The difference in heart rate was not significant. The results of this experiment show that although MR devices are superior in terms of “Safety” and “Breathability”, there is a tendency that MR devices can easily make users feel bad, and MR devices are not always more comfortable to play than VR devices.
{"title":"A Research On The Compatibility Between MR/VR Devices And Exergaming","authors":"Taiyo Taguchi, Tomokazu Ishikawa","doi":"10.1109/NicoInt55861.2022.00010","DOIUrl":"https://doi.org/10.1109/NicoInt55861.2022.00010","url":null,"abstract":"In recent years, Exergaming which allows people to exercise easily indoors has been gaining attention. A lot of Exergaming is often played using VR devices or dedicated devices, however, since Exergaming is played indoors, there is a risk of colliding with objects in the room or people around the player. We hypothesized that Exergaming using MR devices would solve the problems of VR devices. Therefore, we conducted a comparison of Exergaming using MR and VR devices. Subjects were asked to play Exergaming with the same level using an MR device (HoloLens2) and a VR device (Oculus Quest 2), and then compared in three categories: “Breathability”, “Op-erability”, and “Safety”. Additionally, the difference in gaming sickness and heart rate between the two devices were investigated. An MR device was superior in terms of “Breathability” and “Safety”, and a VR device was superior in terms of “Operability”. For the difference in gaming sickness, VR Exergaming resulted in no problem in all items, however, MR gave serious symptoms in “Oculomotor disturbance”, “Disorientation” and “Total”. The difference in heart rate was not significant. The results of this experiment show that although MR devices are superior in terms of “Safety” and “Breathability”, there is a tendency that MR devices can easily make users feel bad, and MR devices are not always more comfortable to play than VR devices.","PeriodicalId":328114,"journal":{"name":"2022 Nicograph International (NicoInt)","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2022-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127953733","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-06-01DOI: 10.1109/NicoInt55861.2022.00034
Tshering Tenji Sherpa, Akinori Kimura
This research aims to develop a complete portable aid for visually impaired persons. It focuses on detecting the pedestrian traffic lights and obstacles in the way of a visually impaired person who crosses a pedestrian crossing. The system recognizes the pedestrian traffic lights in real-time with deep learning and provides signals to the visually impaired person to cross the pedestrian crossing in the form of audio output. It should work in both the day and night time and have good reliability. For the good reliability of the system, different sorts of images and video data have been collected, edited, and stored in a large amount of training data to get a good model.
{"title":"Pedestrian Crossing Lights and Obstacles Detections for Visually Impaired Person","authors":"Tshering Tenji Sherpa, Akinori Kimura","doi":"10.1109/NicoInt55861.2022.00034","DOIUrl":"https://doi.org/10.1109/NicoInt55861.2022.00034","url":null,"abstract":"This research aims to develop a complete portable aid for visually impaired persons. It focuses on detecting the pedestrian traffic lights and obstacles in the way of a visually impaired person who crosses a pedestrian crossing. The system recognizes the pedestrian traffic lights in real-time with deep learning and provides signals to the visually impaired person to cross the pedestrian crossing in the form of audio output. It should work in both the day and night time and have good reliability. For the good reliability of the system, different sorts of images and video data have been collected, edited, and stored in a large amount of training data to get a good model.","PeriodicalId":328114,"journal":{"name":"2022 Nicograph International (NicoInt)","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2022-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124911040","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-06-01DOI: 10.1109/NicoInt55861.2022.00012
Harumi Kawamura, Yuki Nemoto, Saeko Maruyama, Akiko Kozai, N. Tosa, R. Nakatsu
Recently digital art using digital technology has emerged and has been well recognized. On the other hand, digital textile printing technology has recently emerged in the fashion world, making it relatively easy to produce small-lot, high-mix garments. Combining this digital art with digital textile printing creates new possibilities for art to enter our everyday life as clothing. In this paper, we will describe the contents of our attempt to create fashion from digital art under the concept of “wearing art” through joint research between companies and universities.
{"title":"Development of Art Fashion by Integrating Digital Art and Digital Textile Printing","authors":"Harumi Kawamura, Yuki Nemoto, Saeko Maruyama, Akiko Kozai, N. Tosa, R. Nakatsu","doi":"10.1109/NicoInt55861.2022.00012","DOIUrl":"https://doi.org/10.1109/NicoInt55861.2022.00012","url":null,"abstract":"Recently digital art using digital technology has emerged and has been well recognized. On the other hand, digital textile printing technology has recently emerged in the fashion world, making it relatively easy to produce small-lot, high-mix garments. Combining this digital art with digital textile printing creates new possibilities for art to enter our everyday life as clothing. In this paper, we will describe the contents of our attempt to create fashion from digital art under the concept of “wearing art” through joint research between companies and universities.","PeriodicalId":328114,"journal":{"name":"2022 Nicograph International (NicoInt)","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2022-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128240948","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-06-01DOI: 10.1109/NicoInt55861.2022.00033
Mayu Morioka, H. Takemura, Mitsunori Matsushita
The purpose of this research is to enable users to experience co-creative communication with a distant partner using multiple drawing systems. To share information in real-time is worth generating new ideas and stimulating communication, however, COVID-19 has made face-to-face communication difficult. Thus, it is necessary to realize an environment where people can communicate even at a distance. From these backgrounds, we developed a co-creative drawing system inspired by “performance calligraphy.” The performance calligraphy is a collaborative drawing with someone attending on body action. With this system, multiple users can draw together even when users are in different locations. This paper presents our developed system and evaluates the usability of this system.
{"title":"Designing A Co-creative Communication Experience Utilizing Collaborative Drawing System","authors":"Mayu Morioka, H. Takemura, Mitsunori Matsushita","doi":"10.1109/NicoInt55861.2022.00033","DOIUrl":"https://doi.org/10.1109/NicoInt55861.2022.00033","url":null,"abstract":"The purpose of this research is to enable users to experience co-creative communication with a distant partner using multiple drawing systems. To share information in real-time is worth generating new ideas and stimulating communication, however, COVID-19 has made face-to-face communication difficult. Thus, it is necessary to realize an environment where people can communicate even at a distance. From these backgrounds, we developed a co-creative drawing system inspired by “performance calligraphy.” The performance calligraphy is a collaborative drawing with someone attending on body action. With this system, multiple users can draw together even when users are in different locations. This paper presents our developed system and evaluates the usability of this system.","PeriodicalId":328114,"journal":{"name":"2022 Nicograph International (NicoInt)","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2022-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130145027","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：