Pierre Raimbaud, Alberto Jovane, Katja Zibrek, C. Pacchierotti, M. Christie, Ludovic Hoyet, J. Pettré, A. Olivier
Non-verbal communication body cues are paramount to interact. In this preliminary work, we explore ways to let Intelligent Virtual Agents (IVAs) simulating nonverbal communication capabilities. We propose an approach to control IVAs' reactive behaviour from the analysis of other agents' apparent motions, in a situation of "observed" IVAs that act and "observers" that react. For that, first a viewpoint-driven analysis of the observed agent's motion is done, and then a synthesis of this analysis induces the observers' reaction.
{"title":"Reactive Virtual Agents: A Viewpoint-Driven Approach for Bodily Nonverbal Communication","authors":"Pierre Raimbaud, Alberto Jovane, Katja Zibrek, C. Pacchierotti, M. Christie, Ludovic Hoyet, J. Pettré, A. Olivier","doi":"10.1145/3472306.3478351","DOIUrl":"https://doi.org/10.1145/3472306.3478351","url":null,"abstract":"Non-verbal communication body cues are paramount to interact. In this preliminary work, we explore ways to let Intelligent Virtual Agents (IVAs) simulating nonverbal communication capabilities. We propose an approach to control IVAs' reactive behaviour from the analysis of other agents' apparent motions, in a situation of \"observed\" IVAs that act and \"observers\" that react. For that, first a viewpoint-driven analysis of the observed agent's motion is done, and then a synthesis of this analysis induces the observers' reaction.","PeriodicalId":148152,"journal":{"name":"Proceedings of the 21st ACM International Conference on Intelligent Virtual Agents","volume":"4 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-09-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116813147","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Maha Elgarf, Gabriel Skantze, Christopher E. Peters
Creativity is a vital inherent human trait. In an attempt to stimulate children's creativity, we present the design and evaluation of an interaction between a child and a social robot in a storytelling context. Using a software interface, children were asked to collaboratively create a story with the robot. We conducted a study with 38 children in two conditions. In one condition, the children interacted with a robot exhibiting creative behavior while in the other condition, they interacted with a robot exhibiting non creative behavior. The robot's creativity was defined as verbal and performance creativity. The robot's creative and non creative behaviors were extracted from a previously collected data set and were validated in an online survey with 100 participants. Contrary to our initial hypothesis, children's creativity measures were not higher in the creative condition than in the non creative condition. Our results suggest that merely the robot's creative behavior is insufficient to stimulate creativity in children in a child robot interaction. We further discuss other design factors that may facilitate sparking creativity in children in similar settings in the future.
{"title":"Once Upon a Story: Can a Creative Storyteller Robot Stimulate Creativity in Children?","authors":"Maha Elgarf, Gabriel Skantze, Christopher E. Peters","doi":"10.1145/3472306.3478359","DOIUrl":"https://doi.org/10.1145/3472306.3478359","url":null,"abstract":"Creativity is a vital inherent human trait. In an attempt to stimulate children's creativity, we present the design and evaluation of an interaction between a child and a social robot in a storytelling context. Using a software interface, children were asked to collaboratively create a story with the robot. We conducted a study with 38 children in two conditions. In one condition, the children interacted with a robot exhibiting creative behavior while in the other condition, they interacted with a robot exhibiting non creative behavior. The robot's creativity was defined as verbal and performance creativity. The robot's creative and non creative behaviors were extracted from a previously collected data set and were validated in an online survey with 100 participants. Contrary to our initial hypothesis, children's creativity measures were not higher in the creative condition than in the non creative condition. Our results suggest that merely the robot's creative behavior is insufficient to stimulate creativity in children in a child robot interaction. We further discuss other design factors that may facilitate sparking creativity in children in similar settings in the future.","PeriodicalId":148152,"journal":{"name":"Proceedings of the 21st ACM International Conference on Intelligent Virtual Agents","volume":"349 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-09-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132763273","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Ana Antunes, Joana Campos, João Dias, P. A. Santos, R. Prada
Social sharing of emotions (SSE) occurs when one communicates their feelings and reactions to a certain event in the course of a social interaction. The phenomenon is part of our social fabric and plays an important role in creating empathetic responses and establishing rapport. Intelligent social agents capable of SSE will have a mechanism to create and build long-term interaction with humans. In this paper, we present the Emotional Episode Generation (EEG) model, a fine-tuned GPT-2 model capable of generating emotional social talk regarding multiple event tuples in a human-like manner. Human evaluation results show that the model successfully translates one or more event-tuples into emotional episodes, reaching quality levels close to human performance. Furthermore, the model clearly expresses one emotion in each episode as well as humans. To train this model we used a public dataset and built upon it using event extraction techniques1.
{"title":"EEG Model: Emotional Episode Generation for Social Sharing of Emotions","authors":"Ana Antunes, Joana Campos, João Dias, P. A. Santos, R. Prada","doi":"10.1145/3472306.3478342","DOIUrl":"https://doi.org/10.1145/3472306.3478342","url":null,"abstract":"Social sharing of emotions (SSE) occurs when one communicates their feelings and reactions to a certain event in the course of a social interaction. The phenomenon is part of our social fabric and plays an important role in creating empathetic responses and establishing rapport. Intelligent social agents capable of SSE will have a mechanism to create and build long-term interaction with humans. In this paper, we present the Emotional Episode Generation (EEG) model, a fine-tuned GPT-2 model capable of generating emotional social talk regarding multiple event tuples in a human-like manner. Human evaluation results show that the model successfully translates one or more event-tuples into emotional episodes, reaching quality levels close to human performance. Furthermore, the model clearly expresses one emotion in each episode as well as humans. To train this model we used a public dataset and built upon it using event extraction techniques1.","PeriodicalId":148152,"journal":{"name":"Proceedings of the 21st ACM International Conference on Intelligent Virtual Agents","volume":"8 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-09-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116975125","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Chatbots use conversational interfaces to simulate human communication and recently have been applied to different domains due to advancing techniques of natural language understanding and generation. In particular, in the domain of digital journalism, chatbots provide a new channel for audiences to engage with news. However, most chatbots operated by news organizations have so far failed to achieve business growth. In this paper, we have conducted a qualitative user study to better understand users' attitudes towards news chatbots. Specifically, 15 participants were asked to interact with several news chatbots implemented on Facebook Messenger by large international news organizations (e.g., ABC News, NBC News, and BBC News), by issuing a set of 22 sample questions covering various search and recommendation goals related to COVID-19. Then the participants expressed their expectations of news chatbots and the advantages/disadvantages perceived in using them. From these findings, we derive several design guidelines on effectiveness, informativeness, efficiency, humanization, and facility, suggesting developments to news chatbots that may better serve users' needs.
{"title":"Informing the Design of a News Chatbot","authors":"Zhirun Zhang, Xinzhi Zhang, Li Chen","doi":"10.1145/3472306.3478358","DOIUrl":"https://doi.org/10.1145/3472306.3478358","url":null,"abstract":"Chatbots use conversational interfaces to simulate human communication and recently have been applied to different domains due to advancing techniques of natural language understanding and generation. In particular, in the domain of digital journalism, chatbots provide a new channel for audiences to engage with news. However, most chatbots operated by news organizations have so far failed to achieve business growth. In this paper, we have conducted a qualitative user study to better understand users' attitudes towards news chatbots. Specifically, 15 participants were asked to interact with several news chatbots implemented on Facebook Messenger by large international news organizations (e.g., ABC News, NBC News, and BBC News), by issuing a set of 22 sample questions covering various search and recommendation goals related to COVID-19. Then the participants expressed their expectations of news chatbots and the advantages/disadvantages perceived in using them. From these findings, we derive several design guidelines on effectiveness, informativeness, efficiency, humanization, and facility, suggesting developments to news chatbots that may better serve users' needs.","PeriodicalId":148152,"journal":{"name":"Proceedings of the 21st ACM International Conference on Intelligent Virtual Agents","volume":"10 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-09-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121027843","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Ryo Ishii, Xutong Ren, Michal Muszynski, Louis-Philippe Morency
The listener's backchannel has the important function of encouraging a current speaker to hold their turn and continue to speak, which enables smooth conversation. The listener monitors the speaker's turn-management (a.k.a. speaking and listening) willingness and his/her own willingness to display backchannel behavior. Many studies have focused on predicting the appropriate timing of the backchannel so that conversational agents can display backchannel behavior in response to a user who is speaking. To the best of our knowledge, none of them added the prediction of turn-changing and participants' turn-management willingness to the backchannel prediction model in dyad interactions. In this paper, we proposed a novel backchannel prediction model that can jointly predict turn-changing and turn-management willingness. We investigated the impact of modeling turn-changing and willingness to improve backchannel prediction. Our proposed model is based on trimodal inputs, that is, acoustic, linguistic, and visual cues from conversations. Our results suggest that adding turn-management willingness as a prediction task improves the performance of backchannel prediction within the multi-modal multi-task learning approach, while adding turn-changing prediction is not useful for improving the performance of backchannel prediction.
{"title":"Multimodal and Multitask Approach to Listener's Backchannel Prediction: Can Prediction of Turn-changing and Turn-management Willingness Improve Backchannel Modeling?","authors":"Ryo Ishii, Xutong Ren, Michal Muszynski, Louis-Philippe Morency","doi":"10.1145/3472306.3478360","DOIUrl":"https://doi.org/10.1145/3472306.3478360","url":null,"abstract":"The listener's backchannel has the important function of encouraging a current speaker to hold their turn and continue to speak, which enables smooth conversation. The listener monitors the speaker's turn-management (a.k.a. speaking and listening) willingness and his/her own willingness to display backchannel behavior. Many studies have focused on predicting the appropriate timing of the backchannel so that conversational agents can display backchannel behavior in response to a user who is speaking. To the best of our knowledge, none of them added the prediction of turn-changing and participants' turn-management willingness to the backchannel prediction model in dyad interactions. In this paper, we proposed a novel backchannel prediction model that can jointly predict turn-changing and turn-management willingness. We investigated the impact of modeling turn-changing and willingness to improve backchannel prediction. Our proposed model is based on trimodal inputs, that is, acoustic, linguistic, and visual cues from conversations. Our results suggest that adding turn-management willingness as a prediction task improves the performance of backchannel prediction within the multi-modal multi-task learning approach, while adding turn-changing prediction is not useful for improving the performance of backchannel prediction.","PeriodicalId":148152,"journal":{"name":"Proceedings of the 21st ACM International Conference on Intelligent Virtual Agents","volume":"55 7","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-09-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"120941521","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Understanding an opponent's wants is crucial for maximizing the outcomes of a multi-issue negotiation. To do this, automated systems must build an "opponent model" from information conveyed during a negotiation. Bayesian and frequentist models are the most commonly used. Bayesian models have a principled way to incorporate prior knowledge about an opponent's preferences. However, frequentist models have outperformed Bayesian approaches in practice, dominating the yearly agent-verses-agent negotiation competitions. With growing interest in agents that negotiate with people, this presumed dominance needs to be revisited. Human opponents convey far less information than automated agents, and people often share similar preferences (e.g., in a salary negotiation, most people care the most about salary). Thus, the theoretical advantage of Bayesian approaches may translate into practice for agent-versus-human negotiation. In this work, we compare the performance of Bayesian models against a leading frequentist approach in an agent-versus-human multi-issue salary negotiation. Although we show that frequentist opponent models outperform Bayesian models when using a uniform prior, Bayesian approaches excel when using two common priors. The best performance is achieved with an empirically-derived prior (i.e., biasing the model space using the distribution of preferences found in past human negotiators). Yet, strong performance is also observed when using a "fixed-pie bias", the prior used by most human negotiators. We discuss the implication of these findings for research on human-agent negotiation.
{"title":"Comparing The Accuracy of Frequentist and Bayesian Models in Human-Agent Negotiation","authors":"Emmanuel Johnson, J. Gratch","doi":"10.1145/3472306.3478354","DOIUrl":"https://doi.org/10.1145/3472306.3478354","url":null,"abstract":"Understanding an opponent's wants is crucial for maximizing the outcomes of a multi-issue negotiation. To do this, automated systems must build an \"opponent model\" from information conveyed during a negotiation. Bayesian and frequentist models are the most commonly used. Bayesian models have a principled way to incorporate prior knowledge about an opponent's preferences. However, frequentist models have outperformed Bayesian approaches in practice, dominating the yearly agent-verses-agent negotiation competitions. With growing interest in agents that negotiate with people, this presumed dominance needs to be revisited. Human opponents convey far less information than automated agents, and people often share similar preferences (e.g., in a salary negotiation, most people care the most about salary). Thus, the theoretical advantage of Bayesian approaches may translate into practice for agent-versus-human negotiation. In this work, we compare the performance of Bayesian models against a leading frequentist approach in an agent-versus-human multi-issue salary negotiation. Although we show that frequentist opponent models outperform Bayesian models when using a uniform prior, Bayesian approaches excel when using two common priors. The best performance is achieved with an empirically-derived prior (i.e., biasing the model space using the distribution of preferences found in past human negotiators). Yet, strong performance is also observed when using a \"fixed-pie bias\", the prior used by most human negotiators. We discuss the implication of these findings for research on human-agent negotiation.","PeriodicalId":148152,"journal":{"name":"Proceedings of the 21st ACM International Conference on Intelligent Virtual Agents","volume":"54 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-09-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114754659","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers. L’archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d’enseignement et de recherche français ou étrangers, des laboratoires publics ou privés. Generation of Multimodal Behaviors in the Greta platform Michele Grimaldi, Catherine Pelachaud
它是一个多学科的开放获取档案，用于科学研究文件的存储和传播，无论它们是否出版。这些文件可能来自法国或国外的教学和研究机构，也可能来自公共或私人研究中心。HAL开放多学科档案旨在存放和传播来自法国或外国教育和研究机构、公共或私人实验室的已发表或未发表的研究级科学文件。in the Generation of多式Behaviors Greta platform (Michele Grimaldi凯瑟琳Pelachaud
{"title":"Generation of Multimodal Behaviors in the Greta platform","authors":"Michele Grimaldi, C. Pelachaud","doi":"10.1145/3472306.3478368","DOIUrl":"https://doi.org/10.1145/3472306.3478368","url":null,"abstract":"HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers. L’archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d’enseignement et de recherche français ou étrangers, des laboratoires publics ou privés. Generation of Multimodal Behaviors in the Greta platform Michele Grimaldi, Catherine Pelachaud","PeriodicalId":148152,"journal":{"name":"Proceedings of the 21st ACM International Conference on Intelligent Virtual Agents","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-09-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129027452","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
A major impediment to the success of virtual agents is the inability of non-technical experts to easily author content. To address this barrier we present VHMason, a multimodal authoring tool designed to help creative authors build embodied conversational agents. We introduce the novel aspects of this authoring tool and explore a use case of the creation of an agent-led educational experience implemented at Children's Hospital Los Angeles (CHLA).
{"title":"Introducing VHMason: A Visual, Integrated, Multimodal Virtual Human Authoring Tool","authors":"Arno Hartholt, Edward Fast, A. Leeds, S. Mozgai","doi":"10.1145/3472306.3478363","DOIUrl":"https://doi.org/10.1145/3472306.3478363","url":null,"abstract":"A major impediment to the success of virtual agents is the inability of non-technical experts to easily author content. To address this barrier we present VHMason, a multimodal authoring tool designed to help creative authors build embodied conversational agents. We introduce the novel aspects of this authoring tool and explore a use case of the creation of an agent-led educational experience implemented at Children's Hospital Los Angeles (CHLA).","PeriodicalId":148152,"journal":{"name":"Proceedings of the 21st ACM International Conference on Intelligent Virtual Agents","volume":"19 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-09-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122934521","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Prior research often finds increased altruism following natural disasters. One explanation is the social heuristic hypothesis: humans are prosocial by nature but become self-interested when they have the opportunity to deliberate. As the stress of a disaster lowers people's ability to engage in effortful deliberation, their heuristic prosocial tendencies emerge. However, this link has often been explored with very simple tasks like the dictator game. Here we study the impact of COVID-related stress on outcomes in multi-issue negotiations with a computational virtual agent. These tasks are interesting because they share some of the characteristics of dictator games (some pot of resources must be divided) but they also involve presumably effortful perspective taking (that can grow the size of the pot). Furthermore, the interaction of humans with virtual agents allows us to explore the extent to which humans apply the CASA (computers as social actors) paradigm to negotiation when under considerable stress. In two experiments with a virtual negotiation partner, we provide evidence for two distinct pathways for how COVID-19 stress shapes prosocial behavior. Consistent with the social heuristic hypothesis, COVID-stress increases giving, mediated by heuristic thinking. But COVID-stress also seems to enhance information-exchange and perspective taking, which allowed participants to grow more value which they could give away. Our results give new insights into the relationship between stress, cognition, and prosocial behavior.
{"title":"Pandemic Panic: The Effect of Disaster-Related Stress on Negotiation Outcomes","authors":"Johnathan Mell, Gale M. Lucas, J. Gratch","doi":"10.1145/3472306.3478353","DOIUrl":"https://doi.org/10.1145/3472306.3478353","url":null,"abstract":"Prior research often finds increased altruism following natural disasters. One explanation is the social heuristic hypothesis: humans are prosocial by nature but become self-interested when they have the opportunity to deliberate. As the stress of a disaster lowers people's ability to engage in effortful deliberation, their heuristic prosocial tendencies emerge. However, this link has often been explored with very simple tasks like the dictator game. Here we study the impact of COVID-related stress on outcomes in multi-issue negotiations with a computational virtual agent. These tasks are interesting because they share some of the characteristics of dictator games (some pot of resources must be divided) but they also involve presumably effortful perspective taking (that can grow the size of the pot). Furthermore, the interaction of humans with virtual agents allows us to explore the extent to which humans apply the CASA (computers as social actors) paradigm to negotiation when under considerable stress. In two experiments with a virtual negotiation partner, we provide evidence for two distinct pathways for how COVID-19 stress shapes prosocial behavior. Consistent with the social heuristic hypothesis, COVID-stress increases giving, mediated by heuristic thinking. But COVID-stress also seems to enhance information-exchange and perspective taking, which allowed participants to grow more value which they could give away. Our results give new insights into the relationship between stress, cognition, and prosocial behavior.","PeriodicalId":148152,"journal":{"name":"Proceedings of the 21st ACM International Conference on Intelligent Virtual Agents","volume":"2 4","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-09-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114104055","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
S. Fitrianie, Merijn Bruijnes, Fengxiang Li, Willem-Paul Brinkman
In this paper, we report on the multi-year Intelligent Virtual Agents (IVA) community effort, involving more than 90 researchers worldwide, researching the IVA community interests and practice in evaluating human interaction with an artificial social agent (ASA). The joint efforts have previously generated a unified set of 19 constructs that capture more than 80% of constructs used in empirical studies published in the IVA conference between 2013 to 2018. In this paper, we present expert-content-validated 131 questionnaire items for the constructs and their dimensions, and investigate the level of reliability. We establish this in three phases. Firstly, eight experts generated 431 potential construct items. Secondly, 20 experts rated whether items measure (only) their intended construct, resulting in 207 content-validated items. Next, a reliability analysis was conducted, involving 192 crowd-workers who were asked to rate a human interaction with an ASA, which resulted in 131 items (about 5 items per measurement, with Cronbach's alpha ranged [.60 -- .87]). These are the starting points for the questionnaire instrument of human-ASA interaction.
{"title":"Questionnaire Items for Evaluating Artificial Social Agents - Expert Generated, Content Validated and Reliability Analysed","authors":"S. Fitrianie, Merijn Bruijnes, Fengxiang Li, Willem-Paul Brinkman","doi":"10.1145/3472306.3478341","DOIUrl":"https://doi.org/10.1145/3472306.3478341","url":null,"abstract":"In this paper, we report on the multi-year Intelligent Virtual Agents (IVA) community effort, involving more than 90 researchers worldwide, researching the IVA community interests and practice in evaluating human interaction with an artificial social agent (ASA). The joint efforts have previously generated a unified set of 19 constructs that capture more than 80% of constructs used in empirical studies published in the IVA conference between 2013 to 2018. In this paper, we present expert-content-validated 131 questionnaire items for the constructs and their dimensions, and investigate the level of reliability. We establish this in three phases. Firstly, eight experts generated 431 potential construct items. Secondly, 20 experts rated whether items measure (only) their intended construct, resulting in 207 content-validated items. Next, a reliability analysis was conducted, involving 192 crowd-workers who were asked to rate a human interaction with an ASA, which resulted in 131 items (about 5 items per measurement, with Cronbach's alpha ranged [.60 -- .87]). These are the starting points for the questionnaire instrument of human-ASA interaction.","PeriodicalId":148152,"journal":{"name":"Proceedings of the 21st ACM International Conference on Intelligent Virtual Agents","volume":"20 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-09-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123650689","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：