Marios Belk, C. Fidas, J. Bowles, E. Athanasopoulos, A. Pitsillides
The Second International Workshop on Adaptive and Personalized Privacy and Security (APPS 2020) aims to bring together researchers and practitioners working on diverse topics related to understanding and improving the usability of privacy and security software and systems, by applying user modeling, adaptation and personalization principles. Our special focus in 2020 is on healthcare systems, more specifically on ensuring security and privacy of medical data in smart patient-centric healthcare systems. The second edition of the workshop includes interdisciplinary contributions from Austria, Canada, China, Cyprus, Denmark, Germany, Greece, Israel, the Netherlands, Turkey and the UK that introduce new and disruptive ideas, suggest novel solutions, and present research results about various aspects (theory, applications, tools) for bringing user modeling, adaptation and personalization principles into privacy and systems security. This summary gives a brief overview of APPS 2020, held online in conjunction with the 28th ACM Conference on User Modeling, Adaptation and Personalization (ACM UMAP 2020).
{"title":"APPS 2020: Second International Workshop on Adaptive and Personalized Privacy and Security","authors":"Marios Belk, C. Fidas, J. Bowles, E. Athanasopoulos, A. Pitsillides","doi":"10.1145/3340631.3398674","DOIUrl":"https://doi.org/10.1145/3340631.3398674","url":null,"abstract":"The Second International Workshop on Adaptive and Personalized Privacy and Security (APPS 2020) aims to bring together researchers and practitioners working on diverse topics related to understanding and improving the usability of privacy and security software and systems, by applying user modeling, adaptation and personalization principles. Our special focus in 2020 is on healthcare systems, more specifically on ensuring security and privacy of medical data in smart patient-centric healthcare systems. The second edition of the workshop includes interdisciplinary contributions from Austria, Canada, China, Cyprus, Denmark, Germany, Greece, Israel, the Netherlands, Turkey and the UK that introduce new and disruptive ideas, suggest novel solutions, and present research results about various aspects (theory, applications, tools) for bringing user modeling, adaptation and personalization principles into privacy and systems security. This summary gives a brief overview of APPS 2020, held online in conjunction with the 28th ACM Conference on User Modeling, Adaptation and Personalization (ACM UMAP 2020).","PeriodicalId":417607,"journal":{"name":"Proceedings of the 28th ACM Conference on User Modeling, Adaptation and Personalization","volume":"59 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-07-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124377382","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
E-commerce and online services are getting more and more ubiquitous day by day. Like many other e-commerce paradigms, online grocery services can highly benefit from recommender systems, especially when it comes to predicting users' shopping behavior. This specific scenario owns peculiar characteristics, such as repetitiveness and loyalty, which makes the task very different from the standard recommendations. In this work, we present an efficient solution to compute the next basket recommendation, under a more general top-n recommendation framework. We propose a set of collaborative filtering based techniques able to capture users' shopping patterns. Furthermore, we analyzed how recency plays a key role in this particular task. We finally compare our method with state-of-the-art algorithms on two online grocery service datasets.
{"title":"Recency Aware Collaborative Filtering for Next Basket Recommendation","authors":"G. Faggioli, Mirko Polato, F. Aiolli","doi":"10.1145/3340631.3394850","DOIUrl":"https://doi.org/10.1145/3340631.3394850","url":null,"abstract":"E-commerce and online services are getting more and more ubiquitous day by day. Like many other e-commerce paradigms, online grocery services can highly benefit from recommender systems, especially when it comes to predicting users' shopping behavior. This specific scenario owns peculiar characteristics, such as repetitiveness and loyalty, which makes the task very different from the standard recommendations. In this work, we present an efficient solution to compute the next basket recommendation, under a more general top-n recommendation framework. We propose a set of collaborative filtering based techniques able to capture users' shopping patterns. Furthermore, we analyzed how recency plays a key role in this particular task. We finally compare our method with state-of-the-art algorithms on two online grocery service datasets.","PeriodicalId":417607,"journal":{"name":"Proceedings of the 28th ACM Conference on User Modeling, Adaptation and Personalization","volume":"2016 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-07-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128066960","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Peer grading, in which students grade each other's work, can provide an educational opportunity for students and reduce grading effort for instructors. A variety of methods have been proposed for synthesizing peer-assigned grades into accurate submission grades. However, when the assumptions behind these methods are not met, they may underperform a simple baseline of averaging the peer grades. We introduce SABTXT, which improves over previous work through two mechanisms. First, SABTXT uses a limited amount of historical instructor ground truth to model and correct for each peer's grading bias. Secondly, SABTXT models the thoroughness of a peer review based on its textual content, and puts more weight on the more thorough peer reviews when computing submission grades. In our experiments with over ten thousand peer reviews collected over four courses, we show that SABTXT outperforms existing approaches on our collected data, and achieves a mean squared error that is 6% lower than the strongest baseline on average.
{"title":"Practical Methods for Semi-automated Peer Grading in a Classroom Setting","authors":"Zheng Yuan, Doug Downey","doi":"10.1145/3340631.3394878","DOIUrl":"https://doi.org/10.1145/3340631.3394878","url":null,"abstract":"Peer grading, in which students grade each other's work, can provide an educational opportunity for students and reduce grading effort for instructors. A variety of methods have been proposed for synthesizing peer-assigned grades into accurate submission grades. However, when the assumptions behind these methods are not met, they may underperform a simple baseline of averaging the peer grades. We introduce SABTXT, which improves over previous work through two mechanisms. First, SABTXT uses a limited amount of historical instructor ground truth to model and correct for each peer's grading bias. Secondly, SABTXT models the thoroughness of a peer review based on its textual content, and puts more weight on the more thorough peer reviews when computing submission grades. In our experiments with over ten thousand peer reviews collected over four courses, we show that SABTXT outperforms existing approaches on our collected data, and achieves a mean squared error that is 6% lower than the strongest baseline on average.","PeriodicalId":417607,"journal":{"name":"Proceedings of the 28th ACM Conference on User Modeling, Adaptation and Personalization","volume":"20 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-07-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132883550","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Player modelling is an important task for almost any game creator, which helps in understanding the player-base. One of the major issues is an early leave of players which makes modelling them challenging. In our research, we focus on the cold-start problem by utilizing information about a player from multiple games or other players in a given game. Although multiple studies focus on cross-game modelling, they still often require manual mapping of features or don't consider a player's behaviour specific to the given game. Our proposed method is based on transfer learning and unsupervised translation. In addition, we propose a combination of group-based and individual player models.
{"title":"Cross-Game Modeling of Player's Behaviour in Free-To-Play Games","authors":"Andrej Vítek","doi":"10.1145/3340631.3398677","DOIUrl":"https://doi.org/10.1145/3340631.3398677","url":null,"abstract":"Player modelling is an important task for almost any game creator, which helps in understanding the player-base. One of the major issues is an early leave of players which makes modelling them challenging. In our research, we focus on the cold-start problem by utilizing information about a player from multiple games or other players in a given game. Although multiple studies focus on cross-game modelling, they still often require manual mapping of features or don't consider a player's behaviour specific to the given game. Our proposed method is based on transfer learning and unsupervised translation. In addition, we propose a combination of group-based and individual player models.","PeriodicalId":417607,"journal":{"name":"Proceedings of the 28th ACM Conference on User Modeling, Adaptation and Personalization","volume":"45 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-07-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123996597","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Popular approaches in learner modeling explore response-time as observational data supplemental to response correctness, to enrich the predictive models of learner knowledge. It has been argued that the relationship between response-time and knowledge mastery is non-linear. Determining the degree of association (dependence structure) between those two observations is an open question. To address this objective, we propose an approach based on copulas, i.e., a statistical tool suitable for capturing dependence structure between two variables. All of the information about the dependence structures can be estimated by copula models separately, allowing for the construction of more flexible joint distributions than existing multivariate distributions. This paper puts into practice a two-step pipeline for building the analytical models. Specifically, we propose a flexible copula-based approach that describes the dependence structure between students' response-time and mastery, in learning and testing contexts, and apply the methodology on four datasets. The two datasets are coming from Intelligent Tutoring Systems and are shared via an online repository, and the other two were collected during the validation of an (adaptive) assessment system. The results reveal five generic patterns of associations across-datasets, for various types of activities, domains and learner characteristics (i.e., not across-contexts). We elaborate on those findings and on the implications of our approach for adaptive systems.
{"title":"On the Dependence Structure Between Learners' Response-time and Knowledge Mastery: If Not Linear, Then What?","authors":"Z. Papamitsiou, K. Sharma, M. Giannakos","doi":"10.1145/3340631.3394865","DOIUrl":"https://doi.org/10.1145/3340631.3394865","url":null,"abstract":"Popular approaches in learner modeling explore response-time as observational data supplemental to response correctness, to enrich the predictive models of learner knowledge. It has been argued that the relationship between response-time and knowledge mastery is non-linear. Determining the degree of association (dependence structure) between those two observations is an open question. To address this objective, we propose an approach based on copulas, i.e., a statistical tool suitable for capturing dependence structure between two variables. All of the information about the dependence structures can be estimated by copula models separately, allowing for the construction of more flexible joint distributions than existing multivariate distributions. This paper puts into practice a two-step pipeline for building the analytical models. Specifically, we propose a flexible copula-based approach that describes the dependence structure between students' response-time and mastery, in learning and testing contexts, and apply the methodology on four datasets. The two datasets are coming from Intelligent Tutoring Systems and are shared via an online repository, and the other two were collected during the validation of an (adaptive) assessment system. The results reveal five generic patterns of associations across-datasets, for various types of activities, domains and learner characteristics (i.e., not across-contexts). We elaborate on those findings and on the implications of our approach for adaptive systems.","PeriodicalId":417607,"journal":{"name":"Proceedings of the 28th ACM Conference on User Modeling, Adaptation and Personalization","volume":"18 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-07-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125620139","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Language provides a unique window into thoughts, enabling direct assessment of mental-state alterations. Due to their increasing popularity, online social media platforms have become promising means to study different mental disorders. However, the lack of available datasets can hinder the development of innovative diagnostic methods. Tools to assist health practitioners in screening and monitoring individuals under potential risk are essential. In this paper, we present a new a dataset to foster the research on automatic detection of depression. To this end, we present a methodology for automatically collecting large samples of depression and non-depression posts from online social media. Furthermore, we perform a benchmark on the dataset to establish a point of reference for researchers who are interested in using it.
{"title":"A Dataset for Research on Depression in Social Media","authors":"E. A. Ríssola, Seyed Ali Bahrainian, F. Crestani","doi":"10.1145/3340631.3394879","DOIUrl":"https://doi.org/10.1145/3340631.3394879","url":null,"abstract":"Language provides a unique window into thoughts, enabling direct assessment of mental-state alterations. Due to their increasing popularity, online social media platforms have become promising means to study different mental disorders. However, the lack of available datasets can hinder the development of innovative diagnostic methods. Tools to assist health practitioners in screening and monitoring individuals under potential risk are essential. In this paper, we present a new a dataset to foster the research on automatic detection of depression. To this end, we present a methodology for automatically collecting large samples of depression and non-depression posts from online social media. Furthermore, we perform a benchmark on the dataset to establish a point of reference for researchers who are interested in using it.","PeriodicalId":417607,"journal":{"name":"Proceedings of the 28th ACM Conference on User Modeling, Adaptation and Personalization","volume":"60 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-07-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132445712","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Yuan Tian, K. Zhou, M. Lalmas, Yiqun Liu, D. Pelleg
Smartphones utilize context signals, such as time and location, to predict users' app usage tailored to individual users. To be effective, such personalization relies on access to sufficient information about each user's behavioral habits. For new users, the behavior information may be sparse or non-existent. To handle these cases, app category usage prediction approaches can employ signals from users who are similar along one or more dimensions, i.e., those in the same cohort. In this paper, we describe a characterization and evaluation of the use of such cohort modeling to enhance app category usage prediction. We experiment with pre-defined cohorts from three taxonomies - demographics, psychographics, and behavioral patterns - independently and in combination. We also evaluate various approaches to assign users into the corresponding cohorts. We show, through extensive experiments with large-scale mobile app usage logs from a mobile advertising company, that leveraging cohort behavior can yield significant prediction performance gains than when using the personalized signals at the individual prediction level. In addition, compared to the personalized model, the cohort-based approach can significantly alleviate the cold-start problem, achieving strong predictive performance even with limited amount of user interactions.
{"title":"Cohort Modeling Based App Category Usage Prediction","authors":"Yuan Tian, K. Zhou, M. Lalmas, Yiqun Liu, D. Pelleg","doi":"10.1145/3340631.3394849","DOIUrl":"https://doi.org/10.1145/3340631.3394849","url":null,"abstract":"Smartphones utilize context signals, such as time and location, to predict users' app usage tailored to individual users. To be effective, such personalization relies on access to sufficient information about each user's behavioral habits. For new users, the behavior information may be sparse or non-existent. To handle these cases, app category usage prediction approaches can employ signals from users who are similar along one or more dimensions, i.e., those in the same cohort. In this paper, we describe a characterization and evaluation of the use of such cohort modeling to enhance app category usage prediction. We experiment with pre-defined cohorts from three taxonomies - demographics, psychographics, and behavioral patterns - independently and in combination. We also evaluate various approaches to assign users into the corresponding cohorts. We show, through extensive experiments with large-scale mobile app usage logs from a mobile advertising company, that leveraging cohort behavior can yield significant prediction performance gains than when using the personalized signals at the individual prediction level. In addition, compared to the personalized model, the cohort-based approach can significantly alleviate the cold-start problem, achieving strong predictive performance even with limited amount of user interactions.","PeriodicalId":417607,"journal":{"name":"Proceedings of the 28th ACM Conference on User Modeling, Adaptation and Personalization","volume":"21 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-07-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131938536","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Automated complex word identification (CWI) is a crucial task in several applications, from readability assessment to lexical simplification. So far, several works have modeled CWI with the goal of targeting the needs of non-native speakers. However, studies in language acquisition show that different native languages can create positive or negative interferences w.r.t. reading comprehension, favouring or hindering the understanding of a document in a foreign language. Therefore, we propose to modify CWI to address the specific difficulties connected to different native languages. In particular, we present a pipeline that, based on the user native language, identifies complex terms by automatically detecting cognates and false friends on the fly. The selection presented by the CWI module is adaptive in that it changes depending on the native language of the user. We implement and evaluate our approach for four different native languages (French, English, German and Spanish), in a setting where documents are written in Italian and should be read by language learners with low proficiency. We show that a personalised strategy based on false friend detection identifies complex terms that are different from those usually selected with standard approaches based on word frequency.
{"title":"Adaptive Complex Word Identification through False Friend Detection","authors":"Alessio Palmero Aprosio, S. Menini, Sara Tonelli","doi":"10.1145/3340631.3394857","DOIUrl":"https://doi.org/10.1145/3340631.3394857","url":null,"abstract":"Automated complex word identification (CWI) is a crucial task in several applications, from readability assessment to lexical simplification. So far, several works have modeled CWI with the goal of targeting the needs of non-native speakers. However, studies in language acquisition show that different native languages can create positive or negative interferences w.r.t. reading comprehension, favouring or hindering the understanding of a document in a foreign language. Therefore, we propose to modify CWI to address the specific difficulties connected to different native languages. In particular, we present a pipeline that, based on the user native language, identifies complex terms by automatically detecting cognates and false friends on the fly. The selection presented by the CWI module is adaptive in that it changes depending on the native language of the user. We implement and evaluate our approach for four different native languages (French, English, German and Spanish), in a setting where documents are written in Italian and should be read by language learners with low proficiency. We show that a personalised strategy based on false friend detection identifies complex terms that are different from those usually selected with standard approaches based on word frequency.","PeriodicalId":417607,"journal":{"name":"Proceedings of the 28th ACM Conference on User Modeling, Adaptation and Personalization","volume":"8 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-07-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126683170","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Patrícia Alves, Pedro M. Saraiva, João Carneiro, Pedro F. Campos, Helena Martins, P. Novais, G. Marreiros
Personalization is increasingly being perceived as an important factor for the effectiveness of Recommender Systems (RS). This is especially true in the tourism domain, where travelling comprises emotionally charged experiences, and therefore, the more about the tourist is known, better recommendations can be made. The inclusion of psychological aspects to generate recommendations, such as personality, is a growing trend in RS and they are being studied to provide more personalized approaches. However, although many studies on the psychology of tourism exist, studies on the prediction of tourist preferences based on their personality are limited. Therefore, we undertook a large-scale study in order to determine how the Big Five personality dimensions influence tourists' preferences for tourist attractions, gathering data from an online questionnaire, sent to Portuguese individuals from the academic sector and their respective relatives/friends (n=508). Using Exploratory and Confirmatory Factor Analysis, we extracted 11 main categories of tourist attractions and analyzed which personality dimensions were predictors (or not) of preferences for those tourist attractions. As a result, we propose the first model that relates the five personality dimensions with preferences for tourist attractions, which intends to offer a base for researchers of RS for tourism to automatically model tourist preferences based on their personality.
{"title":"Modeling Tourists' Personality in Recommender Systems: How Does Personality Influence Preferences for Tourist Attractions?","authors":"Patrícia Alves, Pedro M. Saraiva, João Carneiro, Pedro F. Campos, Helena Martins, P. Novais, G. Marreiros","doi":"10.1145/3340631.3394843","DOIUrl":"https://doi.org/10.1145/3340631.3394843","url":null,"abstract":"Personalization is increasingly being perceived as an important factor for the effectiveness of Recommender Systems (RS). This is especially true in the tourism domain, where travelling comprises emotionally charged experiences, and therefore, the more about the tourist is known, better recommendations can be made. The inclusion of psychological aspects to generate recommendations, such as personality, is a growing trend in RS and they are being studied to provide more personalized approaches. However, although many studies on the psychology of tourism exist, studies on the prediction of tourist preferences based on their personality are limited. Therefore, we undertook a large-scale study in order to determine how the Big Five personality dimensions influence tourists' preferences for tourist attractions, gathering data from an online questionnaire, sent to Portuguese individuals from the academic sector and their respective relatives/friends (n=508). Using Exploratory and Confirmatory Factor Analysis, we extracted 11 main categories of tourist attractions and analyzed which personality dimensions were predictors (or not) of preferences for those tourist attractions. As a result, we propose the first model that relates the five personality dimensions with preferences for tourist attractions, which intends to offer a base for researchers of RS for tourism to automatically model tourist preferences based on their personality.","PeriodicalId":417607,"journal":{"name":"Proceedings of the 28th ACM Conference on User Modeling, Adaptation and Personalization","volume":"53 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-07-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116445764","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
This paper investigates the user experience of visualizations of a machine learning (ML) system that recognizes objects in images. This is important since even good systems can fail in unexpected ways as misclassifications on photo-sharing websites showed. In our study, we exposed users with a background in ML to three visualizations of three systems with different levels of accuracy. In interviews, we explored how the visualization helped users assess the accuracy of systems in use and how the visualization and the accuracy of the system affected trust and reliance. We found that participants do not only focus on accuracy when assessing ML systems. They also take the perceived plausibility and severity of misclassification into account and prefer seeing the probability of predictions. Semantically plausible errors are judged as less severe than errors that are implausible, which means that system accuracy could be communicated through the types of errors.
{"title":"More Than Accuracy: Towards Trustworthy Machine Learning Interfaces for Object Recognition","authors":"Hendrik Heuer, A. Breiter","doi":"10.1145/3340631.3394873","DOIUrl":"https://doi.org/10.1145/3340631.3394873","url":null,"abstract":"This paper investigates the user experience of visualizations of a machine learning (ML) system that recognizes objects in images. This is important since even good systems can fail in unexpected ways as misclassifications on photo-sharing websites showed. In our study, we exposed users with a background in ML to three visualizations of three systems with different levels of accuracy. In interviews, we explored how the visualization helped users assess the accuracy of systems in use and how the visualization and the accuracy of the system affected trust and reliance. We found that participants do not only focus on accuracy when assessing ML systems. They also take the perceived plausibility and severity of misclassification into account and prefer seeing the probability of predictions. Semantically plausible errors are judged as less severe than errors that are implausible, which means that system accuracy could be communicated through the types of errors.","PeriodicalId":417607,"journal":{"name":"Proceedings of the 28th ACM Conference on User Modeling, Adaptation and Personalization","volume":"86 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-07-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130432243","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：