Pub Date : 2021-10-01DOI: 10.1109/trex53765.2021.00002
{"title":"Title Page iii","authors":"","doi":"10.1109/trex53765.2021.00002","DOIUrl":"https://doi.org/10.1109/trex53765.2021.00002","url":null,"abstract":"","PeriodicalId":345585,"journal":{"name":"2021 IEEE Workshop on TRust and EXpertise in Visual Analytics (TREX)","volume":"635 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132666996","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-10-01DOI: 10.1109/TREX53765.2021.00009
Dirk Leffrang, Oliver Müller
Time series forecasts are ubiquitous, ranging from daily weather forecasts to projections of pandemics such as COVID-19. Communicating the uncertainty associated with such forecasts is important, because it may affect users’ trust in a forecasting model and, in turn, the decisions made based on the model. Although there exists a growing body of research on visualizing uncertainty in general, the important case of visualizing prediction uncertainty in time series forecasting is under-researched. Against this background, we investigated how different visualizations of predictive uncertainty affect the extent to which people follow predictions of a time series forecasting model. More specifically, we conducted an online experiment on forecasting occupied hospital beds due to the COVID-19 pandemic, measuring the influence of uncertainty visualization of algorithmic predictions on participants’ own predictions. In contrast to prior studies, our empirical results suggest that more salient visualizations of uncertainty lead to decreased willingness to follow algorithmic forecasts.
{"title":"Should I Follow this Model? The Effect of Uncertainty Visualization on the Acceptance of Time Series Forecasts","authors":"Dirk Leffrang, Oliver Müller","doi":"10.1109/TREX53765.2021.00009","DOIUrl":"https://doi.org/10.1109/TREX53765.2021.00009","url":null,"abstract":"Time series forecasts are ubiquitous, ranging from daily weather forecasts to projections of pandemics such as COVID-19. Communicating the uncertainty associated with such forecasts is important, because it may affect users’ trust in a forecasting model and, in turn, the decisions made based on the model. Although there exists a growing body of research on visualizing uncertainty in general, the important case of visualizing prediction uncertainty in time series forecasting is under-researched. Against this background, we investigated how different visualizations of predictive uncertainty affect the extent to which people follow predictions of a time series forecasting model. More specifically, we conducted an online experiment on forecasting occupied hospital beds due to the COVID-19 pandemic, measuring the influence of uncertainty visualization of algorithmic predictions on participants’ own predictions. In contrast to prior studies, our empirical results suggest that more salient visualizations of uncertainty lead to decreased willingness to follow algorithmic forecasts.","PeriodicalId":345585,"journal":{"name":"2021 IEEE Workshop on TRust and EXpertise in Visual Analytics (TREX)","volume":"21 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114530921","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-10-01DOI: 10.1109/TREX53765.2021.00012
John E. Wenskovitch, C. Fallon, Kate Miller, Aritra Dasgupta
Detect the expected, discover the unexpected was the founding principle of the field of visual analytics. This mantra implies that human stakeholders, like a domain expert or data analyst, could leverage visual analytics techniques to seek answers to known unknowns and discover unknown unknowns in the course of the data sense-making process. We argue that in the era of AI-driven automation, we need to recalibrate the roles of humans and machines (e.g., a machine learning model) as teammates. We posit that by realizing human-machine teams as a stakeholder unit, we can better achieve the best of both worlds: automation transparency and human reasoning efficacy. However, this also increases the burden on analysts and domain experts towards performing more cognitively demanding tasks than what they are used to. In this paper, we reflect on the complementary roles in a human-machine team through the lens of cognitive psychology and map them to existing and emerging research in the visual analytics community. We discuss open questions and challenges around the nature of human agency and analyze the shared responsibilities in human-machine teams.
{"title":"Beyond Visual Analytics: Human-Machine Teaming for AI-Driven Data Sensemaking","authors":"John E. Wenskovitch, C. Fallon, Kate Miller, Aritra Dasgupta","doi":"10.1109/TREX53765.2021.00012","DOIUrl":"https://doi.org/10.1109/TREX53765.2021.00012","url":null,"abstract":"Detect the expected, discover the unexpected was the founding principle of the field of visual analytics. This mantra implies that human stakeholders, like a domain expert or data analyst, could leverage visual analytics techniques to seek answers to known unknowns and discover unknown unknowns in the course of the data sense-making process. We argue that in the era of AI-driven automation, we need to recalibrate the roles of humans and machines (e.g., a machine learning model) as teammates. We posit that by realizing human-machine teams as a stakeholder unit, we can better achieve the best of both worlds: automation transparency and human reasoning efficacy. However, this also increases the burden on analysts and domain experts towards performing more cognitively demanding tasks than what they are used to. In this paper, we reflect on the complementary roles in a human-machine team through the lens of cognitive psychology and map them to existing and emerging research in the visual analytics community. We discuss open questions and challenges around the nature of human agency and analyze the shared responsibilities in human-machine teams.","PeriodicalId":345585,"journal":{"name":"2021 IEEE Workshop on TRust and EXpertise in Visual Analytics (TREX)","volume":"3 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"134369004","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-10-01DOI: 10.1109/trex53765.2021.00005
{"title":"Welcome from the TREX 2021 Organizers","authors":"","doi":"10.1109/trex53765.2021.00005","DOIUrl":"https://doi.org/10.1109/trex53765.2021.00005","url":null,"abstract":"","PeriodicalId":345585,"journal":{"name":"2021 IEEE Workshop on TRust and EXpertise in Visual Analytics (TREX)","volume":"16 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130434321","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-10-01DOI: 10.1109/TREX53765.2021.00014
C. Gillmann, D. Saur, G. Scheuermann
Recently, machine learning is massively on the rise in medical applications providing the ability to predict diseases, plan treatment and monitor progress. Still, the use in a clinical context of this technology is rather rare, mostly due to the missing trust of clinicians. In this position paper, we aim to show how uncertainty is introduced in the machine learning process when applying it to medical imaging at multiple points and how this influences the decision-making process of clinicians in machine learning approaches. Based on this knowledge, we aim to refine the guidelines for trust in visual analytics to assist clinicians in using and understanding systems that are based on machine learning.
{"title":"How to deal with Uncertainty in Machine Learning for Medical Imaging?","authors":"C. Gillmann, D. Saur, G. Scheuermann","doi":"10.1109/TREX53765.2021.00014","DOIUrl":"https://doi.org/10.1109/TREX53765.2021.00014","url":null,"abstract":"Recently, machine learning is massively on the rise in medical applications providing the ability to predict diseases, plan treatment and monitor progress. Still, the use in a clinical context of this technology is rather rare, mostly due to the missing trust of clinicians. In this position paper, we aim to show how uncertainty is introduced in the machine learning process when applying it to medical imaging at multiple points and how this influences the decision-making process of clinicians in machine learning approaches. Based on this knowledge, we aim to refine the guidelines for trust in visual analytics to assist clinicians in using and understanding systems that are based on machine learning.","PeriodicalId":345585,"journal":{"name":"2021 IEEE Workshop on TRust and EXpertise in Visual Analytics (TREX)","volume":"2 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129606468","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-10-01DOI: 10.1109/TREX53765.2021.00006
A. Lohfink, V. M. Memmesheimer, Frederike Gartzky, C. Garth
We present evaluation results of our enhancements to the Security in Process System [14] developed by Lohfink et al. to support triage analysis in operational technology networks. To ensure fast and appropriate reactions to anomalies in device readings, this system communicates anomaly detection results and device readings to incorporate human expertise and experience. It exploits periodical behavior in the data combining spiral plots with results from anomaly detection. To support decisions, increase trust, and support cooperation in the system we enhanced it to be knowledge-assisted. A central knowledge base allows sharing knowledge between users and support during analysis. It consists of an ontology describing incidents, and a data base holding collections of exemplary sensor readings with annotations and visualization parameters. Related knowledge is proposed automatically and incorporated directly in the visualization to provide assistance that is closely coupled to the application, without additional hurdles. This integration is designed aiming on additional support for correct and fast detection of anomalies in the visualized device readings. We evaluate our enhancements to the Security in Process System in terms of effectiveness, efficiency, user satisfaction, and cognitive load with a detailed user study. Comparing the original and enhanced system, we are able to draw conclusions as to how our design narrows the knowledge gap between professional analysts and laymen. Furthermore, we present and discuss the results and impact on our future research.
{"title":"The Enhanced Security in Process System - Evaluating Knowledge Assistance","authors":"A. Lohfink, V. M. Memmesheimer, Frederike Gartzky, C. Garth","doi":"10.1109/TREX53765.2021.00006","DOIUrl":"https://doi.org/10.1109/TREX53765.2021.00006","url":null,"abstract":"We present evaluation results of our enhancements to the Security in Process System [14] developed by Lohfink et al. to support triage analysis in operational technology networks. To ensure fast and appropriate reactions to anomalies in device readings, this system communicates anomaly detection results and device readings to incorporate human expertise and experience. It exploits periodical behavior in the data combining spiral plots with results from anomaly detection. To support decisions, increase trust, and support cooperation in the system we enhanced it to be knowledge-assisted. A central knowledge base allows sharing knowledge between users and support during analysis. It consists of an ontology describing incidents, and a data base holding collections of exemplary sensor readings with annotations and visualization parameters. Related knowledge is proposed automatically and incorporated directly in the visualization to provide assistance that is closely coupled to the application, without additional hurdles. This integration is designed aiming on additional support for correct and fast detection of anomalies in the visualized device readings. We evaluate our enhancements to the Security in Process System in terms of effectiveness, efficiency, user satisfaction, and cognitive load with a detailed user study. Comparing the original and enhanced system, we are able to draw conclusions as to how our design narrows the knowledge gap between professional analysts and laymen. Furthermore, we present and discuss the results and impact on our future research.","PeriodicalId":345585,"journal":{"name":"2021 IEEE Workshop on TRust and EXpertise in Visual Analytics (TREX)","volume":"75 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132748206","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-10-01DOI: 10.1109/TREX53765.2021.00013
N. Boukhelifa, E. Lutton, A. Bezerianos
In this paper, we demonstrate how analytic provenance can be exploited to re-construct user trust in a guided Visual Analytics (VA) system, and suggest that interaction log data analysis can be a valuable tool for on-line trust monitoring. Our approach explores objective trust measures that can be continuously tracked and updated during the exploration, and reflect both the confidence of the user in system suggestions, and the uncertainty of the system with regards to user goals. We argue that this approach is more suitable for guided VA systems such as ours, where user strategies, goals and even trust can evolve over time, in reaction to new system feedback and insights from the exploration. Through the analysis of log data from a past user study with twelve participants performing a guided visual analysis task, we found that the stability of user exploration strategies is a promising factor to study trust. However, indirect metrics based on provenance, such as user evaluation counts and disagreement rates, are alone not sufficient to study trust reliably in guided VA. We conclude with open challenges and opportunities for exploiting analytic provenance to support trust monitoring in guided VA systems.
{"title":"A Case Study of Using Analytic Provenance to Reconstruct User Trust in a Guided Visual Analytics System","authors":"N. Boukhelifa, E. Lutton, A. Bezerianos","doi":"10.1109/TREX53765.2021.00013","DOIUrl":"https://doi.org/10.1109/TREX53765.2021.00013","url":null,"abstract":"In this paper, we demonstrate how analytic provenance can be exploited to re-construct user trust in a guided Visual Analytics (VA) system, and suggest that interaction log data analysis can be a valuable tool for on-line trust monitoring. Our approach explores objective trust measures that can be continuously tracked and updated during the exploration, and reflect both the confidence of the user in system suggestions, and the uncertainty of the system with regards to user goals. We argue that this approach is more suitable for guided VA systems such as ours, where user strategies, goals and even trust can evolve over time, in reaction to new system feedback and insights from the exploration. Through the analysis of log data from a past user study with twelve participants performing a guided visual analysis task, we found that the stability of user exploration strategies is a promising factor to study trust. However, indirect metrics based on provenance, such as user evaluation counts and disagreement rates, are alone not sufficient to study trust reliably in guided VA. We conclude with open challenges and opportunities for exploiting analytic provenance to support trust monitoring in guided VA systems.","PeriodicalId":345585,"journal":{"name":"2021 IEEE Workshop on TRust and EXpertise in Visual Analytics (TREX)","volume":"57 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125191477","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-10-01DOI: 10.1109/TREX53765.2021.00011
Yafeng Lu, Michael Steptoe, Verica Buchanan, Nancy J. Cooke, Ross Maciejewski
In this paper, we explore the intersection of knowledge and the forecasting accuracy of humans when supported by visual analytics. We have recruited 40 experts in machine learning and trained them in the use of a box office forecasting visual analytics system. Our goal was to explore the impact of visual analytics and knowledge in human-machine forecasting. This paper reports on how participants explore and reason with data and develop a forecast when provided with a predictive model of middling performance (R2 ≈ .7). We vary the knowledge base of the participants through training, compare the forecasts to the baseline model, and discuss performance in the context of previous work on algorithmic aversion and trust.
{"title":"Evaluating Forecasting, Knowledge, and Visual Analytics","authors":"Yafeng Lu, Michael Steptoe, Verica Buchanan, Nancy J. Cooke, Ross Maciejewski","doi":"10.1109/TREX53765.2021.00011","DOIUrl":"https://doi.org/10.1109/TREX53765.2021.00011","url":null,"abstract":"In this paper, we explore the intersection of knowledge and the forecasting accuracy of humans when supported by visual analytics. We have recruited 40 experts in machine learning and trained them in the use of a box office forecasting visual analytics system. Our goal was to explore the impact of visual analytics and knowledge in human-machine forecasting. This paper reports on how participants explore and reason with data and develop a forecast when provided with a predictive model of middling performance (R2 ≈ .7). We vary the knowledge base of the participants through training, compare the forecasts to the baseline model, and discuss performance in the context of previous work on algorithmic aversion and trust.","PeriodicalId":345585,"journal":{"name":"2021 IEEE Workshop on TRust and EXpertise in Visual Analytics (TREX)","volume":"2016 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127381155","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-10-01DOI: 10.1109/TREX53765.2021.00008
Wenkai Han
Decision making and trust have both become rising topics in the research community of Visual Analytics (VA). Many efforts have been made to understand and facilitate making decisions with VA, as well as build and calibrate trust. However, previous research largely took VA as a tool to facilitate decision making, but did not explore the possibility to dissect each analytical step in VA as decision making and discuss how decision making theories can be utilized to improve the trustworthiness of decisions in VA. Therefore, this paper instead proposes such alternative take on the relation between decision making and VA, inspects the processes of visually analyzing data as decision making, and discusses how to leverage decision making theories to facilitate trustworthy decision making in VA.
{"title":"Making and Trusting Decisions in Visual Analytics","authors":"Wenkai Han","doi":"10.1109/TREX53765.2021.00008","DOIUrl":"https://doi.org/10.1109/TREX53765.2021.00008","url":null,"abstract":"Decision making and trust have both become rising topics in the research community of Visual Analytics (VA). Many efforts have been made to understand and facilitate making decisions with VA, as well as build and calibrate trust. However, previous research largely took VA as a tool to facilitate decision making, but did not explore the possibility to dissect each analytical step in VA as decision making and discuss how decision making theories can be utilized to improve the trustworthiness of decisions in VA. Therefore, this paper instead proposes such alternative take on the relation between decision making and VA, inspects the processes of visually analyzing data as decision making, and discusses how to leverage decision making theories to facilitate trustworthy decision making in VA.","PeriodicalId":345585,"journal":{"name":"2021 IEEE Workshop on TRust and EXpertise in Visual Analytics (TREX)","volume":"76 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"117331652","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-09-27DOI: 10.1109/TREX53765.2021.00010
U. Schlegel, D. Keim
Attributions are a common local explanation technique for deep learning models on single samples as they are easily extractable and demonstrate the relevance of input values. In many cases, heatmaps visualize such attributions for samples, for instance, on images. However, heatmaps are not always the ideal visualization to explain certain model decisions for other data types. In this review, we focus on attribution visualizations for time series. We collect attribution heatmap visualizations and some alternatives, discuss the advantages as well as disadvantages and give a short position towards future opportunities for attributions and explanations for time series.
{"title":"Time Series Model Attribution Visualizations as Explanations","authors":"U. Schlegel, D. Keim","doi":"10.1109/TREX53765.2021.00010","DOIUrl":"https://doi.org/10.1109/TREX53765.2021.00010","url":null,"abstract":"Attributions are a common local explanation technique for deep learning models on single samples as they are easily extractable and demonstrate the relevance of input values. In many cases, heatmaps visualize such attributions for samples, for instance, on images. However, heatmaps are not always the ideal visualization to explain certain model decisions for other data types. In this review, we focus on attribution visualizations for time series. We collect attribution heatmap visualizations and some alternatives, discuss the advantages as well as disadvantages and give a short position towards future opportunities for attributions and explanations for time series.","PeriodicalId":345585,"journal":{"name":"2021 IEEE Workshop on TRust and EXpertise in Visual Analytics (TREX)","volume":"148 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-09-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123237746","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：