Pub Date : 2020-10-01DOI: 10.1109/beliv51497.2020.00006
Eun Kyoung Choe
Alfie Abdul-Rahman, King's College London, UK Lyn Bartam, Simon Fraser University, Canada Jürgen Bernard, University of British Columbia, Canada Enrico Bertini, New York University, USA Alan Blackwell, University of Cambridge, UK Michelle Borkin, Northeastern University, USA Matthew Brehmer, Tableau, USA Remco Chang, Tufts University, USA Min Chen, University of Oxford, UK Eun Kyoung Choe, University of Maryland, USA Michael Correll, Tableau, USA Anamaria Crisan, Tableau, USA Evanthia Dimara, University of Konstanz, Germany Pierre Dragicevic, Inria, France Niklas Elmqvist, University of Maryland, USA Carla Freitas, Federal University of Rio Grande do Sul, Brazil Michael Gleicher, University of Wisconsin, Madison, USA Steve Haroz, Northwestern University, USA Tobias Isenberg, Inria, France Yvonne Jansen, Sorbonne University, France Robert Kosara, Tableau, USA Søren Knudsen, University of Calgary, Canada Kuno Kurzhals, ETH Zürich, Switzerland Heidi Lam, Tableau, USA Bongshin Lee, Microsoft Research, USA Eva Mayr, Danube University Krems, Austria Charles Perin, University of Victoria, Canada Catherine Plaisant, University of Maryland, US Margit Pohl, TU Wien, Austria Harald Reiterer, University of Konstanz, Germany Roy Ruddle, University of Leeds, UK Beatriz Sousa Santos, University of Aveiro, Portugal Giuseppe Santucci, Sapienza University of Rome, Italy Bahador Saket, Georgia Institute of Technology, USA Günther Schreder, Danube University Krems, Austria John Stasko, Georgia Institute of Technology, USA Danielle Szafir, Univeristy of Colorado Boulder, USA André Calero Valdez, Aachen University, Germany Chat Wacharamanotham, University of Zürich, Switzerland Markus Wagner, St. Pölten University of Applied Sciences Jagoda Walny, Canada Energy Regulator, Canada Chris Weaver, University of Oklahoma, USA
{"title":"BELIV 2020 Committees","authors":"Eun Kyoung Choe","doi":"10.1109/beliv51497.2020.00006","DOIUrl":"https://doi.org/10.1109/beliv51497.2020.00006","url":null,"abstract":"Alfie Abdul-Rahman, King's College London, UK Lyn Bartam, Simon Fraser University, Canada Jürgen Bernard, University of British Columbia, Canada Enrico Bertini, New York University, USA Alan Blackwell, University of Cambridge, UK Michelle Borkin, Northeastern University, USA Matthew Brehmer, Tableau, USA Remco Chang, Tufts University, USA Min Chen, University of Oxford, UK Eun Kyoung Choe, University of Maryland, USA Michael Correll, Tableau, USA Anamaria Crisan, Tableau, USA Evanthia Dimara, University of Konstanz, Germany Pierre Dragicevic, Inria, France Niklas Elmqvist, University of Maryland, USA Carla Freitas, Federal University of Rio Grande do Sul, Brazil Michael Gleicher, University of Wisconsin, Madison, USA Steve Haroz, Northwestern University, USA Tobias Isenberg, Inria, France Yvonne Jansen, Sorbonne University, France Robert Kosara, Tableau, USA Søren Knudsen, University of Calgary, Canada Kuno Kurzhals, ETH Zürich, Switzerland Heidi Lam, Tableau, USA Bongshin Lee, Microsoft Research, USA Eva Mayr, Danube University Krems, Austria Charles Perin, University of Victoria, Canada Catherine Plaisant, University of Maryland, US Margit Pohl, TU Wien, Austria Harald Reiterer, University of Konstanz, Germany Roy Ruddle, University of Leeds, UK Beatriz Sousa Santos, University of Aveiro, Portugal Giuseppe Santucci, Sapienza University of Rome, Italy Bahador Saket, Georgia Institute of Technology, USA Günther Schreder, Danube University Krems, Austria John Stasko, Georgia Institute of Technology, USA Danielle Szafir, Univeristy of Colorado Boulder, USA André Calero Valdez, Aachen University, Germany Chat Wacharamanotham, University of Zürich, Switzerland Markus Wagner, St. Pölten University of Applied Sciences Jagoda Walny, Canada Energy Regulator, Canada Chris Weaver, University of Oklahoma, USA","PeriodicalId":282674,"journal":{"name":"2020 IEEE Workshop on Evaluation and Beyond - Methodological Approaches to Visualization (BELIV)","volume":"2 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122686433","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2020-10-01DOI: 10.1109/BELIV51497.2020.00009
Priscilla Balestrucci, Katrin Angerbauer, C. Morariu, Robin Welsch, Lewis L. Chuang, D. Weiskopf, M. Ernst, M. Sedlmair
Among the many changes brought about by the COVID-19 pandemic, one of the most pressing for scientific research concerns user testing. For the researchers who conduct studies with human participants, the requirements for social distancing have created a need for reflecting on methodologies that previously seemed relatively straightforward. It has become clear from the emerging literature on the topic and from first-hand experiences of researchers that the restrictions due to the pandemic affect every aspect of the research pipeline. The current paper offers an initial reflection on user-based research, drawing on the authors’ own experiences and on the results of a survey that was conducted among researchers in different disciplines, primarily psychology, human-computer interaction (HCI), and visualization communities. While this sampling of researchers is by no means comprehensive, the multi-disciplinary approach and the consideration of different aspects of the research pipeline allow us to examine current and future challenges for user-based research. Through an exploration of these issues, this paper also invites others in the VIS—as well as in the wider—research community, to reflect on and discuss the ways in which the current crisis might also present new and previously unexplored opportunities.
{"title":"Pipelines Bent, Pipelines Broken: Interdisciplinary Self-Reflection on the Impact of COVID-19 on Current and Future Research (Position Paper)","authors":"Priscilla Balestrucci, Katrin Angerbauer, C. Morariu, Robin Welsch, Lewis L. Chuang, D. Weiskopf, M. Ernst, M. Sedlmair","doi":"10.1109/BELIV51497.2020.00009","DOIUrl":"https://doi.org/10.1109/BELIV51497.2020.00009","url":null,"abstract":"Among the many changes brought about by the COVID-19 pandemic, one of the most pressing for scientific research concerns user testing. For the researchers who conduct studies with human participants, the requirements for social distancing have created a need for reflecting on methodologies that previously seemed relatively straightforward. It has become clear from the emerging literature on the topic and from first-hand experiences of researchers that the restrictions due to the pandemic affect every aspect of the research pipeline. The current paper offers an initial reflection on user-based research, drawing on the authors’ own experiences and on the results of a survey that was conducted among researchers in different disciplines, primarily psychology, human-computer interaction (HCI), and visualization communities. While this sampling of researchers is by no means comprehensive, the multi-disciplinary approach and the consideration of different aspects of the research pipeline allow us to examine current and future challenges for user-based research. Through an exploration of these issues, this paper also invites others in the VIS—as well as in the wider—research community, to reflect on and discuss the ways in which the current crisis might also present new and previously unexplored opportunities.","PeriodicalId":282674,"journal":{"name":"2020 IEEE Workshop on Evaluation and Beyond - Methodological Approaches to Visualization (BELIV)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130081787","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2020-10-01DOI: 10.1109/BELIV51497.2020.00011
Annie Bares, Stephanie Zeller, Cullen D. Jackson, Daniel F. Keefe, F. Samsel
Visualization research and practice that incorporates the arts make claims to being more effective in connecting with users on a human level. However, these claims are difficult to measure quantitatively. In this paper, we present a follow-on study to use close reading, a humanities method from literary studies, to evaluate visualizations created using artistic processes [Bares 2020]. Close reading is a method in literary studies that we’ve previously explored as a method for evaluating visualizations. To use close reading as an evaluation method, we guide participants through a series of steps designed to prompt them to interpret the visualization’s formal, informational, and contextual features. Here we elaborate on our motivations for using close reading as a method to evaluate visualizations, and enumerate the procedures we used in the study to evaluate a 2D visualization, including modifications made because of the COVID-19 pandemic. Key findings of this study include that close reading is an effective formative method to elicit information related to interpretation and critique; user subject position; and suspicion or skepticism. Information gained through close reading is valuable in the visualization design and iteration processes, both related to designing features and other formal elements more effectively, as well as in considering larger questions of context and framing.
{"title":"Using Close Reading as a Method for Evaluating Visualizations","authors":"Annie Bares, Stephanie Zeller, Cullen D. Jackson, Daniel F. Keefe, F. Samsel","doi":"10.1109/BELIV51497.2020.00011","DOIUrl":"https://doi.org/10.1109/BELIV51497.2020.00011","url":null,"abstract":"Visualization research and practice that incorporates the arts make claims to being more effective in connecting with users on a human level. However, these claims are difficult to measure quantitatively. In this paper, we present a follow-on study to use close reading, a humanities method from literary studies, to evaluate visualizations created using artistic processes [Bares 2020]. Close reading is a method in literary studies that we’ve previously explored as a method for evaluating visualizations. To use close reading as an evaluation method, we guide participants through a series of steps designed to prompt them to interpret the visualization’s formal, informational, and contextual features. Here we elaborate on our motivations for using close reading as a method to evaluate visualizations, and enumerate the procedures we used in the study to evaluate a 2D visualization, including modifications made because of the COVID-19 pandemic. Key findings of this study include that close reading is an effective formative method to elicit information related to interpretation and critique; user subject position; and suspicion or skepticism. Information gained through close reading is valuable in the visualization design and iteration processes, both related to designing features and other formal elements more effectively, as well as in considering larger questions of context and framing.","PeriodicalId":282674,"journal":{"name":"2020 IEEE Workshop on Evaluation and Beyond - Methodological Approaches to Visualization (BELIV)","volume":"52 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130263458","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2020-10-01DOI: 10.1109/beliv51497.2020.00007
{"title":"BELIV 2020 Keynote: John Burn-Murdoch","authors":"","doi":"10.1109/beliv51497.2020.00007","DOIUrl":"https://doi.org/10.1109/beliv51497.2020.00007","url":null,"abstract":"","PeriodicalId":282674,"journal":{"name":"2020 IEEE Workshop on Evaluation and Beyond - Methodological Approaches to Visualization (BELIV)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131258125","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2020-10-01DOI: 10.1109/BELIV51497.2020.00017
Christian Knoll, Asil Çetin, Torsten Möller, Miriah D. Meyer
Participatory design is an approach in human-computer interaction that involves all relevant stakeholders coequally in the design process. A recent participatory method for visualization design is the creative visualization-opportunities (CVO) workshop, which is used to efficiently develop visualization design requirements in the early stages of applied visualization work. In this paper we report on our experiences of running four CVO workshops in different domains with diverse participants to explore new methods and variations of workshop variables. Through reflection on our experiences, we propose two contributions that extend existing guidance for planning, executing, and analyzing CVO workshops: a set of 12 pragmatic recommendations that extend and complement existing ones; and a recommended method for analyzing workshop results, called user stories. Additionally, we report on the outcomes of our successful workshops to provide evidence for the efficacy of CVO workshops.
{"title":"Extending Recommendations for Creative Visualization-Opportunities Workshops","authors":"Christian Knoll, Asil Çetin, Torsten Möller, Miriah D. Meyer","doi":"10.1109/BELIV51497.2020.00017","DOIUrl":"https://doi.org/10.1109/BELIV51497.2020.00017","url":null,"abstract":"Participatory design is an approach in human-computer interaction that involves all relevant stakeholders coequally in the design process. A recent participatory method for visualization design is the creative visualization-opportunities (CVO) workshop, which is used to efficiently develop visualization design requirements in the early stages of applied visualization work. In this paper we report on our experiences of running four CVO workshops in different domains with diverse participants to explore new methods and variations of workshop variables. Through reflection on our experiences, we propose two contributions that extend existing guidance for planning, executing, and analyzing CVO workshops: a set of 12 pragmatic recommendations that extend and complement existing ones; and a recommended method for analyzing workshop results, called user stories. Additionally, we report on the outcomes of our successful workshops to provide evidence for the efficacy of CVO workshops.","PeriodicalId":282674,"journal":{"name":"2020 IEEE Workshop on Evaluation and Beyond - Methodological Approaches to Visualization (BELIV)","volume":"55 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116758380","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2020-09-04DOI: 10.1109/BELIV51497.2020.00008
Tatiana Losev, Sarah Storteboom, Sheelagh Carpendale, S. Knudsen
We reflect on our experiences as designers of COVID-19 data visualizations working in a distributed synchronous design space during the pandemic. This is especially relevant as the pandemic posed new challenges to distributed collaboration amidst civic lockdown measures and an increased dependency on spatially distributed teamwork across almost all sectors. Working from home being ‘the new normal’, we explored potential solutions for collaborating and prototyping remotely from our own homes using the existing tools at our disposal. Since members of our cross-disciplinary team had different technical skills, we used a range of synchronous remote design tools and methods. We aimed to preserve the richness of co-located collaboration such as face-to-face physical presence, body gestures, facial expressions, and the making and sharing of physical artifacts. While meeting over Zoom, we sketched on paper and used digital collaboration tools, such as Miro and Google Docs. Using an auto-ethnographic approach, we articulate our challenges and strategies throughout the process, providing useful insights about synchronous distributed collaboration.
{"title":"Distributed Synchronous Visualization Design: Challenges and Strategies","authors":"Tatiana Losev, Sarah Storteboom, Sheelagh Carpendale, S. Knudsen","doi":"10.1109/BELIV51497.2020.00008","DOIUrl":"https://doi.org/10.1109/BELIV51497.2020.00008","url":null,"abstract":"We reflect on our experiences as designers of COVID-19 data visualizations working in a distributed synchronous design space during the pandemic. This is especially relevant as the pandemic posed new challenges to distributed collaboration amidst civic lockdown measures and an increased dependency on spatially distributed teamwork across almost all sectors. Working from home being ‘the new normal’, we explored potential solutions for collaborating and prototyping remotely from our own homes using the existing tools at our disposal. Since members of our cross-disciplinary team had different technical skills, we used a range of synchronous remote design tools and methods. We aimed to preserve the richness of co-located collaboration such as face-to-face physical presence, body gestures, facial expressions, and the making and sharing of physical artifacts. While meeting over Zoom, we sketched on paper and used digital collaboration tools, such as Miro and Google Docs. Using an auto-ethnographic approach, we articulate our challenges and strategies throughout the process, providing useful insights about synchronous distributed collaboration.","PeriodicalId":282674,"journal":{"name":"2020 IEEE Workshop on Evaluation and Beyond - Methodological Approaches to Visualization (BELIV)","volume":"15 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-09-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129031962","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2020-09-03DOI: 10.1109/BELIV51497.2020.00015
Damla Çay, T. Nagel, A. Yantaç
During the coronavirus pandemic, visualizations gained a new level of popularity and meaning for a wider audience. People were bombarded with a wide set of public health visualizations ranging from simple graphs to complex interactive dashboards. In a pandemic setting, where large amounts of the world population are socially distancing themselves, it becomes an urgent need to refine existing user experience evaluation methods for remote settings to understand how people make sense out of COVID-19 related visualizations. When evaluating visualizations aimed towards the general public with vastly different socio-demographic backgrounds and varying levels of technical savviness and data literacy, it is important to understand user feedback beyond aspects such as speed, task accuracy, or usability problems. As a part of this wider evaluation perspective, micro-phenomenology has been used to evaluate static and narrative visualizations to reveal the lived experience in a detailed way. Building upon these studies, we conducted a user study to understand how to employ Elicitation (aka Micro-phenomenological) interviews in remote settings. In a case study, we investigated what experiences the participants had with map-based interactive visualizations. Our findings reveal positive and negative aspects of conducting Elicitation interviews remotely. Our results can inform the process of planning and executing remote Elicitation interviews to evaluate interactive visualizations. In addition, we share recommendations regarding visualization techniques and interaction design about public health data.
{"title":"Understanding User Experience of COVID-19 Maps through Remote Elicitation Interviews","authors":"Damla Çay, T. Nagel, A. Yantaç","doi":"10.1109/BELIV51497.2020.00015","DOIUrl":"https://doi.org/10.1109/BELIV51497.2020.00015","url":null,"abstract":"During the coronavirus pandemic, visualizations gained a new level of popularity and meaning for a wider audience. People were bombarded with a wide set of public health visualizations ranging from simple graphs to complex interactive dashboards. In a pandemic setting, where large amounts of the world population are socially distancing themselves, it becomes an urgent need to refine existing user experience evaluation methods for remote settings to understand how people make sense out of COVID-19 related visualizations. When evaluating visualizations aimed towards the general public with vastly different socio-demographic backgrounds and varying levels of technical savviness and data literacy, it is important to understand user feedback beyond aspects such as speed, task accuracy, or usability problems. As a part of this wider evaluation perspective, micro-phenomenology has been used to evaluate static and narrative visualizations to reveal the lived experience in a detailed way. Building upon these studies, we conducted a user study to understand how to employ Elicitation (aka Micro-phenomenological) interviews in remote settings. In a case study, we investigated what experiences the participants had with map-based interactive visualizations. Our findings reveal positive and negative aspects of conducting Elicitation interviews remotely. Our results can inform the process of planning and executing remote Elicitation interviews to evaluate interactive visualizations. In addition, we share recommendations regarding visualization techniques and interaction design about public health data.","PeriodicalId":282674,"journal":{"name":"2020 IEEE Workshop on Evaluation and Beyond - Methodological Approaches to Visualization (BELIV)","volume":"45 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-09-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122970217","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：