首页 > 最新文献

Workshop on Beyond Time and Errors: Novel Evaluation Methods for Visualization最新文献

英文 中文
Lowering the Barrier for Successful Replication and Evaluation 降低成功复制和评估的障碍
Pub Date : 2018-10-01 DOI: 10.1109/BELIV.2018.8634201
Hendrik Lücke-Tieke, Marcel Beuth, Philipp Schader, T. May, J. Bernard, J. Kohlhammer
Evaluation of a visualization technique is complex and time-consuming. We present a system that aims at easing design, creation and execution of controlled experiments for visualizations in the web. We include of parameterizable visualization generation services, thus separating the visualization implementation from study design and execution. This enables experimenters to design and run multiple experiments on the same visualization service in parallel, replicate experiments, and compare different visualization services quickly. The system supports the range from simple questionnaires to visualization-specific interaction techniques as well as automated task generation based on dynamic sampling of parameter spaces. We feature two examples to demonstrate our service-based approach. One example demonstrates how a suite of successive experiments can be conducted, while the other example includes an extended replication study.
可视化技术的评估既复杂又耗时。我们提出了一个系统,旨在简化设计,创建和执行在网络可视化控制实验。我们包括可参数化的可视化生成服务,从而将可视化实现从研究设计和执行中分离出来。这使实验人员能够在同一可视化服务上并行设计和运行多个实验,复制实验,并快速比较不同的可视化服务。该系统支持从简单的问卷调查到可视化特定交互技术的范围,以及基于参数空间动态采样的自动任务生成。我们将提供两个示例来演示我们基于服务的方法。一个例子演示了如何进行一系列连续的实验,而另一个例子包括一个扩展的复制研究。
{"title":"Lowering the Barrier for Successful Replication and Evaluation","authors":"Hendrik Lücke-Tieke, Marcel Beuth, Philipp Schader, T. May, J. Bernard, J. Kohlhammer","doi":"10.1109/BELIV.2018.8634201","DOIUrl":"https://doi.org/10.1109/BELIV.2018.8634201","url":null,"abstract":"Evaluation of a visualization technique is complex and time-consuming. We present a system that aims at easing design, creation and execution of controlled experiments for visualizations in the web. We include of parameterizable visualization generation services, thus separating the visualization implementation from study design and execution. This enables experimenters to design and run multiple experiments on the same visualization service in parallel, replicate experiments, and compare different visualization services quickly. The system supports the range from simple questionnaires to visualization-specific interaction techniques as well as automated task generation based on dynamic sampling of parameter spaces. We feature two examples to demonstrate our service-based approach. One example demonstrates how a suite of successive experiments can be conducted, while the other example includes an extended replication study.","PeriodicalId":235801,"journal":{"name":"Workshop on Beyond Time and Errors: Novel Evaluation Methods for Visualization","volume":"44 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130536930","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 5
Design Study Contributions Come in Different Guises: Seven Guiding Scenarios 设计研究的贡献有不同的面貌:七个指导方案
Pub Date : 2016-10-24 DOI: 10.1145/2993901.2993913
M. Sedlmair
Design studies are projects in which visualization researchers seek to design visualization tools that help solving challenging real-world problems faced by domain experts. While design studies have become a vital component of visualization research, reflecting on actionable contributions from them often poses challenges. The goal of this paper is to better characterize different contributions that can result from design study projects. Towards this goal, a set of seven guiding scenarios for characterizing design study contributions is proposed. The scenarios are meant to help authors identify and depict design study contributions that are interesting and actionable for other visualization researchers. They are also meant to provide better guidance in evaluating design study contributions in the reviewing process.
设计研究是可视化研究人员寻求设计可视化工具来帮助解决领域专家面临的具有挑战性的现实问题的项目。虽然设计研究已成为可视化研究的重要组成部分,但反映他们的可操作贡献往往带来挑战。本文的目的是更好地描述设计研究项目可能产生的不同贡献。为了实现这一目标,本文提出了一套七个指导方案来描述设计研究的贡献。这些场景旨在帮助作者识别和描述设计研究的贡献,这些贡献对其他可视化研究人员来说是有趣和可操作的。它们也意味着在评审过程中为评估设计研究贡献提供更好的指导。
{"title":"Design Study Contributions Come in Different Guises: Seven Guiding Scenarios","authors":"M. Sedlmair","doi":"10.1145/2993901.2993913","DOIUrl":"https://doi.org/10.1145/2993901.2993913","url":null,"abstract":"Design studies are projects in which visualization researchers seek to design visualization tools that help solving challenging real-world problems faced by domain experts. While design studies have become a vital component of visualization research, reflecting on actionable contributions from them often poses challenges. The goal of this paper is to better characterize different contributions that can result from design study projects. Towards this goal, a set of seven guiding scenarios for characterizing design study contributions is proposed. The scenarios are meant to help authors identify and depict design study contributions that are interesting and actionable for other visualization researchers. They are also meant to provide better guidance in evaluating design study contributions in the reviewing process.","PeriodicalId":235801,"journal":{"name":"Workshop on Beyond Time and Errors: Novel Evaluation Methods for Visualization","volume":"11 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-10-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126362425","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 18
Evaluating Information Visualization on Mobile Devices: Gaps and Challenges in the Empirical Evaluation Design Space 评估移动设备上的信息可视化:经验评估设计空间的差距和挑战
Pub Date : 2016-10-24 DOI: 10.1145/2993901.2993906
Kerstin Blumenstein, C. Niederer, Markus Wagner, Grischa Schmiedl, A. Rind, W. Aigner
With their increasingly widespread use, mobile devices have become a highly relevant target environment for Information Visualization. However, far too little attention has been paid to evaluation of interactive visualization techniques on mobile devices. To fill this gap, this paper provides a structured overview of the commonly used evaluation approaches for mobile visualization. For this, it systematically reviews the scientific literature of major InfoVis and HCI venues and categorizes the relevant work based on six dimensions circumscribing the design and evaluation space for visualization on mobile devices. Based on the 21 evaluations reviewed, reproducibility, device variety and usage environment surface as the three main issues in evaluation of information visualization on mobile devices. To overcome these issues, we argue for a transparent description of all research aspects and propose to focus more on context of usage and technology.
随着移动设备的日益广泛使用,移动设备已成为信息可视化的一个高度相关的目标环境。然而,人们对移动设备上的交互式可视化技术的评价关注甚少。为了填补这一空白,本文提供了移动可视化常用评估方法的结构化概述。为此,本文系统地回顾了主要InfoVis和HCI场所的科学文献,并基于六个维度对移动设备可视化设计和评估空间进行了分类。基于所回顾的21项评价,再现性、设备多样性和使用环境成为移动设备信息可视化评价的三个主要问题。为了克服这些问题,我们主张对所有研究方面进行透明的描述,并建议更多地关注使用和技术的背景。
{"title":"Evaluating Information Visualization on Mobile Devices: Gaps and Challenges in the Empirical Evaluation Design Space","authors":"Kerstin Blumenstein, C. Niederer, Markus Wagner, Grischa Schmiedl, A. Rind, W. Aigner","doi":"10.1145/2993901.2993906","DOIUrl":"https://doi.org/10.1145/2993901.2993906","url":null,"abstract":"With their increasingly widespread use, mobile devices have become a highly relevant target environment for Information Visualization. However, far too little attention has been paid to evaluation of interactive visualization techniques on mobile devices. To fill this gap, this paper provides a structured overview of the commonly used evaluation approaches for mobile visualization. For this, it systematically reviews the scientific literature of major InfoVis and HCI venues and categorizes the relevant work based on six dimensions circumscribing the design and evaluation space for visualization on mobile devices. Based on the 21 evaluations reviewed, reproducibility, device variety and usage environment surface as the three main issues in evaluation of information visualization on mobile devices. To overcome these issues, we argue for a transparent description of all research aspects and propose to focus more on context of usage and technology.","PeriodicalId":235801,"journal":{"name":"Workshop on Beyond Time and Errors: Novel Evaluation Methods for Visualization","volume":"39 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-10-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114513891","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 20
A Nested Workflow Model for Visual Analytics Design and Validation 可视化分析设计和验证的嵌套工作流模型
Pub Date : 2016-10-24 DOI: 10.1145/2993901.2993915
P. Federico, Albert Amor-Amoros, S. Miksch
Characterizing the problem domain and understanding users' practices and processes are recognized as important steps in order to design and validate visualization, but are often disregarded in practice, also because of their complexity. We introduce the nested workflow model for design and validation of visual analytics, aimed at providing designers with a powerful and expressive modelling tool. This model enables the description of visual analytics processes, at different design levels, in terms of tasks, data, and users, including complex workflow patterns, data and knowledge flows, and collaboration between users. We discuss its application to two visual analytics projects, demonstrating its usefulness for their design and validation.
描述问题域和理解用户的实践和过程被认为是设计和验证可视化的重要步骤,但在实践中经常被忽视,也是因为它们的复杂性。我们介绍了用于可视化分析设计和验证的嵌套工作流模型,旨在为设计人员提供一个强大而富有表现力的建模工具。该模型支持在不同的设计级别上,根据任务、数据和用户描述可视化分析过程,包括复杂的工作流模式、数据和知识流,以及用户之间的协作。我们讨论了它在两个可视化分析项目中的应用,展示了它对设计和验证的有用性。
{"title":"A Nested Workflow Model for Visual Analytics Design and Validation","authors":"P. Federico, Albert Amor-Amoros, S. Miksch","doi":"10.1145/2993901.2993915","DOIUrl":"https://doi.org/10.1145/2993901.2993915","url":null,"abstract":"Characterizing the problem domain and understanding users' practices and processes are recognized as important steps in order to design and validate visualization, but are often disregarded in practice, also because of their complexity. We introduce the nested workflow model for design and validation of visual analytics, aimed at providing designers with a powerful and expressive modelling tool. This model enables the description of visual analytics processes, at different design levels, in terms of tasks, data, and users, including complex workflow patterns, data and knowledge flows, and collaboration between users. We discuss its application to two visual analytics projects, demonstrating its usefulness for their design and validation.","PeriodicalId":235801,"journal":{"name":"Workshop on Beyond Time and Errors: Novel Evaluation Methods for Visualization","volume":"102 ","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-10-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114049455","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 8
Cognitive Stages in Visual Data Exploration 视觉数据探索中的认知阶段
Pub Date : 2016-10-24 DOI: 10.1145/2993901.2993902
M. A. Yalçın, N. Elmqvist, B. Bederson
Data exploration requires forming analysis goals, planning actions and evaluating results effectively, all of which are complex cognitive activities. Therefore, the data exploration and analysis process can be improved through a principled and comprehensive approach to analyzing the cognitive activities of the user given a data exploration tool. However, many taxonomies and evaluations focus on a specific tool or specific design guides instead of cognitive activities comprehensively. In this paper, we first present the Cognitive Exploration Framework that identifies six stages of cognitive activities in visual data exploration. These stages are a combination of two activities---planning and assessing---across data analysis, interaction, and visualization. Cognitive barriers in each stage can lower the success and speed of data exploration. The framework also identifies the factors of decision-making, existing knowledge and motivation that influence cognitive activities. We argue that cognitive stages can be supported by improving the design of tools rather than their computing capabilities. We demonstrate how the framework clarifies the structured relationship between design guides to specific cognitive stages. In particular, the framework can also be used to guide evaluation of data exploration tools. To reveal cognitive barriers in each stage, we focused on the failures instead of success stories, and on motivating self-driven open-ended exploration instead of using benchmarked tasks on fixed datasets. With these goals, we studied short-term casual use of an exploratory tool by novices with limited training. Our results reveal cognitive barriers across all stages. We also discuss directions for future research and applications.
数据探索需要形成分析目标、规划行动、有效评价结果,这些都是复杂的认知活动。因此,通过对给定数据探索工具的用户的认知活动进行原则性和综合性的分析,可以改进数据探索和分析过程。然而,许多分类法和评估侧重于特定的工具或特定的设计指南,而不是全面的认知活动。在本文中,我们首先提出了认知探索框架,该框架确定了视觉数据探索中认知活动的六个阶段。这些阶段是两个活动的组合——计划和评估——跨越数据分析、交互和可视化。每个阶段的认知障碍都会降低数据探索的成功率和速度。该框架还确定了影响认知活动的决策因素、现有知识和动机。我们认为,认知阶段可以通过改进工具的设计而不是它们的计算能力来支持。我们演示了框架如何澄清设计指南与特定认知阶段之间的结构化关系。特别是,该框架还可用于指导数据探索工具的评估。为了揭示每个阶段的认知障碍,我们专注于失败而不是成功的故事,并专注于激励自我驱动的开放式探索,而不是在固定数据集上使用基准任务。为了达到这些目标,我们研究了训练有限的新手短期随意使用探索性工具的情况。我们的研究结果揭示了各个阶段的认知障碍。讨论了今后的研究和应用方向。
{"title":"Cognitive Stages in Visual Data Exploration","authors":"M. A. Yalçın, N. Elmqvist, B. Bederson","doi":"10.1145/2993901.2993902","DOIUrl":"https://doi.org/10.1145/2993901.2993902","url":null,"abstract":"Data exploration requires forming analysis goals, planning actions and evaluating results effectively, all of which are complex cognitive activities. Therefore, the data exploration and analysis process can be improved through a principled and comprehensive approach to analyzing the cognitive activities of the user given a data exploration tool. However, many taxonomies and evaluations focus on a specific tool or specific design guides instead of cognitive activities comprehensively. In this paper, we first present the Cognitive Exploration Framework that identifies six stages of cognitive activities in visual data exploration. These stages are a combination of two activities---planning and assessing---across data analysis, interaction, and visualization. Cognitive barriers in each stage can lower the success and speed of data exploration. The framework also identifies the factors of decision-making, existing knowledge and motivation that influence cognitive activities. We argue that cognitive stages can be supported by improving the design of tools rather than their computing capabilities. We demonstrate how the framework clarifies the structured relationship between design guides to specific cognitive stages. In particular, the framework can also be used to guide evaluation of data exploration tools. To reveal cognitive barriers in each stage, we focused on the failures instead of success stories, and on motivating self-driven open-ended exploration instead of using benchmarked tasks on fixed datasets. With these goals, we studied short-term casual use of an exploratory tool by novices with limited training. Our results reveal cognitive barriers across all stages. We also discuss directions for future research and applications.","PeriodicalId":235801,"journal":{"name":"Workshop on Beyond Time and Errors: Novel Evaluation Methods for Visualization","volume":"120 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-10-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114256299","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 23
Beyond Usability and Performance: A Review of User Experience-focused Evaluations in Visualization 超越可用性和性能:以用户体验为中心的可视化评估综述
Pub Date : 2016-10-24 DOI: 10.1145/2993901.2993903
B. Saket, A. Endert, J. Stasko
Traditionally, studies of data visualization techniques and systems have evaluated visualizations with respect to usability goals such as effectiveness and efficiency. These studies assess performance-related metrics such as time and correctness of participants completing analytic tasks. Alternatively, several studies in InfoVis recently have evaluated visualizations by investigating user experience goals such as memorability, engagement, enjoyment and fun. These studies employ somewhat different evaluation methodologies to assess these other goals. The growing number of these studies, their alternative methodologies, and disagreements concerning their importance have motivated us to more carefully examine them. In this article, we review this growing collection of visualization evaluations that examine user experience goals and we discuss multiple issues regarding the studies including questions about their motivation and utility. Our aim is to provide a resource for future work that plans to evaluate visualizations using these goals.
传统上,数据可视化技术和系统的研究已经根据可用性目标(如有效性和效率)来评估可视化。这些研究评估与表现相关的指标,如参与者完成分析任务的时间和正确性。另外,InfoVis最近的一些研究通过调查用户体验目标(如可记忆性、参与度、享受和乐趣)来评估可视化。这些研究采用一些不同的评估方法来评估这些其他目标。这些研究的数量越来越多,它们的替代方法,以及对它们重要性的分歧,促使我们更仔细地研究它们。在这篇文章中,我们回顾了越来越多的用于检查用户体验目标的可视化评估,并讨论了有关这些研究的多个问题,包括它们的动机和效用问题。我们的目的是为未来的工作提供一个资源,计划使用这些目标来评估可视化。
{"title":"Beyond Usability and Performance: A Review of User Experience-focused Evaluations in Visualization","authors":"B. Saket, A. Endert, J. Stasko","doi":"10.1145/2993901.2993903","DOIUrl":"https://doi.org/10.1145/2993901.2993903","url":null,"abstract":"Traditionally, studies of data visualization techniques and systems have evaluated visualizations with respect to usability goals such as effectiveness and efficiency. These studies assess performance-related metrics such as time and correctness of participants completing analytic tasks. Alternatively, several studies in InfoVis recently have evaluated visualizations by investigating user experience goals such as memorability, engagement, enjoyment and fun. These studies employ somewhat different evaluation methodologies to assess these other goals. The growing number of these studies, their alternative methodologies, and disagreements concerning their importance have motivated us to more carefully examine them. In this article, we review this growing collection of visualization evaluations that examine user experience goals and we discuss multiple issues regarding the studies including questions about their motivation and utility. Our aim is to provide a resource for future work that plans to evaluate visualizations using these goals.","PeriodicalId":235801,"journal":{"name":"Workshop on Beyond Time and Errors: Novel Evaluation Methods for Visualization","volume":"77 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-10-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127620013","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 92
Why Evaluating Uncertainty Visualization is Error Prone 为什么评估不确定性可视化容易出错
Pub Date : 2016-10-24 DOI: 10.1145/2993901.2993919
J. Hullman
Evaluating a visualization that depicts uncertainty is fraught with challenges due to the complex psychology of uncertainty. However, relatively little attention is paid to selecting and motivating a chosen interpretation or elicitation method for subjective probabilities in the uncertainty visualization literature. I survey existing evaluation work in uncertainty visualization, and examine how research in judgment and decision-making that focuses on subjective uncertainty elicitation sheds light on common approaches in visualization. I propose suggestions for practice aimed at reducing errors and noise related to how ground truth is defined for subjective probability estimates, the choice of an elicitation method, and the strategies used by subjects making judgments with an uncertainty visualization.
由于不确定性的复杂心理,评估描绘不确定性的可视化是充满挑战的。然而,在不确定性可视化文献中,对主观概率的选择和激发选择的解释或启发方法的关注相对较少。本文综述了不确定性可视化中现有的评估工作,并研究了以主观不确定性启发为重点的判断和决策研究如何揭示了可视化中的常用方法。我提出了一些实践建议,旨在减少与如何为主观概率估计定义基础真理有关的错误和噪音,启发方法的选择,以及受试者使用不确定性可视化做出判断所使用的策略。
{"title":"Why Evaluating Uncertainty Visualization is Error Prone","authors":"J. Hullman","doi":"10.1145/2993901.2993919","DOIUrl":"https://doi.org/10.1145/2993901.2993919","url":null,"abstract":"Evaluating a visualization that depicts uncertainty is fraught with challenges due to the complex psychology of uncertainty. However, relatively little attention is paid to selecting and motivating a chosen interpretation or elicitation method for subjective probabilities in the uncertainty visualization literature. I survey existing evaluation work in uncertainty visualization, and examine how research in judgment and decision-making that focuses on subjective uncertainty elicitation sheds light on common approaches in visualization. I propose suggestions for practice aimed at reducing errors and noise related to how ground truth is defined for subjective probability estimates, the choice of an elicitation method, and the strategies used by subjects making judgments with an uncertainty visualization.","PeriodicalId":235801,"journal":{"name":"Workshop on Beyond Time and Errors: Novel Evaluation Methods for Visualization","volume":"50 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-10-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133192772","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 22
Information Theoretic Measures for Visual Analytics: The Silver Ticket? 视觉分析的信息理论措施:银券?
Pub Date : 2016-10-24 DOI: 10.1145/2993901.2993920
Laura A. McNamara, Travis L. Bauer, Michael J. Haass, Laura E. Matzen
In this paper, we argue that information theoretic measures may provide a robust, broadly applicable, repeatable metric to assess how a system enables people to reduce high-dimensional data into topically relevant subsets of information. Explosive growth in electronic data necessitates the development of systems that balance automation with human cognitive engagement to facilitate pattern discovery, analysis and characterization, variously described as "cognitive augmentation" or "insight generation." However, operationalizing the concept of insight in any measurable way remains a difficult challenge for visualization researchers. The "golden ticket" of insight evaluation would be a precise, generalizable, repeatable, and ecologically valid metric that indicates the relative utility of a system in heightening cognitive performance or facilitating insights. Unfortunately, the golden ticket does not yet exist. In its place, we are exploring information theoretic measures derived from Shannon's ideas about information and entropy as a starting point for precise, repeatable, and generalizable approaches for evaluating analytic tools. We are specifically concerned with needle-in-haystack workflows that require interactive search, classification, and reduction of very large heterogeneous datasets into manageable, task-relevant subsets of information. We assert that systems aimed at facilitating pattern discovery, characterization and analysis -- i.e., "insight" - must afford an efficient means of sorting the needles from the chaff; and simple compressibility measures provide a way of tracking changes in information content as people shape meaning from data.
在本文中,我们认为信息理论度量可以提供一个健壮的,广泛适用的,可重复的度量来评估一个系统如何使人们将高维数据减少到主题相关的信息子集。电子数据的爆炸式增长需要开发平衡自动化与人类认知参与的系统,以促进模式发现、分析和表征,这被称为“认知增强”或“洞察生成”。然而,以任何可测量的方式实现洞察力的概念仍然是可视化研究人员面临的一个困难挑战。洞察力评估的“金券”将是一个精确的、可推广的、可重复的、生态有效的度量,它表明系统在提高认知表现或促进洞察力方面的相对效用。不幸的是,金奖券还不存在。取而代之的是,我们正在探索从香农关于信息和熵的思想中衍生出来的信息理论度量,作为评估分析工具的精确、可重复和可推广方法的起点。我们特别关注大海捞针的工作流程,它需要交互式搜索、分类和将非常大的异构数据集减少到可管理的、与任务相关的信息子集。我们断言,旨在促进模式发现、表征和分析的系统——即“洞察力”——必须提供一种有效的方法,从谷壳中挑选出针;当人们从数据中塑造意义时,简单的可压缩性度量提供了一种跟踪信息内容变化的方法。
{"title":"Information Theoretic Measures for Visual Analytics: The Silver Ticket?","authors":"Laura A. McNamara, Travis L. Bauer, Michael J. Haass, Laura E. Matzen","doi":"10.1145/2993901.2993920","DOIUrl":"https://doi.org/10.1145/2993901.2993920","url":null,"abstract":"In this paper, we argue that information theoretic measures may provide a robust, broadly applicable, repeatable metric to assess how a system enables people to reduce high-dimensional data into topically relevant subsets of information. Explosive growth in electronic data necessitates the development of systems that balance automation with human cognitive engagement to facilitate pattern discovery, analysis and characterization, variously described as \"cognitive augmentation\" or \"insight generation.\" However, operationalizing the concept of insight in any measurable way remains a difficult challenge for visualization researchers. The \"golden ticket\" of insight evaluation would be a precise, generalizable, repeatable, and ecologically valid metric that indicates the relative utility of a system in heightening cognitive performance or facilitating insights. Unfortunately, the golden ticket does not yet exist. In its place, we are exploring information theoretic measures derived from Shannon's ideas about information and entropy as a starting point for precise, repeatable, and generalizable approaches for evaluating analytic tools. We are specifically concerned with needle-in-haystack workflows that require interactive search, classification, and reduction of very large heterogeneous datasets into manageable, task-relevant subsets of information. We assert that systems aimed at facilitating pattern discovery, characterization and analysis -- i.e., \"insight\" - must afford an efficient means of sorting the needles from the chaff; and simple compressibility measures provide a way of tracking changes in information content as people shape meaning from data.","PeriodicalId":235801,"journal":{"name":"Workshop on Beyond Time and Errors: Novel Evaluation Methods for Visualization","volume":"63 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-10-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132576981","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 1
Using Concrete and Realistic Data in Evaluating Initial Visualization Designs 使用具体和现实的数据评估初始可视化设计
Pub Date : 2016-10-24 DOI: 10.1145/2993901.2993917
S. Knudsen, Jeppe Gerner Pedersen, Thor Herdal, J. E. Larsen
We explore means of designing and evaluating initial visualization ideas, with concrete and realistic data in cases where data is not readily available. Our approach is useful in exploring new domains and avenues for visualization, and contrasts other visualization work, which typically operate under the assumption that data has already been collected, and is ready to be visualized. We argue that it is sensible to understand data requirements and evaluate the potential value of visualization before devising means of automatic data collection. We base our exploration on three cases selected to span a range of factors, such as the role of the person doing the data collection and the type of instrumentation used. The three cases relate to visualizing sports, construction, and cooking domain data, and use primarily time-domain data and visualizations. For each case, we briefly describe the design case and problem, the manner in which we collected data, and the findings obtained from evaluations. Afterwards, we describe four factors of our data collection approach, and discuss potential outcomes from it.
我们探索设计和评估初始可视化想法的方法,在数据不容易获得的情况下,使用具体和现实的数据。我们的方法在探索可视化的新领域和途径方面很有用,并与其他可视化工作形成对比,这些工作通常是在数据已经收集并准备好进行可视化的假设下进行的。我们认为,在设计自动数据收集方法之前,了解数据需求并评估可视化的潜在价值是明智的。我们的研究基于三个案例,这些案例选择了一系列因素,如数据收集人员的角色和使用的仪器类型。这三个案例涉及到可视化运动、建筑和烹饪领域数据,并主要使用时域数据和可视化。对于每个案例,我们简要地描述了设计案例和问题,我们收集数据的方式,以及从评估中获得的发现。之后,我们描述了我们的数据收集方法的四个因素,并讨论了它的潜在结果。
{"title":"Using Concrete and Realistic Data in Evaluating Initial Visualization Designs","authors":"S. Knudsen, Jeppe Gerner Pedersen, Thor Herdal, J. E. Larsen","doi":"10.1145/2993901.2993917","DOIUrl":"https://doi.org/10.1145/2993901.2993917","url":null,"abstract":"We explore means of designing and evaluating initial visualization ideas, with concrete and realistic data in cases where data is not readily available. Our approach is useful in exploring new domains and avenues for visualization, and contrasts other visualization work, which typically operate under the assumption that data has already been collected, and is ready to be visualized. We argue that it is sensible to understand data requirements and evaluate the potential value of visualization before devising means of automatic data collection. We base our exploration on three cases selected to span a range of factors, such as the role of the person doing the data collection and the type of instrumentation used. The three cases relate to visualizing sports, construction, and cooking domain data, and use primarily time-domain data and visualizations. For each case, we briefly describe the design case and problem, the manner in which we collected data, and the findings obtained from evaluations. Afterwards, we describe four factors of our data collection approach, and discuss potential outcomes from it.","PeriodicalId":235801,"journal":{"name":"Workshop on Beyond Time and Errors: Novel Evaluation Methods for Visualization","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-10-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132401932","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 4
A Survey on Interaction Log Analysis for Evaluating Exploratory Visualizations 用于评价探索性可视化的交互日志分析综述
Pub Date : 2016-10-24 DOI: 10.1145/2993901.2993912
Omar Eltayeby, Wenwen Dou
The trend of exploratory visualization development has driven the visual analytics (VA) community to design special evaluation methods. The main goals of these evaluations are to understand the exploration process and improve it by recording users' interactions and thoughts. Some of the recent works have focused on performing manual evaluations of the interaction logs, however, lately some researchers have taken the step towards automating the process using interaction logs. In this paper we show the capability of how interaction log analysis can be automated by summarizing previous works' steps into building blocks. In addition, we demonstrate the use of each building block by showing their methodologies as use case scenarios, such as how to encode and segment interactions and what machine learning algorithms can automate the process. We also link the studies reviewed with sensemaking aspects and interaction taxonomies selection.
探索性可视化的发展趋势推动了可视化分析界设计特殊的评价方法。这些评估的主要目标是了解探索过程,并通过记录用户的交互和想法来改进它。最近的一些工作集中在执行交互日志的手动评估上,然而,最近一些研究人员已经采取了使用交互日志实现过程自动化的步骤。在本文中,我们通过将以前工作的步骤总结为构建块来展示交互日志分析如何自动化的能力。此外,我们通过将每个构建块的方法显示为用例场景来演示每个构建块的使用,例如如何编码和分割交互以及哪些机器学习算法可以使该过程自动化。我们还将这些研究与语义构建和相互作用分类选择联系起来。
{"title":"A Survey on Interaction Log Analysis for Evaluating Exploratory Visualizations","authors":"Omar Eltayeby, Wenwen Dou","doi":"10.1145/2993901.2993912","DOIUrl":"https://doi.org/10.1145/2993901.2993912","url":null,"abstract":"The trend of exploratory visualization development has driven the visual analytics (VA) community to design special evaluation methods. The main goals of these evaluations are to understand the exploration process and improve it by recording users' interactions and thoughts. Some of the recent works have focused on performing manual evaluations of the interaction logs, however, lately some researchers have taken the step towards automating the process using interaction logs. In this paper we show the capability of how interaction log analysis can be automated by summarizing previous works' steps into building blocks. In addition, we demonstrate the use of each building block by showing their methodologies as use case scenarios, such as how to encode and segment interactions and what machine learning algorithms can automate the process. We also link the studies reviewed with sensemaking aspects and interaction taxonomies selection.","PeriodicalId":235801,"journal":{"name":"Workshop on Beyond Time and Errors: Novel Evaluation Methods for Visualization","volume":"25 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-10-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121832947","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 12
期刊
Workshop on Beyond Time and Errors: Novel Evaluation Methods for Visualization
全部 Acc. Chem. Res. ACS Applied Bio Materials ACS Appl. Electron. Mater. ACS Appl. Energy Mater. ACS Appl. Mater. Interfaces ACS Appl. Nano Mater. ACS Appl. Polym. Mater. ACS BIOMATER-SCI ENG ACS Catal. ACS Cent. Sci. ACS Chem. Biol. ACS Chemical Health & Safety ACS Chem. Neurosci. ACS Comb. Sci. ACS Earth Space Chem. ACS Energy Lett. ACS Infect. Dis. ACS Macro Lett. ACS Mater. Lett. ACS Med. Chem. Lett. ACS Nano ACS Omega ACS Photonics ACS Sens. ACS Sustainable Chem. Eng. ACS Synth. Biol. Anal. Chem. BIOCHEMISTRY-US Bioconjugate Chem. BIOMACROMOLECULES Chem. Res. Toxicol. Chem. Rev. Chem. Mater. CRYST GROWTH DES ENERG FUEL Environ. Sci. Technol. Environ. Sci. Technol. Lett. Eur. J. Inorg. Chem. IND ENG CHEM RES Inorg. Chem. J. Agric. Food. Chem. J. Chem. Eng. Data J. Chem. Educ. J. Chem. Inf. Model. J. Chem. Theory Comput. J. Med. Chem. J. Nat. Prod. J PROTEOME RES J. Am. Chem. Soc. LANGMUIR MACROMOLECULES Mol. Pharmaceutics Nano Lett. Org. Lett. ORG PROCESS RES DEV ORGANOMETALLICS J. Org. Chem. J. Phys. Chem. J. Phys. Chem. A J. Phys. Chem. B J. Phys. Chem. C J. Phys. Chem. Lett. Analyst Anal. Methods Biomater. Sci. Catal. Sci. Technol. Chem. Commun. Chem. Soc. Rev. CHEM EDUC RES PRACT CRYSTENGCOMM Dalton Trans. Energy Environ. Sci. ENVIRON SCI-NANO ENVIRON SCI-PROC IMP ENVIRON SCI-WAT RES Faraday Discuss. Food Funct. Green Chem. Inorg. Chem. Front. Integr. Biol. J. Anal. At. Spectrom. J. Mater. Chem. A J. Mater. Chem. B J. Mater. Chem. C Lab Chip Mater. Chem. Front. Mater. Horiz. MEDCHEMCOMM Metallomics Mol. Biosyst. Mol. Syst. Des. Eng. Nanoscale Nanoscale Horiz. Nat. Prod. Rep. New J. Chem. Org. Biomol. Chem. Org. Chem. Front. PHOTOCH PHOTOBIO SCI PCCP Polym. Chem.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1