In this position paper, we describe two methods for the analysis of sequences of interaction with information visualization tools -- log file analysis and thinking aloud. Such an analysis is valuable because it can help designers to understand cognition processes of the users and, as a consequence, to improve the design of information visualizations. In this context, we also discuss the issue of categorization of user activities. Categorization helps researchers to generalize results and compare different information visualization tools.
{"title":"Methodologies for the analysis of usage patterns in information visualization","authors":"M. Pohl","doi":"10.1145/2442576.2442593","DOIUrl":"https://doi.org/10.1145/2442576.2442593","url":null,"abstract":"In this position paper, we describe two methods for the analysis of sequences of interaction with information visualization tools -- log file analysis and thinking aloud. Such an analysis is valuable because it can help designers to understand cognition processes of the users and, as a consequence, to improve the design of information visualizations. In this context, we also discuss the issue of categorization of user activities. Categorization helps researchers to generalize results and compare different information visualization tools.","PeriodicalId":235801,"journal":{"name":"Workshop on Beyond Time and Errors: Novel Evaluation Methods for Visualization","volume":"12 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2012-10-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122312068","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
With the growing need for visualization to aid users in understanding large, complex datasets, the ability for users to interact and explore these datasets is critical. As visual analytic systems have advanced to leverage powerful computational models and data analytics capabilities, the modes by which users engage and interact with the information are limited. Often, users are taxed with directly manipulating parameters of these models through traditional GUIs (e.g., using sliders to directly manipulate the value of a parameter). However, the purpose of user interaction in visual analytic systems is to enable visual data exploration -- where users can focus on their task, as opposed to the tool or system. As a result, users can engage freely in data exploration and decision-making, for the purpose of gaining insight. In this position paper, we discuss how evaluating visual analytic systems can be approached through user interaction analysis, where the goal is to minimize the cognitive translation between the visual metaphor and the mode of interaction (i.e., reducing the "interaction junk"). We motivate this concept through a discussion of traditional GUIs used in visual analytics for direct manipulation of model parameters, and the importance of designing interactions the support visual data exploration.
{"title":"Interaction junk: user interaction-based evaluation of visual analytic systems","authors":"A. Endert, Chris North","doi":"10.1145/2442576.2442583","DOIUrl":"https://doi.org/10.1145/2442576.2442583","url":null,"abstract":"With the growing need for visualization to aid users in understanding large, complex datasets, the ability for users to interact and explore these datasets is critical. As visual analytic systems have advanced to leverage powerful computational models and data analytics capabilities, the modes by which users engage and interact with the information are limited. Often, users are taxed with directly manipulating parameters of these models through traditional GUIs (e.g., using sliders to directly manipulate the value of a parameter). However, the purpose of user interaction in visual analytic systems is to enable visual data exploration -- where users can focus on their task, as opposed to the tool or system. As a result, users can engage freely in data exploration and decision-making, for the purpose of gaining insight. In this position paper, we discuss how evaluating visual analytic systems can be approached through user interaction analysis, where the goal is to minimize the cognitive translation between the visual metaphor and the mode of interaction (i.e., reducing the \"interaction junk\"). We motivate this concept through a discussion of traditional GUIs used in visual analytics for direct manipulation of model parameters, and the importance of designing interactions the support visual data exploration.","PeriodicalId":235801,"journal":{"name":"Workshop on Beyond Time and Errors: Novel Evaluation Methods for Visualization","volume":"138 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2012-10-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125749349","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
The position taken in this paper is that the availability of standardized questionnaires specifically developed for measuring users' perception of usability in evaluation studies in information visualization would provide the community with an excellent additional instrument. The need for such an instrument is evident for several important reasons. Pursuing the development, validation and use of questionnaires will add significantly to the evidence base necessary for the community to guide the production of high-quality visualization techniques, facilitate adoption by users, promote successful commercialization and guide future research tasks.
{"title":"Questionnaires for evaluation in information visualization","authors":"C. Forsell, M. Cooper","doi":"10.1145/2442576.2442592","DOIUrl":"https://doi.org/10.1145/2442576.2442592","url":null,"abstract":"The position taken in this paper is that the availability of standardized questionnaires specifically developed for measuring users' perception of usability in evaluation studies in information visualization would provide the community with an excellent additional instrument. The need for such an instrument is evident for several important reasons. Pursuing the development, validation and use of questionnaires will add significantly to the evidence base necessary for the community to guide the production of high-quality visualization techniques, facilitate adoption by users, promote successful commercialization and guide future research tasks.","PeriodicalId":235801,"journal":{"name":"Workshop on Beyond Time and Errors: Novel Evaluation Methods for Visualization","volume":"15 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2012-10-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130028115","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
A data set can be represented in any number of ways. For example, hierarchical data can be presented as a radial node-link diagram, dendrogram, force-directed layout, or tree map. Alternatively, point-observations can be shown with scatter-plots, parallel coordinates, or bar charts. Each technique has different capabilities for representing relationships. These capabilities are further modified by projection and presentation decisions within the technique category. Evaluating the many options is an essential task in visualization development. Currently, evaluation is largely based on heuristics, prior experience, and indefinable aesthetic considerations. This paper presents initial work towards an evaluation technique based in spatial autocorrelation. We find that spatial autocorrelation can be used to construct a separator between visualizations and other image types. Furthermore, this can be done with parameters amenable to interactive use and in a fashion that does not need to take plot schema characteristics as parameters.
{"title":"Spatial autocorrelation-based information visualization evaluation","authors":"Joseph A. Cottam, A. Lumsdaine","doi":"10.1145/2442576.2442584","DOIUrl":"https://doi.org/10.1145/2442576.2442584","url":null,"abstract":"A data set can be represented in any number of ways. For example, hierarchical data can be presented as a radial node-link diagram, dendrogram, force-directed layout, or tree map. Alternatively, point-observations can be shown with scatter-plots, parallel coordinates, or bar charts. Each technique has different capabilities for representing relationships. These capabilities are further modified by projection and presentation decisions within the technique category. Evaluating the many options is an essential task in visualization development. Currently, evaluation is largely based on heuristics, prior experience, and indefinable aesthetic considerations. This paper presents initial work towards an evaluation technique based in spatial autocorrelation. We find that spatial autocorrelation can be used to construct a separator between visualizations and other image types. Furthermore, this can be done with parameters amenable to interactive use and in a fashion that does not need to take plot schema characteristics as parameters.","PeriodicalId":235801,"journal":{"name":"Workshop on Beyond Time and Errors: Novel Evaluation Methods for Visualization","volume":"62 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2012-10-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129249263","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Evan M. Peck, B. Yuksel, Lane Harrison, Alvitta Ottley, Remco Chang
The effects of individual differences on user interaction is a topic that has been explored for the last 25 years in HCI. Recently, the importance of this subject has been carried into the field of information visualization and consequently, there has been a wide range of research conducted in this area. However, there has been no consensus on which evaluation methods best answer the unique needs of information visualization. In this position paper we propose that individual differences are evaluated in three dominant dimensions: cognitive traits, cognitive states and experience/bias. We believe that this is a first step in systematically evaluating the effects of users' individual differences on information visualization and visual analytics.
{"title":"Towards a 3-dimensional model of individual cognitive differences: position paper","authors":"Evan M. Peck, B. Yuksel, Lane Harrison, Alvitta Ottley, Remco Chang","doi":"10.1145/2442576.2442582","DOIUrl":"https://doi.org/10.1145/2442576.2442582","url":null,"abstract":"The effects of individual differences on user interaction is a topic that has been explored for the last 25 years in HCI. Recently, the importance of this subject has been carried into the field of information visualization and consequently, there has been a wide range of research conducted in this area. However, there has been no consensus on which evaluation methods best answer the unique needs of information visualization. In this position paper we propose that individual differences are evaluated in three dominant dimensions: cognitive traits, cognitive states and experience/bias. We believe that this is a first step in systematically evaluating the effects of users' individual differences on information visualization and visual analytics.","PeriodicalId":235801,"journal":{"name":"Workshop on Beyond Time and Errors: Novel Evaluation Methods for Visualization","volume":"7 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2012-10-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"117332547","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Involving analysts in visualisation design has obvious benefits, but the knowledge-gap between domain experts ('analysts') and visualisation designers ('designers') often makes the degree of their involvement fall short of that aspired. By promoting a culture of mutual learning, understanding and contribution between both analysts and designers from the outset, participants can be raised to a level at which all can usefully contribute to both requirement definition and design. We describe the process we use to do this for tightly-scoped and short design exercises -- with meetings/workshops, iterative bursts of design/prototyping over relatively short periods of time, and workplace-based evaluation -- illustrating this with examples of our own experience from recent work with bird ecologists.
{"title":"Experiences in involving analysts in visualisation design","authors":"A. Slingsby, J. Dykes","doi":"10.1145/2442576.2442577","DOIUrl":"https://doi.org/10.1145/2442576.2442577","url":null,"abstract":"Involving analysts in visualisation design has obvious benefits, but the knowledge-gap between domain experts ('analysts') and visualisation designers ('designers') often makes the degree of their involvement fall short of that aspired. By promoting a culture of mutual learning, understanding and contribution between both analysts and designers from the outset, participants can be raised to a level at which all can usefully contribute to both requirement definition and design. We describe the process we use to do this for tightly-scoped and short design exercises -- with meetings/workshops, iterative bursts of design/prototyping over relatively short periods of time, and workplace-based evaluation -- illustrating this with examples of our own experience from recent work with bird ecologists.","PeriodicalId":235801,"journal":{"name":"Workshop on Beyond Time and Errors: Novel Evaluation Methods for Visualization","volume":"46 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2012-10-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132380139","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Crowdsourcing-based user studies have become increasingly popular in information visualization (InfoVis) and visual analytics (VA). However, it is still unclear how to deal with some undesired crowdsourcing workers, especially those who submit random responses simply to gain wages (random clickers, henceforth). In order to mitigate the impacts of random clickers, several studies simply exclude outliers, but this approach has a potential risk of losing data from participants whose performances are extreme even though they participated faithfully. In this paper, we evaluated the degree of randomness in responses from a crowdsourcing worker to infer whether the worker is a random clicker. Thus, we could reliably filter out random clickers and found that resulting data from crowdsourcing-based user studies were comparable with those of a controlled lab study. We also tested three representative reward schemes (piece-rate, quota, and punishment schemes) with four different levels of compensations ($0.00, $0.20, $1.00, and $4.00) on a crowdsourcing platform with a total of 1,500 crowdsourcing workers to investigate the influences that different payment conditions have on the number of random clickers. The results show that higher compensations decrease the proportion of random clickers, but such increase in participation quality cannot justify the associated additional costs. A detailed discussion on how to optimize the payment scheme and amount to obtain high-quality data economically is provided.
{"title":"How to filter out random clickers in a crowdsourcing-based study?","authors":"Sung-Hee Kim, Hyokun Yun, Ji Soo Yi","doi":"10.1145/2442576.2442591","DOIUrl":"https://doi.org/10.1145/2442576.2442591","url":null,"abstract":"Crowdsourcing-based user studies have become increasingly popular in information visualization (InfoVis) and visual analytics (VA). However, it is still unclear how to deal with some undesired crowdsourcing workers, especially those who submit random responses simply to gain wages (random clickers, henceforth). In order to mitigate the impacts of random clickers, several studies simply exclude outliers, but this approach has a potential risk of losing data from participants whose performances are extreme even though they participated faithfully. In this paper, we evaluated the degree of randomness in responses from a crowdsourcing worker to infer whether the worker is a random clicker. Thus, we could reliably filter out random clickers and found that resulting data from crowdsourcing-based user studies were comparable with those of a controlled lab study. We also tested three representative reward schemes (piece-rate, quota, and punishment schemes) with four different levels of compensations ($0.00, $0.20, $1.00, and $4.00) on a crowdsourcing platform with a total of 1,500 crowdsourcing workers to investigate the influences that different payment conditions have on the number of random clickers. The results show that higher compensations decrease the proportion of random clickers, but such increase in participation quality cannot justify the associated additional costs. A detailed discussion on how to optimize the payment scheme and amount to obtain high-quality data economically is provided.","PeriodicalId":235801,"journal":{"name":"Workshop on Beyond Time and Errors: Novel Evaluation Methods for Visualization","volume":"93 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2012-10-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132899754","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
In this position paper, we discuss the problems and advantages of using physiological measurements to to estimate cognitive load in order to evaluate scientific visualization methods. We will present various techniques and technologies designed to measure cognitive load and how they may be leveraged in the context of user evaluation studies for scientific visualization. We also discuss the challenges of experiments designed to use these physiological measurements.
{"title":"Evaluating visualization using cognitive measures","authors":"Erik W. Anderson","doi":"10.1145/2442576.2442581","DOIUrl":"https://doi.org/10.1145/2442576.2442581","url":null,"abstract":"In this position paper, we discuss the problems and advantages of using physiological measurements to to estimate cognitive load in order to evaluate scientific visualization methods. We will present various techniques and technologies designed to measure cognitive load and how they may be leveraged in the context of user evaluation studies for scientific visualization. We also discuss the challenges of experiments designed to use these physiological measurements.","PeriodicalId":235801,"journal":{"name":"Workshop on Beyond Time and Errors: Novel Evaluation Methods for Visualization","volume":"14 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2012-10-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128829584","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Isabel Cristina Siqueira da Silva, C. Freitas, G. Santucci
Visualization of ontologies is based on effective graphical representations and interaction techniques that support users tasks related to different entities and aspects. Ontologies can be very large and complex due to the levels of classes hierarchy as well as the attributes. In this work we present a study involving the investigation and proposition of guidelines to help the design and evaluation of visualization techniques for both the intensional and extensional levels of ontologies.
{"title":"An integrated approach for evaluating the visualization of intensional and extensional levels of ontologies","authors":"Isabel Cristina Siqueira da Silva, C. Freitas, G. Santucci","doi":"10.1145/2442576.2442578","DOIUrl":"https://doi.org/10.1145/2442576.2442578","url":null,"abstract":"Visualization of ontologies is based on effective graphical representations and interaction techniques that support users tasks related to different entities and aspects. Ontologies can be very large and complex due to the levels of classes hierarchy as well as the attributes. In this work we present a study involving the investigation and proposition of guidelines to help the design and evaluation of visualization techniques for both the intensional and extensional levels of ontologies.","PeriodicalId":235801,"journal":{"name":"Workshop on Beyond Time and Errors: Novel Evaluation Methods for Visualization","volume":"137 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2012-10-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115262712","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Linda T. Kaastra, Richard Arias-Hernández, Brian D. Fisher
In this position paper we propose a performance science approach to evaluation of visual analytics systems.
在这篇立场论文中，我们提出了一种评估视觉分析系统的性能科学方法。
{"title":"Evaluating analytic performance","authors":"Linda T. Kaastra, Richard Arias-Hernández, Brian D. Fisher","doi":"10.1145/2442576.2442590","DOIUrl":"https://doi.org/10.1145/2442576.2442590","url":null,"abstract":"In this position paper we propose a performance science approach to evaluation of visual analytics systems.","PeriodicalId":235801,"journal":{"name":"Workshop on Beyond Time and Errors: Novel Evaluation Methods for Visualization","volume":"90 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2012-10-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133747115","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：