Which comes first, usability or utility?

Georges Grinstein Questions often asked when presenting some new model, new theory, new research or new visualization include: How useful or how usable is it? and Have you performed any tests? Visualization is an interface technology and as such includes not just software algorithms and techniques, but computer human interaction issues as well. This makes it draw from both areas, one appearing more focused on utility and the other on usability. One key step in the development of a new theory is the attempt to first solve a problem. That problem or question does not include a usability section; in some domain it may not even contain a utility one from most people's perspectives. We discuss both sides of the issue to clarify the role of each in the development of new visualization technologies. Position Statement Alfred Kobsa In the HCI literature one can find studies which conclude that ease of use is more important than usefulness (Hubona & Blanton 1996), that the opposite holds true (Liao and Landry 2000), and that ease of use is more important for females while the usefulness is more important for males (Yuen and Ma 2002). In this panel contribution, we will present several user studies with information visualization systems, ranging from lab experiments with closed questions to longitudinal adoption studies with administrative data analysts (Gonzales & Kobsa, 2003; Kobsa 2001, 2003; Mark et al. 2003). -------------------------------------------a e-mail: grinstein@cs.uml.edu e-mail: kobsa@uci.edu e-mail: plaisant@cs.umd.edu e-mail ben@cs.umd.edu e-mail statsko@cc.gatech.edu Consistent with general HCI research, the results show that both factors are important in certain situations, but do not indicate a clear superiority of one factor over the other. Position Statement Catherine Plaisant Is an airplane a better vehicle than a Jaguar, a mountain bicycle or a kid scooter? It all depends of where you need to go, what your goal for the travel is, how old you are, what terrain you will encounter on the way, how long you can spend learning, and many other parameters. All those vehicles are fairly usable but they all require training except for adults using the scooter, and their utility varies enormously as a function of the task and the user. The average car drivers benefit from years of human factor engineering and a large amount of standardization, allowing them to switch from a pickup truck to a convertible in a snap. Similarly, the success of a visualization tool depends on how well it fits the needs of the users it attempts to serve, and the tasks they want to accomplish. If utility may come first for an expert tool (e.g. for discovery tasks requires days of data examination and manipulation), usability has to come first in public access information systems that requires "immediate usability" (e.g. interactive displays of census statistics) otherwise users will walk away frustrated. Usability design principles imply that designers and evaluators understand the needs of users to decide which one of the two utility or usability comes first, and to set levels of required utility and usability. Utility and usability are both attainable goals that make each other stronger. Like others, we at the University of Maryland have been developing visualization techniques and have struggled over the years to find the best way to evaluate their benefits. Many evaluations have been controlled experiments and we have found that the most useful evaluations were multi-faceted, including qualitative and quantitative measurements or performance, preference and learnability, and I will be show examples from our research. Often we also find that the observations gathered during the experiment can be as informative as the collected measurements. More recently we have been promoting the development of benchmark datasets and tasks that will allow better comparisons between tools and techniques. We have been involved in the 1st InfoVis contest, which calls for the submission of case studies of pairwise comparison of trees. Three pairs of datasets were provided: philogenies, classifications and file system usage data, 605 Proceedings of the 14th IEEE Visualization Conference (VIS’03) 0-7695-2030-8/03 $ 17.00 © 2003 IEEE and open ended tasks described. Accepted submissions will seed an online repository that can be enhanced over time with additional datasets, tasks, case studies and controlled experiment results. I will report on the results of the contest and reflect on how evaluation repository and benchmark datasets might help us understand how to judge the usability and utility of our tools. The contest is at: http://www.cs.umd.edu/hcil/iv03contest/ Position Statement Ben Shneiderman The answer to the question of usability or utility first depends on your definition of usability. For me, usability is more than the color of widgets and placement of text. Usability is about understanding, stating, and serving user needs. Since these needs are the requirements that shape the tool, they determine the utility. The design of excellent tools depends upon understanding how they will be used; therefore usability is a pre-requisite for successful utility. Position Statement John T. Stasko If the question is, "Which comes first, utility or usability?" my answer is "yes". Both notions are vitally important in the development of information visualization techniques and systems, and they are just two sides of the same coin. When the field of information visualization formed and first grew, the computer graphics and visualization aspects dominated. More recently, an increasing emphasis on the HCI aspects of the field has emerged as we strive to better understand how people can truly benefit from our ideas. When an information visualization technique is implemented in a system, the usability component is crucial. Poor interface design can hamper adoption and cloud the utility benefits that may be possible. In information visualization, usability typically does not equate with the common notion of a system being easy to learn. Information visualization systems are complex and they often will be used extensively for long periods of time. Consequently, making a system efficient and natural to use, making functions and operations visible, and simply paying attention to the user interface are key components. This is especially true in systems where interaction and multiple views are essential. For any information visualization technique to be adopted beyond the initial idea generation, there must be some utility or value in that technique. Our field is not about making pretty pictures. It is about helping people with the complex tasks involved in data analysis and understanding. We need to do a better job of articulating the cognitive tasks that occur in data analysis (location, correlation, emphasis, association, etc.) and articulating how information visualization techniques and systems can help with these tasks. Ultimately, a kind of natural selection will occur: systems with utility and value will be adopted and used, while others will quietly fade away.