R. Bond, A. Moorhead, M. Mulvenna, S. O’neill, C. Potts, Nuala Murphy
Behavioural data analytics and user log analysis can be useful to gain insight into how users interact with technologies. In this study, data analytics were conducted on maternal mental health data generated from the Moment Health app to address the question: What is the temporal behaviour of users when completing ecological momentary assessments (EMA) on a mental health app, with EMAs in the form of full mental health scales versus EMAs in the form of mood logs? The Health Interaction Log Data Analytics (HILDA) pipeline was used to analyse 1,461 users of the app. More users completed single mood logs EMAs (n=6,993) compared to scaled EMAs (n=2,129). Distinct temporal patterns were identified, with more users willing to log moods at 9am and 12pm as opposed to completing a scale. The most common hours for users to complete scaled EMAs are between 8pm and 10pm. The least number of mood logs and scale completions take place on Saturday. Whilst happiness is the dominant mood during day times, anxiety and sadness peak during the night at 1am and 4am respectively. The data indicates that postnatal depression decreases over time for some users (r = -0.23, p-value < 0.01). The overall finding from this work are that users prefer simple EMA approaches and that the temporal behavior of users engaging with the two forms of EMA are distinctly different.
{"title":"Behaviour Analytics of Users Completing Ecological Momentary Assessments in the Form of Mental Health Scales and Mood Logs on a Smartphone App","authors":"R. Bond, A. Moorhead, M. Mulvenna, S. O’neill, C. Potts, Nuala Murphy","doi":"10.1145/3335082.3335111","DOIUrl":"https://doi.org/10.1145/3335082.3335111","url":null,"abstract":"Behavioural data analytics and user log analysis can be useful to gain insight into how users interact with technologies. In this study, data analytics were conducted on maternal mental health data generated from the Moment Health app to address the question: What is the temporal behaviour of users when completing ecological momentary assessments (EMA) on a mental health app, with EMAs in the form of full mental health scales versus EMAs in the form of mood logs? The Health Interaction Log Data Analytics (HILDA) pipeline was used to analyse 1,461 users of the app. More users completed single mood logs EMAs (n=6,993) compared to scaled EMAs (n=2,129). Distinct temporal patterns were identified, with more users willing to log moods at 9am and 12pm as opposed to completing a scale. The most common hours for users to complete scaled EMAs are between 8pm and 10pm. The least number of mood logs and scale completions take place on Saturday. Whilst happiness is the dominant mood during day times, anxiety and sadness peak during the night at 1am and 4am respectively. The data indicates that postnatal depression decreases over time for some users (r = -0.23, p-value < 0.01). The overall finding from this work are that users prefer simple EMA approaches and that the temporal behavior of users engaging with the two forms of EMA are distinctly different.","PeriodicalId":279162,"journal":{"name":"Proceedings of the 31st European Conference on Cognitive Ergonomics","volume":"18 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-09-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126036819","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Dianne M. Murray, G. Veer, G. D. Haan, Anke Dittmar
The goal of this one-day workshop will be to revisit what is meant by Cognitive Ergonomics by bringing together researchers and practitioners who have an interest in carrying forward CE as a meaningful discipline.
{"title":"Rethinking Cognitive Ergonomics","authors":"Dianne M. Murray, G. Veer, G. D. Haan, Anke Dittmar","doi":"10.1145/3335082.3335677","DOIUrl":"https://doi.org/10.1145/3335082.3335677","url":null,"abstract":"The goal of this one-day workshop will be to revisit what is meant by Cognitive Ergonomics by bringing together researchers and practitioners who have an interest in carrying forward CE as a meaningful discipline.","PeriodicalId":279162,"journal":{"name":"Proceedings of the 31st European Conference on Cognitive Ergonomics","volume":"9 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-09-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115068905","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Aleeha Iftikhar, R. Bond, V. Mcgilligan, S. Leslie, K. Rjoob, C. Knoery, A. Peace
The purpose of this study is to provide a comprehensive overview of how dashboards are contributing in aiding and improving the quality of decision making in healthcare. A literature search was performed for the related papers on PubMed, Web-science, IEEE Xplore Digital Library and ACM Digital Library. Both citation and hand searches were conducted to find relevant papers. Sixty-two full text papers were retrieved of which eight papers were included in the review. Significant heterogeneity was found in the process, outcomes analysis methods and techniques. There was considerable evidence that the use of data visualization dashboards in healthcare improves the quality of decision making in healthcare.
{"title":"Role of dashboards in improving decision making in healthcare: Review of the literature","authors":"Aleeha Iftikhar, R. Bond, V. Mcgilligan, S. Leslie, K. Rjoob, C. Knoery, A. Peace","doi":"10.1145/3335082.3335109","DOIUrl":"https://doi.org/10.1145/3335082.3335109","url":null,"abstract":"The purpose of this study is to provide a comprehensive overview of how dashboards are contributing in aiding and improving the quality of decision making in healthcare. A literature search was performed for the related papers on PubMed, Web-science, IEEE Xplore Digital Library and ACM Digital Library. Both citation and hand searches were conducted to find relevant papers. Sixty-two full text papers were retrieved of which eight papers were included in the review. Significant heterogeneity was found in the process, outcomes analysis methods and techniques. There was considerable evidence that the use of data visualization dashboards in healthcare improves the quality of decision making in healthcare.","PeriodicalId":279162,"journal":{"name":"Proceedings of the 31st European Conference on Cognitive Ergonomics","volume":"72 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-09-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123977496","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
The Intelligent Voice SmartTranscript is an interactive HTML5 document that contains the audio, a speech transcription and the key topics from an audio recording. It is designed to enable a quick and efficient review of audio communications by encapsulating the recording with the speech transcript and topics within a single HTML5 file. This paper outlines the rationale for the design of the SmartTranscript user experience. The paper discusses the difficulties of audio review, how there is large potential for misinterpretation associated with reviewing transcripts in isolation, and how additional diarization and topic tagging components augment the audio review process.
{"title":"Smart Transcription","authors":"C. Glackin, Nazim Dugan, Nigel Cannings, J. Wall","doi":"10.1145/3335082.3335114","DOIUrl":"https://doi.org/10.1145/3335082.3335114","url":null,"abstract":"The Intelligent Voice SmartTranscript is an interactive HTML5 document that contains the audio, a speech transcription and the key topics from an audio recording. It is designed to enable a quick and efficient review of audio communications by encapsulating the recording with the speech transcript and topics within a single HTML5 file. This paper outlines the rationale for the design of the SmartTranscript user experience. The paper discusses the difficulties of audio review, how there is large potential for misinterpretation associated with reviewing transcripts in isolation, and how additional diarization and topic tagging components augment the audio review process.","PeriodicalId":279162,"journal":{"name":"Proceedings of the 31st European Conference on Cognitive Ergonomics","volume":"80 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-09-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121292824","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Natalie Clewley, L. Dodd, Victoria Smy, Annamaria Witheridge, P. Louvieris
Purpose: To determine the elicitation methodologies best placed to uncover and capture the expert operator’s reflective cognitive judgements in complex and dynamic military operating environments (e.g., explosive ordinance disposal) in order to develop the specification for a reflective eXplainable Artificial Intelligence (XAI) agent to support the training of domain novices. Approach: A bounded literature review of the latest developments in expert knowledge elicitation was undertaken to determine the ’art-of-the-possible’ in respects to uncovering an expert’s cognitive judgements in complex and dynamic environments. Candidate methodologies were systematically and critically reviewed in order to identify the most promising methodologies for uncovering expert situational awareness and metacognitive evaluations in pursuit of actionable threat mitigation strategies in high-risk contexts. Research outputs are synthesized into an interview protocol for eliciting and understanding the in-situ actions and decisions of experts in high-risk, complex operating environments. Practical implications: Trainees entering high-risk operating environments can benefit from exposure to expert reflective strategies whilst learning the trade. Typical operator training focuses on technical aspects of threat mitigation but often overlooks reflective self-evaluation. The present study represents an initial step towards determining the feasibility of designing a reflective XAI agent to augment the performance of trainees entering high-risk operations. Outputs of the expert knowledge elicitation protocol documented here shall be used to refine a theoretical framework of expert operator judgement, in order to determine decision support strategies of benefit to domain novices.
{"title":"Eliciting Expert Knowledge to Inform Training Design","authors":"Natalie Clewley, L. Dodd, Victoria Smy, Annamaria Witheridge, P. Louvieris","doi":"10.1145/3335082.3335091","DOIUrl":"https://doi.org/10.1145/3335082.3335091","url":null,"abstract":"Purpose: To determine the elicitation methodologies best placed to uncover and capture the expert operator’s reflective cognitive judgements in complex and dynamic military operating environments (e.g., explosive ordinance disposal) in order to develop the specification for a reflective eXplainable Artificial Intelligence (XAI) agent to support the training of domain novices. Approach: A bounded literature review of the latest developments in expert knowledge elicitation was undertaken to determine the ’art-of-the-possible’ in respects to uncovering an expert’s cognitive judgements in complex and dynamic environments. Candidate methodologies were systematically and critically reviewed in order to identify the most promising methodologies for uncovering expert situational awareness and metacognitive evaluations in pursuit of actionable threat mitigation strategies in high-risk contexts. Research outputs are synthesized into an interview protocol for eliciting and understanding the in-situ actions and decisions of experts in high-risk, complex operating environments. Practical implications: Trainees entering high-risk operating environments can benefit from exposure to expert reflective strategies whilst learning the trade. Typical operator training focuses on technical aspects of threat mitigation but often overlooks reflective self-evaluation. The present study represents an initial step towards determining the feasibility of designing a reflective XAI agent to augment the performance of trainees entering high-risk operations. Outputs of the expert knowledge elicitation protocol documented here shall be used to refine a theoretical framework of expert operator judgement, in order to determine decision support strategies of benefit to domain novices.","PeriodicalId":279162,"journal":{"name":"Proceedings of the 31st European Conference on Cognitive Ergonomics","volume":"181 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-09-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121501646","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Jonathan Gay, V. Simms, R. Bond, D. Finlay, H. Purchase
Visuocognitive design accommodates the alignment of visualization to human cognitive processes. Established theory suggests that 1) recognition is easier than recall [1], 2) spatial visualizations are less abstract than temporal ones [2], and 3) aesthetics induce cognitive ease [3]. These principles, and others, underpin our new audit tool that focusses on design for cognition. Theories of form, function and utility have been known for many decades and are well-known in the field of design, but infovisualization is a relatively new field, as are associated fields such as user-experience (UX), user-centered design and information design. Therefore, generally, design schools focus far more (possibly, exclusively) on teaching form, style, function, sustainability and user-experience than on visuocognition. The same emphasis is found in the design industry. This audit tool has been created to provide heuristic evaluations based on a set of visuocognitive design principles and is, therefore, a valuable contribution. To devise the visuocognitive principles, we conducted a narrative review as a method of approach. The tool is composed of one prerequisite and six principles. ‘Informed Engagement’ is the prerequisite to accurately inform the graphics with ground truth, and to give them substance. The six principles are: 1) clarity, 2) arrangement, 3) cued meaning, 4) intuitive meaning, 5) cognitive fit, and 6) cognitive preference. They are divided into three groups: the first two principles concern appearance, the second two principles concern meaning, and the last two principles concern cognition (Figure 1). The term ‘meaning’ can imply intended meaning by the designer (in a graphic representation), or construed meaning by the user. The novelty of this audit tool is that it fixes ‘meaning’ as the pivotal point between aesthetic visual display and mental cognition, with the aim to align construed meaning with intended meaning and achieve fluent cognition.
{"title":"An Audit Tool for Assessing the Visuocognitive Design of Infographics","authors":"Jonathan Gay, V. Simms, R. Bond, D. Finlay, H. Purchase","doi":"10.1145/3335082.3335117","DOIUrl":"https://doi.org/10.1145/3335082.3335117","url":null,"abstract":"Visuocognitive design accommodates the alignment of visualization to human cognitive processes. Established theory suggests that 1) recognition is easier than recall [1], 2) spatial visualizations are less abstract than temporal ones [2], and 3) aesthetics induce cognitive ease [3]. These principles, and others, underpin our new audit tool that focusses on design for cognition. Theories of form, function and utility have been known for many decades and are well-known in the field of design, but infovisualization is a relatively new field, as are associated fields such as user-experience (UX), user-centered design and information design. Therefore, generally, design schools focus far more (possibly, exclusively) on teaching form, style, function, sustainability and user-experience than on visuocognition. The same emphasis is found in the design industry. This audit tool has been created to provide heuristic evaluations based on a set of visuocognitive design principles and is, therefore, a valuable contribution. To devise the visuocognitive principles, we conducted a narrative review as a method of approach. The tool is composed of one prerequisite and six principles. ‘Informed Engagement’ is the prerequisite to accurately inform the graphics with ground truth, and to give them substance. The six principles are: 1) clarity, 2) arrangement, 3) cued meaning, 4) intuitive meaning, 5) cognitive fit, and 6) cognitive preference. They are divided into three groups: the first two principles concern appearance, the second two principles concern meaning, and the last two principles concern cognition (Figure 1). The term ‘meaning’ can imply intended meaning by the designer (in a graphic representation), or construed meaning by the user. The novelty of this audit tool is that it fixes ‘meaning’ as the pivotal point between aesthetic visual display and mental cognition, with the aim to align construed meaning with intended meaning and achieve fluent cognition.","PeriodicalId":279162,"journal":{"name":"Proceedings of the 31st European Conference on Cognitive Ergonomics","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-09-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131203607","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
The purpose of this empirically informed, but mainly conceptual paper, is to understand what we might mean by the word ‘structure’ in cross-media design. The paper draws upon a workplace study in print and online news production at a Swedish local news publisher, where we observed the work of reporters, page planners and web editors. Structure in the context of cross-media design and production is initially defined as the pattern of arrangement of elements in the media. We identified three kinds of structure in our observations of the journalists’ work: category, time, and space. Category: how knowledge is ordered is foundational and explains how functions and content relate to each other. Time: the temporal structure is how functions and content are ordered in time (what comes before and after). Space: The spatial structure is the layout of functions and content in 2-dimensional or 3-dimensional space. These three kinds of structure have corresponding design representations in interaction design: concept maps, flow charts, and wireframes.
{"title":"Category, Time, and Space: Structures in Cross-Media Design and Production","authors":"Mattias Arvola, Mathias Broth","doi":"10.1145/3335082.3335107","DOIUrl":"https://doi.org/10.1145/3335082.3335107","url":null,"abstract":"The purpose of this empirically informed, but mainly conceptual paper, is to understand what we might mean by the word ‘structure’ in cross-media design. The paper draws upon a workplace study in print and online news production at a Swedish local news publisher, where we observed the work of reporters, page planners and web editors. Structure in the context of cross-media design and production is initially defined as the pattern of arrangement of elements in the media. We identified three kinds of structure in our observations of the journalists’ work: category, time, and space. Category: how knowledge is ordered is foundational and explains how functions and content relate to each other. Time: the temporal structure is how functions and content are ordered in time (what comes before and after). Space: The spatial structure is the layout of functions and content in 2-dimensional or 3-dimensional space. These three kinds of structure have corresponding design representations in interaction design: concept maps, flow charts, and wireframes.","PeriodicalId":279162,"journal":{"name":"Proceedings of the 31st European Conference on Cognitive Ergonomics","volume":"46 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-09-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115008384","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Guenter Wallner, M. Pohl, T. V. Landesberger, K. Ballweg
Visual comparison of graphs has become widespread in various disciplines. To support this task, it is necessary to generate graph drawings that facilitate the perception of differences. However, while readability of single graphs has been widely investigated, knowledge about the recognition of differences in graphs in general and directed acyclic graphs (DAGs) specifically is quite limited. Towards this end, we conducted an exploratory qualitative study to ascertain which factors influence the perception of changes in DAGs and to better understand the strategies adopted to compare them. Specifically, we investigated pairs of DAGs of varying structure and size (about 50 to 100 nodes) and displayed side-by-side. Based on video recordings and thinking aloud protocols of 22 participants we identified eight factors including edge crossings, shape, symmetry, white space, and density to affect the perception of differences. In addition, we identified three strategies which were used to compare the graphs: layer-by-layer, hierarchical, and a mixture of these two. These strategies were adapted flexibly by the participants based on the graph structure.
{"title":"Perception of Differences in Directed Acyclic Graphs: Influence Factors & Cognitive Strategies","authors":"Guenter Wallner, M. Pohl, T. V. Landesberger, K. Ballweg","doi":"10.1145/3335082.3335083","DOIUrl":"https://doi.org/10.1145/3335082.3335083","url":null,"abstract":"Visual comparison of graphs has become widespread in various disciplines. To support this task, it is necessary to generate graph drawings that facilitate the perception of differences. However, while readability of single graphs has been widely investigated, knowledge about the recognition of differences in graphs in general and directed acyclic graphs (DAGs) specifically is quite limited. Towards this end, we conducted an exploratory qualitative study to ascertain which factors influence the perception of changes in DAGs and to better understand the strategies adopted to compare them. Specifically, we investigated pairs of DAGs of varying structure and size (about 50 to 100 nodes) and displayed side-by-side. Based on video recordings and thinking aloud protocols of 22 participants we identified eight factors including edge crossings, shape, symmetry, white space, and density to affect the perception of differences. In addition, we identified three strategies which were used to compare the graphs: layer-by-layer, hierarchical, and a mixture of these two. These strategies were adapted flexibly by the participants based on the graph structure.","PeriodicalId":279162,"journal":{"name":"Proceedings of the 31st European Conference on Cognitive Ergonomics","volume":"71 5 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-09-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129655422","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Samuel Holmes, A. Moorhead, R. Bond, Huiru Zheng, V. Coates, M. McTear
Chatbots are becoming increasingly popular as a human-computer interface. The traditional best practices normally applied to User Experience (UX) design cannot easily be applied to chatbots, nor can conventional usability testing techniques guarantee accuracy. WeightMentor is a bespoke self-help motivational tool for weight loss maintenance. This study addresses the following four research questions: How usable is the WeightMentor chatbot, according to conventional usability methods?; To what extend will different conventional usability questionnaires correlate when evaluating chatbot usability?; And how do they correlate to a tailored chatbot usability survey score?; What is the optimum number of users required to identify chatbot usability issues?; How many task repetitions are required for a first-time chatbot users to reach optimum task performance (i.e. efficiency based on task completion times)? This paper describes the procedure for testing the WeightMentor chatbot, assesses correlation between typical usability testing metrics, and suggests that conventional wisdom on participant numbers for identifying usability issues may not apply to chatbots. The study design was a usability study. WeightMentor was tested using a pre-determined usability testing protocol, evaluating ease of task completion, unique usability errors and participant opinions on the chatbot (collected using usability questionnaires). WeightMentor usability scores were generally high, and correlation between questionnaires was strong. The optimum number of users for identifying chatbot usability errors was 26, which challenges previous research. Chatbot users reached optimum proficiency in tasks after just one repetition. Usability test outcomes confirm what is already known about chatbots - that they are highly usable (due to their simple interface and conversation-driven functionality) but conventional methods for assessing usability and user experience may not be as accurate when applied to chatbots.
{"title":"Usability testing of a healthcare chatbot: Can we use conventional methods to assess conversational user interfaces?","authors":"Samuel Holmes, A. Moorhead, R. Bond, Huiru Zheng, V. Coates, M. McTear","doi":"10.1145/3335082.3335094","DOIUrl":"https://doi.org/10.1145/3335082.3335094","url":null,"abstract":"Chatbots are becoming increasingly popular as a human-computer interface. The traditional best practices normally applied to User Experience (UX) design cannot easily be applied to chatbots, nor can conventional usability testing techniques guarantee accuracy. WeightMentor is a bespoke self-help motivational tool for weight loss maintenance. This study addresses the following four research questions: How usable is the WeightMentor chatbot, according to conventional usability methods?; To what extend will different conventional usability questionnaires correlate when evaluating chatbot usability?; And how do they correlate to a tailored chatbot usability survey score?; What is the optimum number of users required to identify chatbot usability issues?; How many task repetitions are required for a first-time chatbot users to reach optimum task performance (i.e. efficiency based on task completion times)? This paper describes the procedure for testing the WeightMentor chatbot, assesses correlation between typical usability testing metrics, and suggests that conventional wisdom on participant numbers for identifying usability issues may not apply to chatbots. The study design was a usability study. WeightMentor was tested using a pre-determined usability testing protocol, evaluating ease of task completion, unique usability errors and participant opinions on the chatbot (collected using usability questionnaires). WeightMentor usability scores were generally high, and correlation between questionnaires was strong. The optimum number of users for identifying chatbot usability errors was 26, which challenges previous research. Chatbot users reached optimum proficiency in tasks after just one repetition. Usability test outcomes confirm what is already known about chatbots - that they are highly usable (due to their simple interface and conversation-driven functionality) but conventional methods for assessing usability and user experience may not be as accurate when applied to chatbots.","PeriodicalId":279162,"journal":{"name":"Proceedings of the 31st European Conference on Cognitive Ergonomics","volume":"24 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-09-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127042506","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Oluwademilade Amos-Oluwole, Benjamin Subhani, Harry L. Claxton, Daisy C. Holmes, Carina E. I. Westling, H. Witchel
INTRODUCTION: Mind wandering is often an interposing mental state in which a person has thoughts unrelated to the primary task they are performing. It leads to diminished performance and error risk on the primary task. Recently mind wandering being proposed as comprising two different states: deliberate/intentional and spontaneous/unintentional. OBJECTIVE: We hypothesise that forced engagement via persistent compliant activity may both increase responsiveness and inhibit non-instrumental activities including deliberate mind wandering. METHODS: Two versions of a sustained attention task (Test of Variables of Attention, ToVA, a go/no-go task) were presented to twenty-eight seated, healthy participants (ages 19-35, 19 female): one with infrequent targets (25%) and another with frequent targets (75%, requiring increased compliant activity). Mind wandering was assessed by thought probes, and thought probe durations were measured using Superlab with an RB530 interaction device. RESULTS: Increased compliant activity decreased deliberate mind wandering while increasing spontaneous mind wandering. Thought probe durations were significantly shortened by increasing target frequency (and by being on-task), while deliberate and spontaneous mind wandering elicited equivalent thought probe durations. CONCLUSION: Responsive compliant activity is a useful tool to interrogate mind wandering. Compliant activity differentially affects deliberate and spontaneous mind wandering. This strongly supports theories that discriminate these two types of mind wandering.
{"title":"Compliant activity inhibits deliberate mind wandering and accelerates thought probe responsiveness compared to compliant inactivity","authors":"Oluwademilade Amos-Oluwole, Benjamin Subhani, Harry L. Claxton, Daisy C. Holmes, Carina E. I. Westling, H. Witchel","doi":"10.1145/3335082.3335115","DOIUrl":"https://doi.org/10.1145/3335082.3335115","url":null,"abstract":"INTRODUCTION: Mind wandering is often an interposing mental state in which a person has thoughts unrelated to the primary task they are performing. It leads to diminished performance and error risk on the primary task. Recently mind wandering being proposed as comprising two different states: deliberate/intentional and spontaneous/unintentional. OBJECTIVE: We hypothesise that forced engagement via persistent compliant activity may both increase responsiveness and inhibit non-instrumental activities including deliberate mind wandering. METHODS: Two versions of a sustained attention task (Test of Variables of Attention, ToVA, a go/no-go task) were presented to twenty-eight seated, healthy participants (ages 19-35, 19 female): one with infrequent targets (25%) and another with frequent targets (75%, requiring increased compliant activity). Mind wandering was assessed by thought probes, and thought probe durations were measured using Superlab with an RB530 interaction device. RESULTS: Increased compliant activity decreased deliberate mind wandering while increasing spontaneous mind wandering. Thought probe durations were significantly shortened by increasing target frequency (and by being on-task), while deliberate and spontaneous mind wandering elicited equivalent thought probe durations. CONCLUSION: Responsive compliant activity is a useful tool to interrogate mind wandering. Compliant activity differentially affects deliberate and spontaneous mind wandering. This strongly supports theories that discriminate these two types of mind wandering.","PeriodicalId":279162,"journal":{"name":"Proceedings of the 31st European Conference on Cognitive Ergonomics","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-09-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129143167","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：