Mario Amrehn, S. Steidl, Reinier Kortekaas, Maddalena Strumia, M. Weingarten, M. Kowarschik, A. Maier
{"title":"A Semi-Automated Usability Evaluation Framework for Interactive Image Segmentation Systems","authors":"Mario Amrehn, S. Steidl, Reinier Kortekaas, Maddalena Strumia, M. Weingarten, M. Kowarschik, A. Maier","doi":"10.1155/2019/1464592","DOIUrl":null,"url":null,"abstract":"For complex segmentation tasks, the achievable accuracy of fully automated systems is inherently limited. Specifically, when a precise segmentation result is desired for a small amount of given data sets, semi-automatic methods exhibit a clear benefit for the user. The optimization of human computer interaction (HCI) is an essential part of interactive image segmentation. Nevertheless, publications introducing novel interactive segmentation systems (ISS) often lack an objective comparison of HCI aspects. It is demonstrated that even when the underlying segmentation algorithm is the same throughout interactive prototypes, their user experience may vary substantially. As a result, users prefer simple interfaces as well as a considerable degree of freedom to control each iterative step of the segmentation. In this article, an objective method for the comparison of ISS is proposed, based on extensive user studies. A summative qualitative content analysis is conducted via abstraction of visual and verbal feedback given by the participants. A direct assessment of the segmentation system is executed by the users via the system usability scale (SUS) and AttrakDiff-2 questionnaires. Furthermore, an approximation of the findings regarding usability aspects in those studies is introduced, conducted solely from the system-measurable user actions during their usage of interactive segmentation prototypes. The prediction of all questionnaire results has an average relative error of 8.9%, which is close to the expected precision of the questionnaire results themselves. This automated evaluation scheme may significantly reduce the resources necessary to investigate each variation of a prototype's user interface (UI) features and segmentation methodologies.","PeriodicalId":47063,"journal":{"name":"International Journal of Biomedical Imaging","volume":null,"pages":null},"PeriodicalIF":3.3000,"publicationDate":"2019-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1155/2019/1464592","citationCount":"6","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"International Journal of Biomedical Imaging","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1155/2019/1464592","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"ENGINEERING, BIOMEDICAL","Score":null,"Total":0}
引用次数: 6
Abstract
For complex segmentation tasks, the achievable accuracy of fully automated systems is inherently limited. Specifically, when a precise segmentation result is desired for a small amount of given data sets, semi-automatic methods exhibit a clear benefit for the user. The optimization of human computer interaction (HCI) is an essential part of interactive image segmentation. Nevertheless, publications introducing novel interactive segmentation systems (ISS) often lack an objective comparison of HCI aspects. It is demonstrated that even when the underlying segmentation algorithm is the same throughout interactive prototypes, their user experience may vary substantially. As a result, users prefer simple interfaces as well as a considerable degree of freedom to control each iterative step of the segmentation. In this article, an objective method for the comparison of ISS is proposed, based on extensive user studies. A summative qualitative content analysis is conducted via abstraction of visual and verbal feedback given by the participants. A direct assessment of the segmentation system is executed by the users via the system usability scale (SUS) and AttrakDiff-2 questionnaires. Furthermore, an approximation of the findings regarding usability aspects in those studies is introduced, conducted solely from the system-measurable user actions during their usage of interactive segmentation prototypes. The prediction of all questionnaire results has an average relative error of 8.9%, which is close to the expected precision of the questionnaire results themselves. This automated evaluation scheme may significantly reduce the resources necessary to investigate each variation of a prototype's user interface (UI) features and segmentation methodologies.
期刊介绍:
The International Journal of Biomedical Imaging is managed by a board of editors comprising internationally renowned active researchers. The journal is freely accessible online and also offered for purchase in print format. It employs a web-based review system to ensure swift turnaround times while maintaining high standards. In addition to regular issues, special issues are organized by guest editors. The subject areas covered include (but are not limited to):
Digital radiography and tomosynthesis
X-ray computed tomography (CT)
Magnetic resonance imaging (MRI)
Single photon emission computed tomography (SPECT)
Positron emission tomography (PET)
Ultrasound imaging
Diffuse optical tomography, coherence, fluorescence, bioluminescence tomography, impedance tomography
Neutron imaging for biomedical applications
Magnetic and optical spectroscopy, and optical biopsy
Optical, electron, scanning tunneling/atomic force microscopy
Small animal imaging
Functional, cellular, and molecular imaging
Imaging assays for screening and molecular analysis
Microarray image analysis and bioinformatics
Emerging biomedical imaging techniques
Imaging modality fusion
Biomedical imaging instrumentation
Biomedical image processing, pattern recognition, and analysis
Biomedical image visualization, compression, transmission, and storage
Imaging and modeling related to systems biology and systems biomedicine
Applied mathematics, applied physics, and chemistry related to biomedical imaging
Grid-enabling technology for biomedical imaging and informatics