Quantifying heuristic evaluation

Abstract

Since Nielsen presented his work on heuristic evaluation in the 90s, it has been seen how this inspection method has been used recurrently not only in academia but also in the industry. The benefits of their application have been studied widely and multiple heuristics for multiple purposes have been designed. Despite it, quantifying their qualitative nature has been challenging. This research presents a novel approach to provide a framework for quantifying an heuristic evaluation based on the calculus of what we called Usability Percentage. We have used an initial study that started in 2018 with 7 different evaluators and we presented a new methodology to calculate the aforementioned percentage and also provide multiple information that might be added into a final report. We have used the platform RUXAILAB that allows us to create heuristic evaluations to prepare and execute the test. Our findings are that by comparing both studies the new methodology presents a narrowed data distribution through the different evaluators and allows us to identify in a more precise and concise way which heuristics might fail in an evaluation test.