Jarinee Chattratichart
Applying user testing data to UEM performance metrics
Chattratichart, Jarinee; Brodie, Jaqueline
Abstract
The lack of standard assessment criteria for reliably comparing usability evaluation methods (UEMs) is an important gap in HCI knowledge. Recently, metrics for assessing thoroughness, validity, and effectiveness of UEMs, based on user data, have been proposed to bridge this gap. This paper reports our findings of applying these proposed metrics in a study that compared heuristic evaluation (HE) to HE-Plus (an extended version of HE). Our experiment showed better overlap among the HE-Plus evaluators than the HE evaluators, demonstrating greater reliability of the method. When evaluation data, from testing the usability of the same website, was used in calculating the UEM performance metrics, HE-Plus was found to be a superior method to HE in all assessment criteria with a 17%, 39%, and 67% improvement in the aspects of thoroughness, validity, and effectiveness, respectively. The paper concludes with a discussion concerning the limitations of the effectiveness of the UEM from which the real users' data was obtained.
Citation
Chattratichart, J., & Brodie, J. (2004, April). Applying user testing data to UEM performance metrics. Presented at Extended abstracts of the 2004 conference on Human factors and computing systems - CHI '04, Vienna
Presentation Conference Type | Conference Paper (published) |
---|---|
Conference Name | Extended abstracts of the 2004 conference on Human factors and computing systems - CHI '04 |
Start Date | Apr 24, 2004 |
End Date | Apr 29, 2014 |
Publication Date | Apr 24, 2004 |
Deposit Date | Apr 11, 2018 |
Publisher | Association for Computing Machinery (ACM) |
Peer Reviewed | Peer Reviewed |
Pages | 1119-1122 |
Book Title | CHI EA '04 CHI '04 Extended Abstracts on Human Factors in Computing Systems |
Chapter Number | 1 |
ISBN | 1581137036 |
DOI | https://doi.org/10.1145/985921.986003 |
Keywords | Heuristic evaluation |
Public URL | http://researchrepository.napier.ac.uk/Output/1150128 |
You might also like
Building Community and Enhancing Belonging Through Assessment and Feedback: A Roadmap for Assessment Design in Criminology
(2024)
Presentation / Conference Contribution
Belonging in Online and Blended Learning - QAA Scotland Interview with Jacqueline Brodie
(2024)
Digital Artefact
Developing Higher Education Educators’ AI Literacy - An Experiential Learning Method
(2023)
Presentation / Conference Contribution
Downloadable Citations
About Edinburgh Napier Research Repository
Administrator e-mail: repository@napier.ac.uk
This application uses the following open-source libraries:
SheetJS Community Edition
Apache License Version 2.0 (http://www.apache.org/licenses/)
PDF.js
Apache License Version 2.0 (http://www.apache.org/licenses/)
Font Awesome
SIL OFL 1.1 (http://scripts.sil.org/OFL)
MIT License (http://opensource.org/licenses/mit-license.html)
CC BY 3.0 ( http://creativecommons.org/licenses/by/3.0/)
Powered by Worktribe © 2025
Advanced Search