Validating the User-Centered Hybrid Assessment Tool (User-CHAT): a comparative usability evaluation

K-REx Repository

Show simple item record Elgin, Peter D. 2007-01-04T16:38:34Z 2007-01-04T16:38:34Z 2007-01-04T16:38:34Z
dc.description.abstract Usability practitioners need effective usability assessment techniques in order to facilitate development of usable consumer products. Many usability evaluation methods have been promoted as the ideal. Few, however, fulfill expectations concerning effectiveness. Additionally, lack of empirical data forces usability practitioners to rely on personal judgments and/or anecdotal statements when deciding upon which usability method best suits their needs. Therefore the present study had two principal objectives: (1) to validate a hybrid usability technique that identifies important and ignores inconsequential usability problems, and (2) to provide empirical performance data for several usability protocols on a variety of contemporary comparative metrics. The User-Centered Hybrid Assessment Tool (User-CHAT) was developed to maximize efficient diagnosis of usability issues from a behaviorally-based perspective while minimizing time and resource limitations typically associated with usability assessment environments. Several characteristics of user-testing, the heuristic evaluation, and the cognitive walkthrough were combined to create the User-CHAT. Prior research has demonstrated that the User-CHAT supports an evaluation within 3-4 hrs, can be used by individuals with limited human factors / usability background, and requires little training to be used competently, even for complex systems. A state-of-the-art suite of avionics displays and a series of benchmark tasks provided the context where the User-CHAT’s performance was measured relative to its parent usability methods. Two techniques generated comparison lists of usability problems – user-testing data and various inclusion criteria for usability problems identified by the User-CHAT, heuristic evaluation, and cognitive walkthrough. Overall the results demonstrated that the User-CHAT attained higher effectiveness scores than the heuristic evaluation and cognitive walkthrough, suggesting that it helped evaluators identify many usability problems that actually impact users, i.e., higher thoroughness, while attenuating time and effort on issues that were not important, i.e., higher validity. Furthermore, the User-CHAT had the greatest proportion of usability problems that were rated as serious, i.e., usability issues that hinder performance and compromise safety. The User-CHAT’s performance suggests that it is an appropriate usability technique to implement into the product development lifecycle. Limitations and future research directions are discussed. en
dc.description.sponsorship Federal Aviation Administration en
dc.format.extent 3471570 bytes
dc.format.mimetype application/PDF
dc.language.iso en_US en
dc.publisher Kansas State University en
dc.subject Usability en
dc.subject Comparative Metrics en
dc.subject Heruistic Evaluation en
dc.subject Cognitive Walkthrough en
dc.subject User-testing en
dc.subject Usability Evaluation Methods en
dc.title Validating the User-Centered Hybrid Assessment Tool (User-CHAT): a comparative usability evaluation en
dc.type Dissertation en Doctor of Philosophy en
dc.description.level Doctoral en
dc.description.department Department of Psychology en
dc.description.advisor John J. Uhlarik en
dc.subject.umi Psychology, Experimental (0623) en 2007 en May en

Files in this item

This item appears in the following Collection(s)

Show simple item record

Search K-REx

Advanced Search


My Account

Center for the

Advancement of Digital


118 Hale Library

Manhattan KS 66506

(785) 532-7444