Heuristic evaluation: Comparing ways of finding and reporting usability problems

Interacting with Computers

Volumn 19, Issue 2, 2007, Pages 225-240

  Hvannberg, Ebba Thora ^a   Law, Effie Lai Chong ^b   Lárusdóttir, Marta Kristín ^c  

^a UNIVERSITY OF ICELAND   (Iceland)

^b Reykjavik University ^*  (Switzerland)

^c REYKJAVIK UNIVERSITY   (Iceland)

Author keywords

Comparison framework; Effectiveness; Efficiency; Heuristic evaluation; Reporting; User interface; Web tool

Indexed keywords

COGNITIVE SYSTEMS; HEURISTIC METHODS; RELIABILITY; USABILITY ENGINEERING; USER INTERFACES; WORLD WIDE WEB;

COMPARISON FRAMEWORK; HEURISTIC EVALUATION; USABILITY PROBLEM REPORTING; WEB TOOL;

PROBLEM SOLVING;

EID: 33846538601     PISSN: 09535438     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.intcom.2006.10.001     Document Type: Article

References (54)

1. Andre T.S., Hartson R.H., Belz S.M., and McCreary F.A. The user action framework: a reliable foundation for usability engineering support tools. International Journal of Human-Computer Studies 54 (2001) 107-136
2. Barnum, C.R., 2003. What's in a number. Available from : , Accessed 2006, Society for Technical Communication.
3. Bastien J.M.C., Scapin D., and Leulier C. The Ergonomic Criteria and the ISO 9241-10 Dialogue Principles: a comparison in an evaluation task. Interacting with Computers 11 (1999) 299-322
4. Blackmon, M.H., Kitajima, M., Polson, P.G., 2005. Tool for accurately predicting website navigation problems, non-problems, problem severity, and effectiveness of repairs. In: Conference on Human Factors in Computing Systems archive Proceedings of the SIGCHI Conference on Human factors in Computing Systems, ACM, Portland, Oregon, USA, pp. 31-40.
5. Brooke J. SUS: A 'quick and dirty' usability scale. In: Jordan P.W., Thomas B., Weerdmeester B.A., and McClelland I.L. (Eds). Usability Evaluation in Industry (1996), Taylor & Francis, London 189-194
6. Chattratichart, J., Brodie, J., 2004. Applying user testing data to UEM performance metrics. In: CHI 2004, ACM, Vienna, Austria, pp. 1119-1122.
7. Cockton, G., Lavery, D., 1999. A framework for usability problem extraction. In: Sasse, A., Johnson C. (Eds.), INTERACT 1999, pp. 347-355.
8. Cockton G., and Woolrych A. Understanding inspection methods: lessons from an assessment of heuristic evaluation. In: Blandford A., Vanderdonckt J., and Gray P.D. (Eds). People and Computers XV (2001), Springer-Verlag, Lille, France 171-182
9. Cockton G., Lavery D., and Woolrych A. Inspection-based evaluation. In: Jacko J.A., and Sears A. (Eds). The Human-Computer Interaction Handbook (2003), Lawrence Erlbaum Associates, NJ
10. Cockton G., Woolrych A., Hall L., and Hindmarch M. Changing analysts' tunes: the surprising impact of a new instrument for usability inspection method assessment. In: Palangue P., Johnson P., and O'Neill E. (Eds). HCI 2003 (2003), Springer-Verlag, Bath 145-162
11. Cockton, G., Woolrych, A., Hindmarch, M., 2004. Reconditioned merchandise: extended structured report formats in usability inspection. In: CHI'2004, ACM, Vienna, Austria, pp. 1433-1436.
12. Connell, I.W., Hammond, N.V., 1999. Comparing usability evaluation principles with heuristics. In: Sasse, A., Johnson C. (Eds.), Proceedings of the 7th IFIP international conference on Human-computer Interaction, INTERACT'99, IOS Press, Edinburgh.
13. Connell I., Blandford A., and Green T. CASSM and cognitive walkthrough: usability issues with ticket vending machines. Behaviour and Information Technology 23 (2004) 307-320
14. Cook, D.J., Bailey, B.P., 2005. Designers' use of paper and the implications for informal tools. In: OZCHI 2005, vol. 122, ACM, Canberra, Australia, pp. 1-10.
15. Desurvire, H.W., Kondziela, J.M., Atwood, M.E., 1992. What is gained and lost using evaluation methods other than empirical testing. In: Monk, A., Diaper, D., Harrison M.D. (Eds.), HCI, Proceedings of the Conference on People and Computers VII, pp. 89-102.
16. Dillon A., and Watson C. User analysis HCI - the historical lessons from individual differences research. International Journal of Human-Computer Studies 45 (1996) 619-638
17. Doubleday, A., Ryan, M., Springett, M., Sutcliffe, A., 1997. A comparison of usability techniques for evaluating design. In: DIS'97, ACM Press, Amsterdam, pp. 101-110.
18. Faulkner L. Beyond the five-user assumption: benefits of increased sample sizes in usability testing. Behavior Research Methods, Instruments, and Computers 35 (2003) 379-383
19. Folmer E., and Bosch J. Architecting for usability: a survey. The Journal of Systems and Software 70 (2004) 61-78
20. Frøkjær, E., Lárusdóttir, M.K., 1999. Predictions of usability: comparing method combinations. In: Managing Information Technology Resources in Organizations in the Next Millennium, Idea group publishing.
21. Gerhardt-Powals J. Cognitive engineering principles for enhancing human-computer performance. International Journal of Human-Computer Interaction 8 (1996) 189-211
22. Gladwell, M., 2002. The social life of paper. In: The New Yorker, The New Yorker Magazine, New york, NY, pp. 92-96.
23. Gray W.D., and Salzman M.C. Damaged merchandise?. Human-Computer Interaction 13 (1998) 203-262
24. Hartson H.R., Andre T.S., and Williges R.C. Criteria for evaluating usability evaluation methods. International Journal of Human-Computer Interaction 13 (2001) 373-410
25. Hertzum M., and Jacobsen N.E. The evaluator effect: a chilling fact about usability evaluation methods. International Journal of Human-Computer Interaction 13 (2001) 421-443
26. Holzinger A. Usability engineering methods for software developers. Communication of the ACM 48 (2005) 71-74
27. Hornbæk K., and Frökjær E. Usability inspection by metaphors of human thinking compared to heuristic evaluation. International Journal of Human-Computer Interaction 17 (2004) 357-374
28. Jeffries, R., Miller, J.R., Wharton, C., Uyeda, K.M., 1991. User interface evaluation in the real world: a comparison of four techniques. In: ACM CHI'91, New Orleans, LA, pp. 119-124.
29. John B.E., and Mashyna M.E. Evaluating a multimedia authoring tool. Journal of the American Society for Information Science 48 (1997) 1004-1022
30. Karat, C., Campbell, R., Fiegel, T., 1992. Comparison of empirical testing and walkthrough methods in user interface evaluation. In: ACM CHI'92 Conference, Monterey, California, pp. 397-404.
31. Lavery, D., Cockton, G., Atkinson, M.P., Heuristic Evaluation. Usability Evaluation Materials, , 1996.
32. Law, E.L.-C., Hvannberg, E.T., 2004a. Analysis of strategies for improving and estimating the effectiveness of heuristic evaluation. In: NordiCHI 2004, Tampere, Finland, pp. 241-250.
33. Law, E.L.-C., Hvannberg, E.T., 2004b. Analysis of the combinatorial user effect of international usability tests. In: CHI 2004, ACM, Vienna, Austria, pp. 9-16.
34. Lewis, J.R., 1991. Psychometric evaluation of an after-scenario questionnaire for computer usability studies: the ASQ. In: ACM SIGCHI Bulletin, vol. 23, pp. 78-81.
35. Mankoff, J., Dey, A.K., Hsieh, G., Kientz, J., Lederer, S., Ames, M., 2003. Heuristic evaluation of ambient displays. In: CHI 2003, ACM Press, Florida, USA., pp. 169-176.
36. Molich, R., Jeffries, R., 2003. Comparative Expert Reviews. In: CHI 2003: New Horizons, ACM, Ft. Lauderdale, Florida, USA, pp. 1060-1061.
37. Molich R., Ede M.R., Kaasgaard K., and Karyukin B. Comparative usability evaluation. Behaviour and Information Technology 23 (2004) 65-74
38. Newman, W., Taylor, A., 1999. Towards a methodology employing critical parameters to deliver performance improvements in interactive systems. In: Sasse, M.A., Tauber M. (Eds.), INTERACT'99, 7th IFIP TC.13 International Conference on Human-Computer Interaction, Edinburgh, Scotland, pp. 605-612.
39. Nielsen J. Usability Engineering (1993), Academic Press, New York
40. Nielsen, J., 1994a. Enhancing the explanatory power of usability heuristics. In: CHI'94, ACM, Boston, Massachusetts USA, pp. 152-158.
41. Nielsen J. Heuristic evaluation. In: Nielsen J., and Mack R.L. (Eds). Usability Inspection Methods (1994), John Wiley & Sons 25-62
42. Nielsen, J., 2000. Why you only need to test with 5 users? Available from: , Accessed 2005.
43. Nielsen, J., Molich, R., 1990. Heuristic evaluation of user interfaces. In: Proceedings ACM CHI'90 Conference, ACM, Seattle, WA, pp. 249-256.
44. Norman D.A. Cognitive engineering. In: Norman D.A., and Draper S.W. (Eds). User Centered System Design: New Perspectives on Human-Computer Interaction (1986), Lawrence Erlbaum Associates, Hillsdale, NJ 31-61
45. Renkl A., and Atkinson R.R. Structuring the transition from example study to problem solving in cognitive skills acquisition: a cognitive load perspective. Educational Psychologists 38 (2003) 15-22
46. Rieman, J., Davies, S., Hair, D.C., Esemplare, M., Polson, P., Lewis, C., 1991. An automated cognitive walkthrough. In: Proceedings of the SIGCHI conference on Human factors in computing systems: Reaching through technology, ACM, New Orleans, Louisiana, United States, pp. 427-428.
47. Rosenbaum, S., Ronn, J.A., Humburg, J., 2000. A Toolkit for Strategic Usability: Results from Workshops, Panels, and Surveys. In: Turner, T., Szwillus, G., Czerwinski, M., Paterno F. (Eds.), CHI'2000, ACM, Hague, Amsterdam, pp. 337-344.
48. Somervell J., and McCrickard D.S. Better discount evaluation: illustrating how critical parameters support heuristic creation. Interacting with Computers 17 (2005) 592-612
49. Somervell, J., Wahid, S., McCrickard, D.S., 2003. Usability heuristics for large screen information exhibits. In: Rauterberg, M., Menozzi, M., Wesson J. (Eds.), INTERACT 2003, Zurich, Switzerland., pp. 904-907.
50. Spool, J., Schroeder, W., 2001. Testing web sites: five users is nowhere near enough. In: CHI '01 Extended Abstracts on Human factors in Computing Cystems, Seattle, Washington, pp. 285-286.
51. Virzi R.A. Refining the test phase of usability evaluation: How many subjects is enough?. Human Factors 34 (1992) 457-468
52. Woolrych A., and Cockton G. Why and when five test users aren't enough. In: Vanderdonckt J., Blandford A., and Derycke A. (Eds). IHM-HCI vol. 2 (2001), Toulouse, France 105-108
53. Woolrych, A., Cockton, G., Hindmarch, M., 2004. Falsification testing for usability inspection method assessment. In: Fincher, S., Markopoulos, P., Moore, D., Ruddle R. (Eds.), HCI, BCS, Bath.
54. Zhang Z., Basili V., and Shneiderman B. Perspective-based usability inspection: an empirical validation of efficacy. Empirical Software Engineering 4 1 (1999) 43-69