Problem prioritization in usability evaluation: From severity assessments toward impact on design

International Journal of Human-Computer Interaction

Volumn 21, Issue 2, 2006, Pages 125-146

  Hertzum, Morten ^a  

^a ROSKILDE UNIVERSITY   (Denmark)

Author keywords

[No Author keywords available]

Indexed keywords

DESIGN RESOURCES; FIELD STUDIES; PARTICIPATORY DESIGN; PRIORITIZATION; USABILITY EVALUATION; USABILITY INSPECTION;

DESIGN;

USABILITY ENGINEERING;

EID: 33751523320     PISSN: 10447318     EISSN: None     Source Type: Journal    
DOI: 10.1207/s15327590ijhc2102_2     Document Type: Article

References (45)

1. Bailey, R. W., Allan, R. W., & Raiello, P. (1992). Usability testing vs. heuristic evaluation: A head-to-head comparison. Proceedings of the Human Factors Society 36th Annual Meeting, 409-413.
2. Bass, L., & John, B. E. (2003). Linking usability to software architecture patterns through general scenarios. Journal of Systems and Software, 66, 187-197.
3. Bias, R. G., & Mayhew, D. J. (Eds.). (2005). Cost-justifying usability: An update for the Internet age, San Francisco: Morgan Kaufmann.
4. Boren, T., & Ramey, J. (2000). Thinking aloud: Reconciling theory and practice. IEEE Transactions on Professional Communication, 43, 261-278.
5. Brooks, P. (1994). Adding value to usability testing. In J. Nielsen & R. L. Mack (Eds.), Usability inspection methods (pp. 255-271). New York: Wiley.
6. Catani, M. B., & Biers, D. W. (1998). Usability evaluation and prototype fidelity: Users and usability professionals. Proceedings of the Human Factors and Ergonomics Society 42nd Annual Meeting, 1331-1335.
7. Caulton, D. A. (2001). Relaxing the homogeneity assumption in usability testing. Behaviour & Information Technology, 20, 1-7.
8. Cockton, G., Lavery, D., & Woolrych, A. (2003). Inspection-based evaluations. In J. Jacko & A. Sears (Eds.), The human-computer interaction handbook (pp. 1118-1138). Mahwah, NJ: Lawrence Erlbaum Associates, Inc.
9. Cockton, G., & Woolrych, A. (2002). Sale must end: Should discount methods be cleared off HCI's shelves? ACM Interactions, 9(5), 13-18.
10. Cordes, R. E. (2001). Task-selection bias: A case for user-defined tasks. International Journal of Human-Computer Interaction, 13, 411-419.
11. Dumas, J. S. (2003). User-based evaluations. In J. Jacko & A. Sears (Eds.), The human-computer interaction handbook (pp. 1093-1117). Mahwah, NJ: Lawrence Erlbaum Associates, Inc.
12. Dumas, J. S., Molich, R., & Jeffries, R. (2004). Describing usability problems: Are we sending the right message? ACM Interactions, 11(4), 24-29.
13. Dumas,J.S.,&Redish,J.C.(1999).Apracticalguidetousabilitytesting(Rev. ed.).Portland,OR:Intellect.
14. Gould, J. D., Boies, S. J., & Lewis, C. (1991). Making usable, useful, productivity-enhancing computer applications. Communications of the ACM, 34(1), 74-85.
15. Hackman, G. S., & Biers, D. W. (1992). Team usability testing: Are two heads better than one? Proceedings of the Human Factors Society 36th Annual Meeting, 1205-1209.
16. Hassenzahl, M. (2000). Prioritizing usability problems: Data-driven and judgement-driven severity estimates. Behaviour & Information Technology, 19, 29-42.
17. Hertzum, M. (1999). User testing in industry: A case study of laboratory, workshop, and field tests. In A. Kobsa & C. Stephanidis (Eds.), User interfaces for all: Proceedings of the 5th ERCIM Workshop (pp. 59-72). Sankt Augustin, DE: GMD.
18. Hertzum, M., & Jacobsen, N. E. (2003). The evaluator effect: A chilling fact about usability evaluation methods. International Journal of Human-Computer Interaction, 15, 183-204.
19. Hertzum, M., Jacobsen, N. E., & Molich, R. (2002). Usability inspections by groups of specialists: Perceived agreement in spite of disparate observations. Proceedings of the CHI 2002 Conference on Human Factors in Computing Systems: Extended Abstracts, 662-663.
20. Hornbæk, K., & Frøkjær, E. (2005). Comparing usability problems and redesign proposals as input to practical systems development. Proceedings of the CHI 2005 Conference on Human Factors in Computing Systems, 391-400.
21. Jacobsen, N. E., Hertzum, M., & John, B. E. (1998). The evaluator effect in usability studies: Problem detection and severity judgments. Proceedings of the Human Factors and Ergonomics Society 42nd Annual Meeting, 1336-1340.
22. John, B. E., & Marks, S. J. (1997). Tracking the effectiveness of usability evaluation methods. Behaviour & Information Technology, 16, 188-202.
23. Kjeldskov, J., & Stage, J. (2004). New techniques for usability evaluation of mobile systems. International Journal of Human-Computer Studies, 60, 599-620.
24. Kumar, K. (1990). Post implementation evaluation of computer-based information systems: Current practices. Communications of the ACM, 33(2), 203-212.
25. Law, E. L.-C., & Hvannberg, E. T. (2004a). Analysis of combinatorial user effect in international usability tests. Proceedings of the CHI 2004 Conference on Human Factors in Computing Systems, 9-16.
26. Law, E. L.-C., & Hvannberg, E. T. (2004b). Analysis of strategies for improving and estimating the effectiveness of heuristic evaluation. NordiCHI 2004: Proceedings of the Third Nordic Conference on Human-Computer Interaction, 241-250.
27. Lesaigle, E. M., & Biers, D. W. (2000). Effect of type of information on real time usability evaluation: Implications for remote usability testing. Proceedings of the XIVth Triennial Congress of the International Ergonomics Association and 44th Annual Meeting of the Human Factors and Ergonomics Society, 6-585-6-588.
28. Lewis, J. R. (1994). Sample sizes for usability studies: Additional considerations. Human Factors, 36, 368-378.
29. Lewis, J. R. (2006). Usability testing. In G. Salvendy (Ed.), Handbook of human factors and ergonomics, 3rd edition (pp. 1275-1316). New York: Wiley.
30. Lund, A. M. (1997). Another approach to justifying the cost of usability. ACM. Interactions, 4(3), 48-56,
31. Marshall, C., McManus, B., & Prail, A. (1990). Usability of product X - Lessons from a real project. Behaviour & Information Technology, 9, 243-253.
32. Nielsen, J. (1993). Usability engineering. Boston: Academic.
33. Nielsen, J. (1994). Heuristic evaluation. In J. Nielsen & R. L. Mack (Eds.), Usability inspection methods (pp. 25-62). New York: Wiley.
34. Nielsen, J., & Molich, R. (1990). Heuristic evaluation of user interfaces. Proceedings of the CHI'90 Conference on Human Factors in Computing Systems, 249-256.
35. Pejtersen, A. M., & Rasmussen, J. (1997). Effectiveness testing of complex systems. In G. Salvendy (Ed.), Handbook of human factors and ergonomics, 2nd edition (pp. 1514-1542). New York: Wiley.
36. Rodden, M. J., Green, W. S., & Kanis, H. (1999). Difficulties in usage of a coffeemaker predicted on the basis of design models. Proceedings of the Human factors and Ergonomics Society 43rd Annual Meeting, 476-480.
37. Sawyer, P., Flanders, A., & Wixon, D. (1996). Making a difference - The impact of inspections. Proceedings of the CHI'96 Conference on Human Factors in Computing Systems, 376-382.
38. Smilowitz, E. D., Darnell, M. J., & Benson, A. E. (1994). Are we overlooking some usability testing methods? A comparison of lab, beta, and forum tests. Behaviour & Information Technology, 13, 183-190.
39. Thomas, J. C., & Kellogg, W. A. (1989). Minimizing ecological gaps in interface design. IEEE Software, 6(1), 78-86.
40. Tversky, A., & Kahneman, D. (1974). Judgment under uncertainty: Heuristics and biases. Science, 185, 1124-1131.
41. Virzi, R. A. (1992). Refining the test phase of usability evaluation: How many subjects is enough? Human Factors, 34, 457-468.
42. Whiteside, J., Bennett, J., & Holtzblatt, K. (1988). Usability engineering: Our experience and evolution. In M. Helander (Ed.), Handbook of human-computer interaction (pp. 791-817). Amsterdam: Elsevier.
43. Woolrych, A., & Cockton, G. (2001). Why and when five test users aren't enough. In J. Vanderdonckt, A. Blandford, & A. Derycke (Eds.), Proceedings of the IHM-HCI 2001 Conference, Vol. II (pp. 105-108). Toulouse, France: Cépadèus Éditions.
44. Wright, P. C., & Monk, A. F. (1991). A cost-effective evaluation method for use by designers. International Journal of Man-Machine Studies, 35, 891-912.
45. Zirkler, D., & Ballman, D. R. (1994). Usability testing in a competitive market: Lessons learned. Behaviour & Information Technology, 13, 191-197.