Determining the Effectiveness of the Usability Problem Inspector: A Theory-Based Model and Tool for Finding Usability Problems

Human Factors

Volumn 45, Issue 3, 2003, Pages 455-482

  Andre, Terence S ^a   Hartson, H Rex ^b   Williges, Robert C ^b  

^a UNITED STATES AIR FORCE ACADEMY   (United States)

^b VIRGINIA POLYTECHNIC INSTITUTE AND STATE UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COGNITIVE SYSTEMS; COMPUTER SOFTWARE; COST EFFECTIVENESS; EVALUATION; HEURISTIC METHODS; KNOWLEDGE ACQUISITION; USABILITY ENGINEERING;

USABILITY PROBLEM INSPECTOR (UPI);

HUMAN ENGINEERING;

ANALYTIC METHOD; ARTICLE; COGNITION; COMPARATIVE STUDY; COMPUTER INTERFACE; COMPUTER PROGRAM; CONCEPT FORMATION; CONTROLLED STUDY; COST EFFECTIVENESS ANALYSIS; LABORATORY; SCORING SYSTEM; THEORY; VALIDATION PROCESS;

ADULT; COMPUTER SIMULATION; DECISION MAKING, COMPUTER-ASSISTED; FEMALE; HUMAN ENGINEERING; HUMANS; MALE; MAN-MACHINE SYSTEMS; MODELS, PSYCHOLOGICAL; PRODUCT LINE MANAGEMENT; REPRODUCIBILITY OF RESULTS; RESEARCH DESIGN; TASK PERFORMANCE AND ANALYSIS; USER-COMPUTER INTERFACE;

EID: 0345098392     PISSN: 00187208     EISSN: None     Source Type: Journal    
DOI: 10.1518/hfes.45.3.455.27255     Document Type: Article

References (48)

1. Andre, T. S., Williges, R. C., & Hartson, H. R. (1999). The effectiveness of usability evaluation methods: Determining the appropriate criteria. In Proceedings of the Human Factors and Ergonomics Society 43rd Annual Meeting (pp. 1090-1094). Santa Monica, CA: Human Factors and Ergonomics Society.
2. Cuomo, D. L. (1994). A method for assessing the usability of graphical, direct-manipulation style interfaces. International Journal of Human-Computer Interaction, 6, 275-297.
3. Cuomo, D. L., & Bowen, C. D. (1994). Understanding usability issues addressed by three user-system interface evaluation techniques. Interacting with Computers, 6, 86-108.
4. Desurvire, H. W. (1994). Faster, cheaper! Are usability inspection methods as effective as empirical testing? In J. Nielsen & R. L. Mack (Eds.), Usability inspection methods (pp. 173-202). New York: Wiley.
5. Doubleday, A., Ryan, M., Springett, M., & Sutcliffe, A. (1997). A comparison of usability techniques for evaluating design. In Designing Interactive Systems (DIS '97) Conference Proceedings (pp. 101-110). New York: Association for Computing Machinery.
6. Dutt, A., Johnson, H., & Johnson, P. (1994). Evaluating evaluation methods. In G. Cockton, S. W. Draper, & G. R. S. Weir (Eds.), People and computers IX (pp. 109-121). Cambridge, UK: Cambridge University Press.
7. Garzotto, F., Matera, M., & Paolini, P. (1998). Model-based heuristic evaluation of hypermedia usability. In Proceedings of the Working Conference on Advanced Visual Interfaces - AVI '98 (pp. 135-145). New York: Association for Computing Machinery.
8. Grammenos, D., Akoumianakis, D., & Stephanidis, C. (2000). Integrated support for working with guidelines: The Sherlock guideline management system. Interacting with Computers, 12, 281-311.
9. Gray, W. D., & Salzman, M. C. (1998). Damaged merchandise? A review of experiments that compare usability evaluation methods. Human-Computer Interaction, 13, 203-261.
10. Grudin, J. (1989). The case against user interface consistency. Communications of the ACM, 32, 1164-1173.
11. Hartson, H. R., Andre, T. S., & Williges, R. C. (2001). Criteria for evaluating usability evaluation methods. International Journal of Human-Computer Interaction, 13, 373-410.
12. Hartson, H. R., Andre, T. S., Williges, R. W., & Van Rens, L. (1999). The user action framework: A theory-based foundation for inspection and classification of usability problems. In H. Bullinger & J. Ziegler (Eds.), Human-computer interaction: Ergonomics and user interfaces (Vol. 1, pp. 1058-1062). Mahwah, NJ: Erlbaum.
13. Henninger, S. (2000). A methodology and tools for applying context-specific usability guidelines to interface design. Interacting with Computers, 12, 225-243.
14. Jeffries, R., Miller, J. R., Wharton, C., & Uyeda, K. M. (1991). User interface evaluation in the real world: A comparison of four techniques. In CHI '91 Conference Proceedings (pp. 119-124). New York: Association for Computing Machinery.
15. John, B. E., & Marks, S. J. (1997). Tracking the effectiveness of usability evaluation methods. Behaviour and Information Technology, 16, 188-202.
16. John, B. E., & Mashyna, M. M. (1997). Evaluating a multimedia authoring tool. Journal of the American Society for Information Science, 48, 1004-1022.
17. Kahn, M. J., & Prail, A. (1994). Formal usability inspections. In J. Nielsen & R. L. Mack (Eds.), Usability inspection methods (pp. 141-171). New York: Wiley.
18. Karat, C. (1994). A comparison of user interface evaluation methods. In J. Nielsen & R. L. Mack (Eds.), Usability inspection methods (pp. 203-233). New York: Wiley.
19. Kieras, D. E., & Polson, P. G. (1985). An approach to the formal analysis of user complexity. International Journal of Man-Machine Studies, 22, 365-394.
20. Lavery, D., Cockton, G., & Atkinson, M. P. (1997). Comparison of evaluation methods using structured usability problem reports. Behaviour and Information Technology, 16, 246-266.
21. Lewis, C., Polson, P., Wharton, C., & Rieman, J. (1990). Testing a walkthrough methodology for theory-based design of walk-up-and-use interfaces. In CHI '90 Conference Proceedings (pp. 235-242). New York: Association for Computing Machinery.
22. Lewis, J. R. (1994). Sample sizes for usability studies: Additional considerations. Human Factors, 36, 368-378.
23. Lim, K. H., Benbasat, I., & Todd, P. (1996). An experimental investigation of the interactive effects of interface style, instructions, and task familiarity on user performance. ACM Transactions on Computer-Human Interaction, 3, 1-37.
24. Lund, A. M. (1998). The need for a standardized set of usability metrics. In Proceedings of the Human Factors and Ergonomics Society 42nd Annual Meeting (pp. 688-691). Santa Monica, CA: Human Factors and Ergonomics Society.
25. McCreary, F. A. (2001). InTouch usability evaluation (Tech. Report TR-01-11). Blacksburg, VA: Virginia Polytechnic Institute and State University, Department of Computer Science.
26. Nielsen, J. (1992). Finding usability problems through heuristic evaluation. In CHI '92 Conference Proceedings (pp. 373-380). New York: Association for Computing Machinery.
27. Nielsen, J. (1993). Usability engineering. Boston: Academic.
28. Nielsen, J. (1994). Heuristic evaluation. In J. Nielsen & R. L. Mack (Eds.), Usability inspection methods (pp. 25-62). New York: Wiley.
29. Nielsen, J., & Mack, R. L. (Eds.). (1994). Usability inspection methods. New York: Wiley.
30. Nielsen, J., & Molich, R. (1990). Heuristic evaluation of user interfaces. In CHI '90 Conference Proceedings (pp. 249-256). New York: Association for Computing Machinery.
31. Norman, D. A. (1986). Cognitive engineering. In D. A. Norman & S. W. Draper (Eds.), User centered system design: New perspectives on human-computer interaction (pp. 31-61). Hillsdale, NJ: Erlbaum.
32. Polson, P., & Lewis, C. (1990). Theory-based design for easily learned interfaces. Human-Computer Interaction, 5, 191-220.
33. Polson, P., Lewis, C., Rieman, J., & Wharton, C. (1992). Cognitive walkthroughs: A method for theory-based evaluation of user interfaces. International Journal of Man-Machine Studies, 36, 741-773.
34. Rizzo, A., Marchigiani, E., & Andreadis, A. (1997). The AVANTI project: Prototyping and evaluation with a cognitive walkthrough based on the Norman's model of action. In Designing Interactive Systems (DIS '97) Conference Proceedings (pp. 305-309). New York: Association for Computing Machinery.
35. Rosenbaum, S., Rohn, J. A., & Humberg, J. (2000). A toolkit for strategic usability: Results from workshops, panels, surveys. In CHI 2000 Conference Proceedings (pp. 337-344). New York: Association for Computing Machinery.
36. Rowley, D. E., & Rhoades, D. G. (1992). The cognitive jogthrough: A fast-paced user interface evaluation procedure. In CHI '92 Conference Proceedings (pp. 389-395). New York: Association for Computing Machinery.
37. Salton, G., & McGill, M. J. (1983). Introduction to modern information retrieval. New York: McGraw-Hill.
38. Sears, A. (1997). Heuristic walkthroughs: Finding the problems without the noise. International Journal of Human-Computer Interaction, 9, 213-234.
39. Sutcliffe, A., Ryan, M., Springett, M., & Doubleday, A. (1996). Model mismatch analysis: Towards a deeper evaluation of users' usability problems (School of Informatics Report). London: City University.
40. Virzi, R. A. (1990). Streamlining the design process: Running fewer subjects. In Proceedings of the Human Factors and Ergonomics Society 34th Annual Meeting (pp. 291-294). Santa Monica, CA: Human Factors and Ergonomics Society.
41. Virzi, R. A. (1992). Refining the test phase of usability evaluation: How many subjects is enough? Human Factors, 34, 457-468.
42. Virzi, R. A. (1997). Usability inspection methods. In M. G. Helander, T. K. Landauer, & P. V. Prabhu (Eds.), Handbook of human-computer interaction (2nd ed., pp. 705-715). Amsterdam: Elsevier Science.
43. Virzi, R. A., Sorce, J., & Herbert, L. B. (1993). A comparison of three usability evaluation methods: Heuristic, think-aloud, and performance testing. In Proceedings of the Human Factors and Ergonomics Society 36th Annual Meeting (pp. 309-313).
44. Santa Monica, CA: Human Factors and Ergonomics Society. Wharton, C. (1992). Cognitive walkthroughs: Instructions, forms, and examples (Tech. Report CU-ICS-92-17). Boulder: University of Colorado.
45. Wharton, C., Bradford, J., Jeffries, R., & Franzke, M. (1992). Applying cognitive walkthroughs to more complex user interfaces: Experiences, issues, and recommendations. In CHI '92 Conference Proceedings (pp. 381-388). New York: Association for Computing Machinery.
46. Wharton, C., Rieman, J., Lewis, C., & Polson, P. (1993). The cognitive walkthrough method: A practitioner's guide (Tech. Report CU-ICS-93-07). Boulder: University of Colorado.
47. Wharton, C., Rieman, J., Lewis, C., & Polson, P. (1994). The cognitive walkthrough method: A practitioner's guide. In J. Nielsen & R. L. Mack (Eds.), Usability inspection methods (pp. 105-140). New York: Wiley.
48. Wright, P., & Monk, A. (1991). A cost-effective evaluation method for designers. International Journal of Man-Machine Studies, 35, 891-912.