Usability Testing

Handbook of Human Factors and Ergonomics: Fourth Edition

Volumn , Issue , 2012, Pages 1267-1312

  Lewis, James R ^a  

^a IBM   (United States)

Author keywords

Sample size estimation; Usability laboratories; Usability testing; Usability testing, yesterday, today, and tomorrow; Wrapping up

Indexed keywords

EID: 84873881858     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.1002/9781118131350.ch46     Document Type: Chapter

References (249)

1. Aaker, D. A., and Day, G. S. (1986), Marketing Research, Wiley, New York.
2. Abelson, R. P. (1995), Statistics as Principled Argument, Lawrence Erlbaum Associates, Mahwah, NJ.
3. Agresti, A., and Coull, B. (1998), "Approximate Is Better Than 'Exact' for Interval Estimation of Binomial Proportions," The American Statistician, Vol. 52, pp. 119-126.
4. Al-Awar, J., Chapanis, A., and Ford, R. (1981), "Tutorials for the First-Time Computer User," IEEE Transactions on Professional Communication, Vol. 24, pp. 30-37.
5. Albert, W., Tullis, T., and Tedesco, D. (2010), Beyond the Usability Lab: Conducting Large-Scale User Experience Studies, Morgan-Kaufmann, Burlington, MA.
6. Alonso-Ríos, D., Vázquez-Garcia, A., Mosqueira-Rey, E., and Moret-Bonillo. (2010), "Usability: A Critical Analysis and a Taxonomy," International Journal of Human- Computer Interaction, Vol. 26, No. 1, pp. 53-74.
7. Alreck, P. L., and Settle, R. B. (1985), The Survey Research Handbook, Richard D. Irwin, Homewood, IL.
8. Al-Wabil, A., and Al-Khalifa, H. (2009), "A Framework for Integrating Usability Methods: The Mawhiba Web Portal Case Study," in Proceedings of Current Trends in Information Technology 2009 , IEEE, Dubai, UAE, pp. 1-6.
9. American National Standards Institute (ANSI) (2001), Common Industry Format for Usability Test Reports, ANSI-NCITS 354-2001, ANSI, Washington, DC.
10. Anastasi, A. (1976), Psychological Testing, Macmillan, New York.
11. Andre, T. S., Belz, S. M, McCreary, F. A., and Hartson, H. R. (2000), "Testing a Framework for Reliable Classification of Usability Problems," in Proceedings of the IEA 2000/HFES 2000 Congress, Human Factors and Ergonomics Society, Santa Monica, CA, pp. 573-576.
12. Andreasen, M. S., Nielsen, H. V., Schroøder, S. O., and Stage, J. (2007), "What Happened to Remote Usability Testing? An Empirical Study of Three Methods," in Proceedings of CHI 2007 , Association for Computing Machinery, San Jose, CA, pp. 1405-1414.
13. Arning, K., and Ziefle, M. (2008), "Development and Validation of a Computer Expertise Questionnaire for Older Adults," Behaviour & Information Technology, Vol. 27, No. 1, pp. 89-93.
14. Aykin, N. M., and Aykin, T. (1991), "Individual Differences in Human-Computer Interaction," Computers and Industrial Engineering, Vol. 20, pp. 373-379.
15. Baecker, R. M. (2008), "Themes in the Early History of HCI-Some Unanswered Questions," Interactions, Vol. 15, No. 2, pp. 22-27.
16. Bailey, G. (1993), "Iterative Methodology and Designer Training in Human-Computer Interface Design," in INTERCHI '93 Conference Proceedings, Association for Computing Machinery, New York, pp. 198-205.
17. Bailey, R. W., Allan, R. W., and Raiello, P. (1992), "Usability Testing vs. Heuristic Evaluation: A Head to Head Comparison," in Proceedings of the Human Factors and Ergonomics Society 36th Annual Meeting, Human Factors and Ergonomics Society, Santa Monica, CA, pp. 409-413.
18. Bailey, R. W., Wolfson, C. A., Nall, J., and Koyani, S. (2009), "Performance-Based Usability Testing: Metrics That Have the Greatest Impact for Improving a System's Usability" in Human Centered Design, HCII 2009 , M. Kurosu, Ed., Springer-Verlag, Heidelberg, Germany, pp. 3-12.
19. Bangor, A., Kortum, P. T., and Miller, J. T. (2008), "An Empirical Evaluation of the System Usability Scale," International Journal of Human-Computer Interaction, Vol. 24, pp. 574-594.
20. Banks, S. (1965), Experimentation in Marketing, McGraw-Hill, New York.
21. Bargas-Avila, J. A., Lötscher, J., Orsini, S., and Opwis, K. (2009), "Intranet Satisfaction Questionnaire: Development and Validation of a Questionnaire to Measure User Satisfaction with the Intranet," Computers in Human Behavior, Vol. 25, pp. 1241-1250.
22. Barnette, J. J. (2000), "Effects of Stem and Likert Response Option Reversals on Survey Internal Consistency: If You Feel the Need, There Is a Better Alternative to Using Those NegativelyWorded Stems," Educational and Psychological Measurement, Vol. 60, pp. 361-370.
23. Barnum, C. (2002), Usability Testing and Research, Longman, New York.
24. Baumgartner, H., and Steenkamp, J. B. E. M. (2001), "Response Styles in Marketing Research: A Cross- National Investigation," Journal of Marketing Research, Vol. 38, pp. 143-156.
25. Bennett, J. L. (1979), "The Commercial Impact of Usability in Interactive Systems," Infotech State of the Art Report: Man/Computer Communication, Vol. 2, pp. 289-297.
26. Berry, D. C., and Broadbent, D. E. (1990), "The Role of Instruction and Verbalization in Improving Performance on Complex Search Tasks," Behaviour & Information Technology, Vol. 9, pp. 175-190.
27. Bevan, N. (2009), "Extending Quality in Use to Provide a Framework for Usability Measurement," in Human Centered Design, HCII 2009 , M. Kurosu, Ed., Springer- Verlag, Heidelberg, Germany, pp. 13-22.
28. Bevan, N., Kirakowski, J., and Maissel, J. (1991), "What Is Usability?" in Human Aspects in Computing, Design and Use of Interactive Systems and Work with Terminals, Proceedings of the 4th International Conference on Human- Computer Interaction, H. J. Bullinger, Ed., Elsevier Science, Stuttgart, Germany, pp. 651-655.
29. Bias, R. G., and Mayhew, D. J. (1994), Cost-Justifying Usability, Academic, Boston.
30. Blalock, H. M. (1972), Social Statistics, McGraw-Hill, New York.
31. Boødker, S., and Sundblad, Y. (2008), "Usability and Interaction Design-New Challenges for the Scandinavian Tradition," Behaviour & Information Technology, Vol. 27, No. 4, pp. 293-300.
32. Boehm, B. W. (1981), Software Engineering Economics, Prentice-Hall, Englewood Cliffs, NJ.
33. Boren, T., and Ramey, J. (2000), "Thinking Aloud: Reconciling Theory and Practice," IEEE Transactions on Professional Communications, Vol. 43, pp. 261-278.
34. Borsci, S., Federici, S., and Lauriola, M. (2009), "On the Dimensionality of the System Usability Scale: A Test of Alternative Measurement Models," Cognitive Processes, Vol. 10, pp. 193-197.
35. Bosenick, T., Kehr, S., Kühn, M., and Nufer, S. (2007), "Remote Usability Tests-An Extension of the Usability Toolbox for Online-Shops," in Universal Access in HCI, Part I, HCII 2007 , C. Stephanidis, Ed., Springer-Verlag, Heidelberg, Germany, pp. 392-398.
36. Bowers, V., and Snyder, H. (1990), "Concurrent versus Retrospective Verbal Protocols for Comparing Window Usability," in Proceedings of the Human Factors Society 34th Annual Meeting, Human Factors Society, Santa Monica, CA, pp. 1270-1274.
37. Bradley, J. V. (1976), Probability; Decision; Statistics, Prentice-Hall, Englewood Cliffs, NJ.
38. Brooke, J. (1996), "SUS: A 'Quick and Dirty' Usability Scale," in Usability Evaluation in Industry, P. Jordan, B. Thomas, and B. Weerdmeester, Eds., Taylor & Francis, London, pp. 189-194.
39. Brown, F. E. (1980), Marketing Research: A Structure for Decision Making, Addison-Wesley, Reading, MA.
40. Campbell, D. T., and Stanley, J. C. (1963), Experimental and Quasi-Experimental Designs for Research, Rand McNally, Chicago.
41. Capra, M. G. (2007), "Comparing Usability Problem Identification and Description by Practitioners and Students," in Proceedings of the Human Factors and Ergonomics Society 51st Annual Meeting, Human Factors and Ergonomics Society, Santa Monica, CA, pp. 474-478.
42. Caulton, D. A. (2001), "Relaxing the Homogeneity Assumption in Usability Testing," Behaviour & Information Technology, Vol. 20, pp. 1-7.
43. Cavallin, H., Martin, W. M., and Heylighen, A. (2007), "How Relative Absolute Can Be: SUMI and the Impact of the Nature of the Task in Measuring Perceived Software Usability," Artificial Intelligence and Society, Vol. 22, pp. 227-235.
44. Chapanis, A. (1981), "Evaluating Ease of Use," unpublished manuscript prepared for IBM, Boca Raton, FL, available from J. R. Lewis.
45. Chapanis, A. (1988), "Some Generalizations about Generalization," Human Factors, Vol. 30, pp. 253-267.
46. Chin, J. P., Diehl, V. A., and Norman, K. L. (1988), "Development of an Instrument Measuring User Satisfaction of the Human-Computer Interface," in CHI '88 Conference Proceedings: Human Factors in Computing Systems, E. Soloway, D. Frye, and S. B. Sheppard, Eds., Association for Computing Machinery, Washington, DC, pp. 213-218.
47. Churchill, G. A., Jr. (1991), Marketing Research: Methodological Foundations, Dryden, Ft. Worth, TX.
48. Clarke, I. (2001), "Extreme Response Style in Cross-Cultural Research," International Marketing Review, Vol. 18, pp. 301-324.
49. Clemmensen, T., Hertzum, M., Hornbæk, K., Shi, Q., and Yammiyavar, P. (2009), "Cultural Cognition in Usability Evaluation," Interacting with Computers, Vol. 21, pp. 212-220.
50. Cliff, N. (1987), Analyzing Multivariate Data, Harcourt Brace Jovanovich, San Diego, CA.
51. Cockton, G., and Lavery, D. (1999), "A Framework for Usability Problem Extraction," in Human Computer Interaction, INTERACT '99 , M. A. Sasse and C. Johnson, Eds., IOS Press, Amsterdam, pp. 344-352.
52. Cordes, R. E. (1993), "The Effects of Running Fewer Subjects on Time-on-Task Measures," International Journal of Human-Computer Interaction, Vol. 5, pp. 393-403.
53. Cordes, R. E. (2001), "Task-Selection Bias: A Case for User-Defined Tasks," International Journal of Human-Computer Interaction, Vol. 13, pp. 411-419.
54. Cordes, R. E., and Lentz, J. L. (1986), Usability-Claims Evaluation Using a Binomial Test , Technical Report 82.0267, International Business Machines Corporation, Tucson, AZ.
55. Desurvire, H. W., Kondziela, J. M., and Atwood, M. E. (1992), "What Is Gained and Lost When Using Evaluation Methods Other Than Empirical Testing," in Proceedings of HCI '92: People and Computers VII , A. Monk, D. Diaper, and M. D. Harrison, Eds., Cambridge University Press, New York, pp. 89-102.
56. Diamond, W. J. (1981), Practical Experiment Designs for Engineers and Scientists, Lifetime Learning Publications, Belmont, CA.
57. Dickens, J. (1987), "The Fresh Cream Cakes Market: The Use of Qualitative Research as Part of a Consumer Research Programme," in Applied Marketing and Social Research, U. Bradley, Ed., Wiley, New York, pp. 23-68.
58. Dow, S., MacIntyre, B., Lee, J., Oczbek, C., Bolter, J. D., and Gandy, M. (2005), "Wizard of Oz Support throughout an Iterative Design Process," Pervasive Computing, Vol. 4, No. 4, pp. 18-26.
59. Downey, L. L. (2007), "Group Usability Testing: Evolution in Usability Techniques," Journal of Usability Studies, Vol. 2, No. 3, pp. 133-144.
60. Dumas, J. S. (2003), "User-Based Evaluations," in The Human-Computer Interaction Handbook, J. A. Jacko and A. Sears, Eds., Lawrence Erlbaum Associates, Mahwah, NJ, pp. 1093-1117.
61. Dumas, J. (2007), "The Great Leap Forward: The Birth of the Usability Profession (1988-1993)," Journal of Usability Studies, Vol. 2, No. 2, pp. 54-60.
62. Dumas, J., and Redish, J. C. (1999), A Practical Guide to Usability Testing, Intellect, Portland, OR.
63. Dumas, J. S., and Salzman, M. C. (2006), "Usability Assessment Methods," in Reviews of Human Factors and Ergonomics, Vol. 2, R. C. Williges, Ed., Human Factors and Ergonomics Society, Santa Monica, CA, pp. 109-140.
64. Dumas, J., Sorce, J., and Virzi, R. (1995), "Expert Reviews: How Many Experts Is Enough?" in Proceedings of the Human Factors and Ergonomics Society 39th Annual Meeting, Human Factors and Ergonomics Society, Santa Monica, CA, pp. 228-232.
65. Ericsson, K. A., and Simon, H. A. (1980), "Verbal Reports as Data," Psychological Review, Vol. 87, pp. 215-251.
66. Faulkner, L. (2003), "Beyond the Five-User Assumption: Benefits of Increased Sample Sizes in Usability Testing," Behavior Research Methods, Instruments, and Computers, Vol. 35, pp. 379-383.
67. Finstad, K. (2006), "The System Usability Scale and Non- Native English Speakers," Journal of Usability Studies, Vol. 1, No. 4, pp. 185-188.
68. Fisher, J. (1991), "Defining the Novice User," Behaviour & Information Technology, Vol. 10, pp. 437-441.
69. Fowler, C. J. H., Macaulay, L. A., and Fowler, J. F. (1985), "The Relationship between Cognitive Style and Dialogue Style: An Exploratory Study," in People and Computers: Designing the Interface, P. Johnson and S. Cook, Eds., Cambridge University Press, Cambridge, pp. 186-198.
70. Fu, L., Salvendy, G., and Turley, L. (2002), "Effectiveness of User Testing and Heuristic Evaluation as a Function of Performance Classification," Behaviour & Information Technology, Vol. 21, pp. 137-143.
71. Gawron, V. J., Drury, C. G., Czaja, S. J., and Wilkins, D. M. (1989), "A Taxonomy of Independent Variables Affecting Human Performance," International Journal of Man-Machine Studies, Vol. 31, pp. 643-672.
72. Genov, A., Keavney, M., and Zazelenchuk, T. (2009), "Usability Testing with Real Data," Journal of Usability Studies, Vol. 4, No. 2, pp. 85-92.
73. Gillan, D., and Bias, R. (2001), "Usability Science I: Foundations," International Journal of Human-Computer Interaction, Vol. 13, No. 4, pp. 351-372.
74. Gordon, W., and Langmaid, R. (1988), Qualitative Market Research: A Practitioner's and Buyer's Guide, Gower, Aldershot, Hampshire, England.
75. Gould, J. D. (1988), "How to Design Usable Systems," in Handbook of Human-Computer Interaction, M. Helander, Ed., North-Holland, Amsterdam, pp. 757-789.
76. Gould, J. D., and Boies, S. J. (1983), "Human Factors Challenges in Creating a Principal Support Office System: The Speech Filing System Approach," ACM Transactions on Information Systems, Vol. 1, pp. 273-298.
77. Gould, J. D., and Lewis, C. (1984), Designing for Usability: Key Principles and What Designers Think, Technical Report RC-10317, International Business Machines Corporation, Yorktown Heights, NY.
78. Gould, J. D., Boies, S. J., Levy, S., Richards, J. T., and Schoonard, J. (1987), "The 1984 Olympic Message System: A Test of Behavioral Principles of System Design," Communications of the ACM, Vol. 30, pp. 758-769.
79. GraphPad. (2004), "Confidence Interval of a Proportion or Count," available: http://graphpad.com/quickcalcs/ ConfInter-val1.cfm.
80. Gray, W. D., and Salzman, M. C. (1998), "Damaged Merchandise? A Review of Experiments That Compare Usability Evaluation Methods," Human-Computer Interaction, Vol. 13, pp. 203-261.
81. Greene, S. L., Gomez, L. M., and Devlin, S. J. (1986), "A Cognitive Analysis of Database Query Production," in Proceedings of the 30th Annual Meeting of the Human Factors Society, Human Factors Society, Santa Monica, CA, pp. 9-13.
82. Grimm, S. D., and Church, A. T. (1999), "A Cross-Cultural Study of Response Biases in Personality Measures," Journal of Research in Personality, Vol. 33, pp. 415-441.
83. Guan, Z., Lee, S., Cuddihy, E., and Ramey, J. (2006), "The Validity of the Stimulated Retrospective Think-Aloud Method as Measured by Eye Tracking," in Proceedings of CHI 2006 , Association for Computing Machinery, Montreal, Quebec, pp. 1253-1262.
84. Hackman, G. S., and Biers, D. W. (1992), "Team Usability Testing: Are Two Heads Better Than One?" in Proceedings of the Human Factors and Ergonomics Society 36th Annual Meeting, Human Factors and Ergonomics Society, Santa Monica, CA, pp. 1205-1209.
85. Harris, R. J. (1985), A Primer of Multivariate Statistics, Academic, Orlando, FL.
86. Hassenzahl, M. (2000), "Prioritizing Usability Problems: Data Driven and Judgement Driven Severity Estimates," Behaviour & Information Technology, Vol. 19, pp. 29-42.
87. Hertzum, M. (2006), "Problem Prioritization in Usability Evaluation: From Severity Assessments to Impact on Design," International Journal of Human-Computer Interaction, Vol. 21, No. 2, pp. 125-146.
88. Hertzum, M. (2010), "Images of Usability," International Journal of Human-Computer Interaction, Vol. 26, No. 6, pp. 567-600.
89. Hertzum, M., and Jacobsen, N. J. (2003), "The Evaluator Effect: A Chilling Fact about Usability Evaluation Methods," International Journal of Human-Computer Interaction, Vol. 15, pp. 183-204.
90. Hertzum, M., Hansen, K. D., and Andersen, H. H. K. (2009), "Scrutinising Usability Evaluation: Does Thinking Aloud Affect Behaviour and Mental Workload," Behaviour & Information Technology, Vol. 28, No. 2, pp. 165-181.
91. Hoøegh, R. T., and Jensen, J. J. (2008), "A Case Study of Three Software Projects: Can Software Developers Anticipate the Usability Problems in Their Software," Behaviour & Information Technology, Vol. 27, No. 4, pp. 307- 312.
92. Hoøegh, R. T., Nielsen, C. M., Overgaard, M., Pedersen, M. B., and Stage, J. (2006), "The Impact of Usability Reports and User Test Observations on Developers' Understanding of Usability Data: An Exploratory Study," International Journal of Human-Computer Interaction, Vol. 21, No. 2, 173-196.
93. Holleran, P. A. (1991), "A Methodological Note on Pitfalls in Usability Testing," Behaviour &Information Technology, Vol. 10, pp. 345-357.
94. Hornbæk, K. (2006), "Current Practice in Measuring Usability: Challenges to Usability Studies and Research," International Journal of Human-Computer Studies, Vol. 64, No. 2, pp. 79-102.
95. Hornbæk, K. (2010), "Dogmas in the Assessment of Usability Evaluation Methods," Behaviour & Information Technology, Vol. 29, No. 1, pp. 97-111.
96. Hornbæk, K., and Froøkjær, E. (2008a), "A Study of the Evaluator Effect in Usability Testing," Human-Computer Interaction, Vol. 23, pp. 251-277.
97. Hornbæk, K., and Froøkjær, E. (2008b), "Comparison of Techniques for Matching of Usability Problem Descriptions," Interacting with Computers, Vol. 20, pp. 505-514.
98. Hornbæk, K., and Law, E. L. (2007), "Meta-Analysis of Correlations among Usability Measures," in Proceedings of CHI 2007 , Association for Computing Machinery, San Jose, CA, pp. 617-626.
99. Howard, T., and Howard, W. (2009), "Unexpected Complexity in User Testing of Information Products," in Proceedings of the Professional Communication Conference, Institute of Electrical and Electronics Engineers, Waikiki, HI, pp. 1-5.
100. Howard, T. W. (2008), "Unexpected Complexity in a Traditional Usability Study," Journal of Usability Studies, Vol. 3, No. 4, pp. 189-205.
101. Hwang, W., and Salvendy, G. (2007), "What Makes Evaluators to Find More Usability Problems,: A Meta-Analysis for Individual Detection Rates," in Human-Computer Interaction, Part I, HCII 2007 , J. Jacko, Ed., Springer- Verlag, Heidelberg, Germany, pp. 499-507.
102. Hwang, W., and Salvendy, G. (2009), "Integration of Usability Evaluation Studies via a Novel Meta-Analytic Approach: What Are Significant Attributes for Effective Evaluation," International Journal of Human-Computer Interaction, Vol. 25, No. 4, pp. 282-306.
103. Hwang, W., and Salvendy, G. (2010), "Number of People Required for Usability Evaluation: The 10+2 Rule," Communications of the ACM, Vol. 53, No. 5, pp. 130-133.
104. Ibrahim, A. M. (2001), "Differential Responding to Positive and Negative Items: The Case of a Negative Item in a Questionnaire for Course and Faculty Evaluation," Psychological Reports, Vol. 88, pp. 497-500.
105. Illmensee, T., and Muff, A. (2009), "5 Users Every Friday: A Case Study in Applied Research," in Proceedings of the 2009 Agile Conference, Institute of Electrical and Electronics Engineers, Chicago, IL, pp. 404-409.
106. International Organization for Standardization (ISO) (1998), Ergonomic Requirements for Office Work with Visual Display Terminals (VDTs), Part 11, Guidance on Usability, ISO 9241-11:1998(E), ISO, Geneva, Switzerland:
107. Jarrett, C., Quesenbery, W., Roddis, I., Allen, S., and Stirling, V. (2009), "Using Measurements from Usability Testing, Search Log Analysis and Web Traffic Analysis to Inform Development of a Complex Web Site Used for Complex Tasks," in Human Centered Design, HCII 2009 , M. Kurosu, Ed., Springer-Verlag, Heidelberg, Germany, pp. 729-738.
108. Jeffries, R., and Desurvire, H. (1992), "Usability Testing vs. Heuristic Evaluation: Was There a Contest," SIGCHI Bulletin, Vol. 24, pp. 39-41.
109. Jokela, T., Koivumaa, J., Pirkola, J., Salminen, P., and Kantola, N. (2006), "Methods for Quantitative Usability Requirements: A Case Study on the Development of the User Interface of a Mobile Phone," Personal and Ubiquitous Computing, Vol. 10, pp. 345-355.
110. Kantner, L., and Rosenbaum, S. (1997), "Usability Studies of WWW Sites: Heuristic Evaluation vs. Laboratory Testing," in Proceedings of SIGDOC 1997 , Association for Computing Machinery, Salt Lake City, UT, pp. 153-160.
111. Karahasanovic, A., Hinkel, U. N., Sjoøberg, D. I. K., and Thomas, R. (2009), "Comparing of Feedback-Collection and Think-Aloud Methods in Program Comprehension Studies," Behaviour & Information Technology, Vol. 28, No. 2, pp. 139-164.
112. Karat, C. (1997), "Cost-Justifying Usability Engineering in the Software Life Cycle," in Handbook of Human-Computer Interaction, 2nd ed., M. Helander, T. K. Landauer, and P. Prabhu, Eds., Elsevier, Amsterdam, The Netherlands, pp. 767-778.
113. Karat, J. (1997), "User-Centered Software Evaluation Methodologies," in Handbook of Human-Computer Interaction, 2nd ed., M. Helander, T. K. Landauer, and P. Prabhu, Eds., Elsevier, Amsterdam, pp. 689-704.
114. Keenan, S. L., Hartson, H. R., Kafura, D. G., and Schulman, R. S. (1999), "The Usability Problem Taxonomy: A Framework for Classification and Analysis," Empirical Software Engineering, Vol. 1, pp. 71-104.
115. Kelley, J. F. (1984), "An Iterative Design Methodology for User-Friendly Natural Language Office Information Applications," ACM Transactions on Information Systems, Vol. 2, pp. 26-41.
116. Kelley, J. F. (1985), "CAL: A Natural Language Program Developed with the OZ Paradigm: Implications for Supercomputing Systems," in Proceedings of the First International Conference on Supercomputing Systems, Association for Computing Machinery, New York, pp. 238-248.
117. Kennedy, P. J. (1982), "Development and Testing of the Operator Training Package for a Small Computer System," in Proceedings of the Human Factors Society 26th Annual Meeting, Human Factors Society, Santa Monica, CA, pp. 715-717.
118. Kessner, M., Wood, J., Dillon, R. F., and West, R. L. (2001), "On the Reliability of Usability Testing," in Conference on Human Factors in Computing Systems: CHI 2001 Extended Abstracts, J. Jacko and A. Sears, Eds., Association for Computing Machinery, Seattle, WA, pp. 97-98.
119. Kirakowski, J. (1996), "The Software Usability Measurement Inventory: Background and Usage," in Usability Evaluation in Industry, P. Jordan, B. Thomas, and B. Weerdmeester, Eds., Taylor & Francis, London, pp. 169-178 (see also http://www.ucc.ie/hfrg/questionnaires/sumi/ index.html).
120. Kirakowski, J., and Corbett, M. (1993), "SUMI: The Software Usability Measurement Inventory," British Journal of Educational Technology, Vol. 24, pp. 210-212.
121. Kirakowski, J., and Dillon, A. (1988), The Computer User Satisfaction Inventory (CUSI): Manual and Scoring Key, Human Factors Research Group, University College of Cork, Cork, Ireland.
122. Kraemer, H. C., and Thiemann, S. (1987), How Many Subjects, Statistical Power Analysis in Research, Sage Publications, Newbury Park, CA.
123. Krahmer, E., and Ummelen, N. (2004), "Thinking about Thinking Aloud: A Comparison of Two Verbal Protocols for Usability Testing," IEEE Transactions on Professional Communication, Vol. 47, No. 2, pp. 105-117.
124. LaLomia, M. J., and Sidowski, J. B. (1990), "Measurements of Computer Satisfaction, Literacy, and Aptitudes: A Review," International Journal of Human-Computer Interaction, Vol. 2, pp. 231-253.
125. Landauer, T. K. (1997), "Behavioral Research Methods in Human-Computer Interaction," in Handbook of Human- Computer Interaction, 2nd ed., M. Helander, T. K. Landauer, and P. Prabhu, Eds., Elsevier, Amsterdam, pp. 203-227.
126. Lascu, D., and Clow, K. E. (2008), "Web Site Interaction Satisfaction: Scale Development Consideration," Journal of Internet Commerce, Vol. 7, No. 3, pp. 359-378.
127. Law, E. L., Hvannberg, E. T., Cockton, G., Palanque, P., Scapin, D., Springett, M., Stary, C., and Vanderdonckt, J. (2005), "Towards the Maturation of IT Usability Evaluation (MAUSE)," In Proceedings of INTERACT 2005, M. F. Costabile and F. Paterno, Eds., IFIP, Rome, Italy, pp. 1134-1137.
128. Lewis, C., and Norman, D. (1986), "Designing for Error," in User Centered System Design: New Perspectives on Human-Computer Interaction, D. A. Norman and S. W. Draper, Eds., Lawrence Erlbaum Associates, Mahwah, NJ, pp. 411-432.
129. Lewis, J. R. (1982), "Testing Small System Customer Set-up," in Proceedings of the Human Factors Society 26th Annual Meeting, Human Factors Society, Santa Monica, CA, pp. 718-720.
130. Lewis, J. R. (1991a), "A Rank-Based Method for the Usability Comparison of Competing Products," in Proceedings of the Human Factors Society 35th Annual Meeting, Human Factors Society, Santa Monica, CA, pp. 1312- 1316.
131. Lewis, J. R. (1991b), "Psychometric Evaluation of an After- Scenario Questionnaire for Computer Usability Studies: The ASQ," SIGCHI Bulletin, Vol. 23, pp. 78-81.
132. Lewis, J. R. (1992), "Psychometric Evaluation of the Post- Study System Usability Questionnaire: The PSSUQ," in Proceedings of the Human Factors Society 36th Annual Meeting, Human Factors Society, Santa Monica, CA, pp. 1259-1263.
133. Lewis, J. R. (1993), "Multipoint Scales: Mean and Median Differences and Observed Significance Levels," International Journal of Human-Computer Interaction, Vol. 5, pp. 382-392.
134. Lewis, J. R. (1994), "Sample Sizes for Usability Studies: Additional Considerations," Human Factors, Vol. 36, pp. 368-378.
135. Lewis, J. R. (1995), "IBM Computer Usability Satisfaction Questionnaires: Psychometric Evaluation and Instructions for Use," International Journal of Human-Computer Interaction, Vol. 7, pp. 57-78.
136. Lewis, J. R. (1996a), "Binomial Confidence Intervals for Small Sample Usability Studies," in Advances in Applied Ergonomics: Proceedings of the 1st International Conference on Applied Ergonomics, ICAE '96 , G. Salvendy and A. Ozok, Eds., USA Publishing, Istanbul, Turkey, pp. 732-737.
137. Lewis, J. R. (1996b), "Reaping the Benefits of Modern Usability Evaluation: The Simon Story," In Advances in Applied Ergonomics: Proceedings of the 1st International Conference on Applied Ergonomics, ICAE '96 , G. Salvendy and A. Ozok, Eds., USA Publishing, Istanbul, Turkey, pp. 752-757.
138. Lewis, J. R. (2001), "Evaluation of Procedures for Adjusting Problem-Discovery Rates Estimated from Small Samples," International Journal of Human-Computer Interaction, Vol. 13, pp. 445-479.
139. Lewis, J. R. (2002), "Psychometric Evaluation of the PSSUQ Using Data from Five Years of Usability Studies," International Journal of Human-Computer Interaction, Vol. 14, pp. 463-488.
140. Lewis, J. R. (2006), "Sample Sizes for Usability Tests: Mostly Math, Not Magic," Interactions, Vol. 13, No. 6, pp. 29-33. See corrected formula in Interactions, Vol. 14, No. 1, p. 4.
141. Lewis, J. R. (2008), "Usability Evaluation of a Speech Recognition IVR," in Measuring the User Experience, Chapter 10: Case Studies, T. Tullis and B. Albert, Eds., Morgan-Kaufmann, Amsterdam, Netherlands, pp. 244-252.
142. Lewis, J. R. (2011), Practical Speech User Interface Design, Taylor & Francis, Boca Raton, FL.
143. Lewis, J. R., and Pallo, S. (1991), Evaluation of Graphic Symbols for Phone and Line, Technical Report 54.572, International Business Machines Corporation, Boca Raton, FL.
144. Lewis, J. R., and Sauro, J. (2006), "When 100% Really Isn't 100%: Improving the Accuracy of Small-Sample Estimates of Completion Rates," Journal of Usability Studies, Vol. 3, No. 1, pp. 136-150.
145. Lewis, J. R., and Sauro, J. (2009), "The Factor Structure of the System Usability Scale," in Human Centered Design, HCII 2009 , M. Kurosu, Ed., Springer-Verlag, Heidelberg, Germany, pp. 94-103.
146. Lewis, J. R., Henry, S. C., and Mack, R. L. (1990), "Integrated Office Software Benchmarks: A Case Study," in Proceedings of the 3rd IFIP Conference on Human-Computer Interaction, INTERACT '90 , D. Diaper et al., Eds., Elsevier Science, Cambridge, pp. 337-343.
147. Lilienfeld, S. O., Wood, J. M., and Garb, H. N. (2000), "The Scientific Status of Projective Techniques," Psychological Science in the Public Interest , Vol. 1, pp. 27-66.
148. Lindgaard, G. (2006), "Notions of Thoroughness, Efficiency, and Validity: Are They Valid in HCI Practice," International Journal of Industrial Ergonomics, Vol. 36, pp. 1069-1074.
149. Lindgaard, G., and Chattratichart, J. (2007), "Usability Testing: What Have We Overlooked," in Proceedings of CHI 2007 , Association for Computing Machinery, San Jose, CA, pp. 1415-1424.
150. Lord, F. M. (1953), "On the Statistical Treatment of Football Numbers," American Psychologist, Vol. 8, pp. 750- 751.
151. Lucey, N. M. (1991), "More Than Meets the I: User- Satisfaction of Computer Systems," Unpublished thesis for Diploma in Applied Psychology, University College Cork, Cork, Ireland.
152. Macefield, R. (2007), "Usability Studies and the Hawthorne Effect," Journal of Usability Studies, Vol. 2, No. 3, pp. 145-154.
153. MacKenzie, I. S., and Read, J. C. (2007), "Using Paper Mockups for Evaluating Soft Keyboard Layouts," in Proceedings of the 2007 Conference of the Center for Advanced Studies on Collaborative Research, Association for Computing Machinery, Richmond Hill, Canada, pp. 98-108.
154. Macleod, M., Bowden, R., Bevan, N., and Curson, I. (1997), "The MUSiC Performance Measurement Method," Behaviour & Information Technology, Vol. 16, pp. 279-293.
155. Mao, J., Vredenburg, K., Smith, P. W., and Carey, T. (2005), "The State of User-Centered Design Practice," Communications of the ACM, Vol. 48, No. 3, pp. 105-109.
156. Marshall, C., Brendan, M., and Prail, A. (1990), "Usability of Product X: Lessons from a Real Product," Behaviour & Information Technology, Vol. 9, pp. 243-253.
157. Mayer, R. E. (1997), "From Novice to Expert," In Handbook of Human-Computer Interaction, 2nd ed., M. G. Helander, T. K. Landauer, and P. V. Prabhu, Eds., Elsevier, Amsterdam, pp. 781-795.
158. McCurdy, M., Connors, C., Pyrzak, G., Kanefsky, B., and Vera, A. (2006), "Breaking the Fidelity Barrier: An Examination of Our Current Characterization of Prototypes and an Example of a Mixed-Fidelity Success," in Proceedings of CHI 2006 , Association for Computing Machinery, Montreal, Canada, pp. 1233-1242.
159. McFadden, E., Hager, D. R., Elie, C. J., and Blackwell, J. M. (2002), "Remote Usability Evaluation: Overview and Case Studies," International Journal of Human-Computer Interaction, Vol. 14, pp. 489-502.
160. McSweeney, R. (1992), "SUMI: A Psychometric Approach to Software Evaluation," unpublished M.A. (Qual.) thesis in applied psychology, University College of Cork, Cork, Ireland.
161. Miller, L. A., Stanney, K. M., and Wooten, W. (1997), "Development and Evaluation of the Windows Computer Experience Questionnaire (WCEQ)," International Journal of Human-Computer Interaction, Vol. 9, pp. 201-212.
162. Moffat, B. (1990), "Normalized Performance Ratio: A Measure of the Degree to Which a Man-Machine Interface Accomplishes Its Operational Objective," International Journal of Man-Machine Studies, Vol. 32, pp. 21-108.
163. Molich, R., and Dumas, J. S. (2008), "Comparative Usability Evaluation (CUE-4)," Behaviour & Information Technology, Vol. 27, No. 3, pp. 263-281.
164. Molich, R., Bevan, N., Curson, I., Butler, S., Kindlund, E., Miller, D., and Kirakowski, J. (1998), "Comparative Evaluation of Usability Tests," in Usability Professionals Association Annual Conference Proceedings, Usability Professionals Association, Washington, DC, pp. 189-200.
165. Molich, R., Ede, M. R., Kaasgaard, K., and Karyukin, B. (2004), "Comparative Usability Evaluation," Behaviour & Information Technology, Vol. 23, pp. 65-74.
166. Molich, R., Jeffries, R., and Dumas, J. S. (2007), "Making Usability Recommendations Useful and Usable," Journal of Usability Studies, Vol. 2, No. 4, pp. 162-179.
167. Morse, E. L. (2000), "The IUSR Project and the Common Industry Reporting Format," in Proceedings of the ACM Conference on Universal Usability, Association for Computing Machinery, Arlington, VA, pp. 155-156.
168. Myers, J. L. (1979), Fundamentals of Experimental Design, Allyn & Bacon, Boston.
169. Nielsen, J. (1992), "Finding Usability Problems through Heuristic Evaluation," in CHI '92 ACM Conference Proceedings, Monterey, CA, pp. 373-380.
170. Nielsen, J. (1993), Usability Engineering, Academic, San Diego, CA.
171. Nielsen, J. (1994), "Usability Laboratories," Behaviour & Information Technology, Vol. 13, pp. 3-8.
172. Nielsen, J. (1997), "Usability Testing," in Handbook of Human Factors and Ergonomics, 2nd ed., G. Salvendy, Ed., Wiley, New York.
173. Nielsen, J., and Landauer, T. K. (1993), "A Mathematical Model of the Finding of Usability Problems," in Proceedings of the ACM INTERCHI'93 Conference, Association for Computing Machinery, Amsterdam. pp. 206-213.
174. Nielsen, J., and Molich, R. (1990), "Heuristic Evaluation of User Interfaces," in Conference Proceedings on Human Factors in Computing Systems, CHI '90 , Association for Computing Machinery, New York, pp. 249-256.
175. Nisbett, R. E., and Wilson, T. D. (1977), "Telling More Than We Can Know: Verbal Reports on Mental Processes," Psychological Review, Vol. 84, pp. 231-259.
176. Noørgaard, M., and Hornbæk, K. (2009), "Exploring the Value of Usability Feedback Formats," International Journal of Human-Computer Interaction, Vol. 25, No. 1, pp. 49-74.
177. Norman, D. (1986), "Cognitive Engineering," in User Centered System Design: New Perspectives on Human-Computer Interaction, D. A. Norman and S. W. Draper, Eds., Lawrence Erlbaum Associates, Mahwah, NJ, pp. 31-61.
178. Norman, D. A. (1983), "Design Rules Based on Analyses of Human Error," Communications of the ACM, Vol. 26, pp. 254-258.
179. Nunnally, J. C. (1978), Psychometric Theory, McGraw-Hill, New York.
180. Ohnemus, K. R., and Biers, D. W. (1993), "Retrospective versus Thinking Aloud in Usability Testing," in Proceedings of the Human Factors and Ergonomics Society 37th Annual Meeting, Human Factors and Ergonomics Society, Seattle, WA, pp. 1127-1131.
181. Olmsted-Hawala, E. L., Murphy, E., Hawala, S., and Ashenfelter, K. T. (2010), "Think-Aloud Protocols: A Comparison of Three Think-Aloud Protocols for Use in Testing Data-Dissemination Web Sites for Usability," in Proceedings of CHI 2010 , Association for Computing Machinery, Atlanta, GA, pp. 2381-2390.
182. Palmquist, R. A., and Kim, K. S. (2000), "Cognitive Style and On-Line Database Search Experience as Predictors of Web Search Performance," Journal of the American Society for Information Science, Vol. 51, pp. 558-566.
183. Parasuraman, A. (1986), "Nonprobability Sampling Methods," in Marketing Research, Addison-Wesley, Reading, MA, pp. 498-516.
184. Prümper, J., Zapf, D., Brodbeck, F. C., and Frese, M. (1992), "Some Surprising Differences between Novice and Expert Errors in Computerized Office Work," Behaviour & Information Technology, Vol. 11, pp. 319-328.
185. Ramli, R., and Jaafar, A. (2009), "Remote Usability Evaluation System (e-RUE)," in Second International Conference on Computer and Electrical Engineering, Institute of Electrical and Electronics Engineers, Dubai, UAE, pp. 639-643.
186. Rasmussen, J. (1986), Information Processing and Human- Machine Interaction: An Approach to Cognitive Engineering, Elsevier, New York.
187. Redish, J. (2007), "Expanding Usability Testing to Evaluate Complex Systems," Journal of Usability Studies, Vol. 2, No. 3, pp. 102-111.
188. Rengger, R. (1991), "Indicators of Usability Based on Performance," in Human Aspects in Computing, Design and Use of Interactive Systems and Work with Terminals: Proceedings of the 4th International Conference on Human-Computer Interaction, H. J. Bullinger, Ed., Elsevier Science, Stuttgart, Germany, pp. 656-660.
189. Rosenbaum, S. (1989), "Usability Evaluations versus Usability Testing: When and Why," IEEE Transactions on Professional Communication, Vol. 32, pp. 210-216.
190. Rubin, J. (1994), Handbook of Usability Testing: How to Plan, Design, and Conduct Effective Tests, Wiley, New York.
191. Rubin, J., and Chisnell, D. (2008), Handbook of Usability Testing: How to Plan, Design, and Conduct Effective Tests, 2nd ed., Wiley, New York.
192. Ruthford, M. A., and Ramey, J. A. (2000), "Design Response to Usability Test Findings: A Case Study Based on Artifacts and Interviews," IEEE International Professional Communication Conference, IEEE Press, Piscataway, NJ, pp. 315-323.
193. Sadowski, W. J. (2001), "Capabilities and Limitations of Wizard of Oz Evaluations of Speech User Interfaces," in Proceedings of HCI International 2001: Usability Evaluation and Interface Design, Lawrence Erlbaum Associates, Mahwah, NJ, pp. 139-143.
194. Safar, J. A., and Turner, C. W. (2005), "Validation of a Two Factor Structure for System Trust," in Proceedings of the Human Factors and Ergonomics Society 49th Annual Meeting, Human Factors and Ergonomics Society, Santa Monica, CA, pp. 497-501.
195. Sauro, J. (2004), "Restoring Confidence in Usability Results," available: http://www.measuringusability.com/conf_ intervals.htm.
196. Sauro, J. (2006), "Why a Completion Rate Is Better with a Confidence Interval," The UPA Voice, Vol. 18, No. 3, available: http://www.upassoc.org/upa_publications/upa_ voice/volumes/2006/june/completion_rate_with_ confidence_interval.html.
197. Sauro, J., and Dumas, J. S. (2009), "Comparison of Three One-Question, Post-Task Usability Questionnaires," in Proceedings of CHI 2009 , Association for Computing Machinery, Boston, MA, pp. 1599-1608.
198. Sauro, J., and Kindlund, E. (2005), "A Method to Standardize Usability Metrics into a Single Score," in Proceedings of CHI 2005 , Association for Computing Machinery, Portland, OR, pp. 401-409.
199. Sauro, J., and Lewis, J. R. (2005), "Estimating Completion Rates from Small Samples Using Binomial Confidence Intervals: Comparisons and Recommendations," In Proceedings of the Human Factors and Ergonomics Society 49th Annual Meeting, Human Factors and Ergonomics Society, Santa Monica, CA, pp. 2100-2104.
200. Sauro, J., and Lewis, J. R. (2009), "Correlations among Prototypical Usability Metrics: Evidence for the Construct of Usability," in Proceedings of CHI 2009 , Association for Computing Machinery, Boston, MA, pp. 1609-1618.
201. Sauro, J., and Lewis, J. R. (2010), "Average Task Times in Usability Tests: What to Report," in Proceedings of CHI 2010 , Association for Computing Machinery, Atlanta, GA, pp. 2347-2350.
202. Sauro, J., and Lewis, J. R. (2011), "When Designing Usability Questionnaires, Does It Hurt to Be Positive," in Proceedings of CHI 2011 , Association for Computing Machinery, Vancouver, Canada, pp. 2215-2223.
203. Sauro, J., and Lewis, J. R. (2012), Quantifying the User Experience: Practical Statistics for User Research, Morgan- Kaufmann, Burlington, MA.
204. Schmettow, M. (2008), "Heterogeneity in the Usability Evaluation Process," in Proceedings of the 22nd British HCI Group Annual Conference on HCI 2008: People and Computers XXII: Culture, Creativity, Interaction, Vol. 1, Association for Computing Machinery, Liverpool, England, pp. 89-98.
205. Schmettow, M. (2009), "Controlling the Usability Evaluation Process under Varying Defect Visibility," in Proceedings of the 2009 British Computer Society Conference on Human-Computer Interaction, Association for Computing Machinery, Cambridge, pp. 188-197.
206. Seffah, A., and Habieb-Mammar, H. (2009), "Usability Engineering Laboratories: Limitations and Challenges toward a Unifying Tools/Practices Environment," Behaviour & Information Technology, Vol. 28, No. 3, pp. 281-291.
207. Seffah, A., Donyaee, M., Kline, R. B., and Padda, H. K. (2006), "Usability Measurement and Metrics: A Consolidated Model," Software Quality Journal, Vol. 14, pp. 159- 178.
208. Shackel, B. (1990), "Human Factors and Usability," in Human-Computer Interaction: Selected Readings, J. Preece and L. Keller, Eds., Prentice Hall International, Hemel Hempstead, Hertfordshire, England, pp. 27-41.
209. Shneiderman, B. (1987), Designing the User Interface: Strategies for Effective Human-Computer Interaction, Addison-Wesley, Reading, MA.
210. Smith, B., Caputi, P., Crittenden, N., Jayasuriya, R., and Rawstorne, P. (1999), "A Review of the Construct of Computer Experience," Computers in Human Behavior, Vol. 15, pp. 227-242.
211. Smith, D. C., Irby, C., Kimball, R., Verplank, B., and Harlem, E. (1982), "Designing the Star User Interface," Byte, Vol. 7, No. 4, pp. 242-282.
212. Sonderegger, A., and Sauer, J. (2010), "The Influence of Design Aesthetics in Usability Testing: Effects on User Performance and Perceived Usability," Applied Ergonomics, Vol. 41, pp. 403-410.
213. Southwest Oncology Group Statistical Center (SOGSC) (2004), "Binomial Confidence Interval," available: http://www. swogstat.org/stat/public/binomial_conf.htm.
214. Steele, R. G. D., and Torrie, J. H. (1960), Principles and Procedures of Statistics, McGraw-Hill, New York.
215. Stevens, S. S. (1951), "Mathematics, Measurement, and Psychophysics," in Handbook of Experimental Psychology, S. S. Stevens, Ed., Wiley, New York, pp. 1-49.
216. Sudman, S. (1976), Applied Sampling, Academic, New York.
217. Swamy, S. (2007), "How Should You Frame Questions to Measure User Attitudes Accurately," in Usability and Internationalization, Part II, HCII 2007 , N. Aykin, Ed., Springer-Verlag, Heidelberg, Germany, pp. 496-505.
218. Sy, D. (2007), "Adapting Usability Investigations for Agile User-Centered Design," Journal of Usability Studies, Vol. 2, No. 3, pp. 112-132.
219. Theofanos, M., and Quesenbery, W. (2005), "Towards the Design of Effective Formative Test Reports," Journal of Usability Studies, Vol. 1, No. 1, pp. 27-45.
220. Thimbleby, H. (2007), "User-Centered Methods Are Insufficient for Safety Critical Systems," in Proceedings of USAB 2007 , A. Holzinger, Ed., Springer-Verlag, Heidelberg, Germany, pp. 1-20.
221. Tohidi, M., Buxton, W., Baecker, R., and Sellen, A. (2006), "Getting the Right Design and the Design Right: Testing Many Is Better Than One," in Proceedings of CHI 2006 , Association for Computing Machinery, Montreal, Canada, pp. 1243-1252.
222. Tullis, T., and Albert, W. (2008), Measuring the User Experience: Collecting, Analyzing, and Presenting Usability Metrics, Morgan-Kaufmann, Burlington, MA.
223. Tullis, T. S., and Stetson, J. N. (2004), "A Comparison of Questionnaires for Assessing Website Usability," paper presented at the UPA Annual Conference, available: home. comcast.net/~tomtullis/publications/ UPA2004TullisStetson.pdf.
224. Tullis, T., Fleischman, S., McNulty, M., Cianchette, C., and Bergel, M. (2002), "An Empirical Comparison of Lab and Remote Usability Testing of Web Sites," in Proceedings of the Usability Professionals Association, Usability Professionals Association, Chicago, IL, pp. 1-8.
225. Turner, C. W., Lewis, J. R., and Nielsen, J. (2006), "Determining Usability Test Sample Size," in The International Encyclopedia of Ergonomics and Human Factors, W. Karwowski, Ed., CRC Press, Boca Raton, FL, pp. 3084-3088.
226. Uldall-Espersen, T., Froøkjær, E., and Hornbæk, K. (2008), "Tracing Impact in a Usability Improvement Process," Interacting with Computers, Vol. 20, pp. 48-63.
227. van de Vijver, F. J. R., and Leung, K. (2001), "Personality in Cultural Context: Methodological Issues," Journal of Personality, Vol. 69, pp. 1007-1031.
228. van den Haak, M. J., and de Jong, D. T. M. (2003), "Exploring Two Methods of Usability Testing: Concurrent versus Retrospective Think-Aloud Protocols," in Proceedings of the International Professional Communication Conference, IPCC 2003 , Institute of Electrical and Electronics Engineers, Orlando, FL, pp. 285-287.
229. van den Haak, M. J., and de Jong, M. D. T. (2005), "Analyzing the Interaction between Facilitator and Participants in Two Variants of the Think-Aloud Method," in 2005 IEEE International Professional Communication Conference Proceedings, Institute of Electrical and Electronics Engineers, Limerick, Ireland, pp. 323-327.
230. van den Haak, M. J., de Jong, M. D. T., and Schellens, P. J. (2006), "Constructive Interaction: An Analysis of Verbal Interaction in a Usability Setting," IEEE Transactions on Professional Communication, Vol. 49, No. 4, pp. 311-324.
231. Vilbergsdóttir, S. G., Hvannberg, E. T., and Law, E. L. (2006), "Classification of Usability Problems (CUP) Scheme: Augmentation and Exploitation," in Proceedings of NordiCHI 2006 , Association for Computing Machinery, Oslo, Norway, pp. 281-290.
232. Virzi, R. A. (1990), "Streamlining the Design Process: Running Fewer Subjects," in Proceedings of the Human Factors Society 34th Annual Meeting, Human Factors Society, Santa Monica, CA, pp. 291-294.
233. Virzi, R. A. (1992), "Refining the Test Phase of Usability Evaluation: How Many Subjects Is Enough," Human Factors, Vol. 34, pp. 457-468.
234. Virzi, R. A. (1997). "Usability Inspection Methods," in Handbook of Human-Computer Interaction, 2nd ed., M. G. Helander, T. K. Landauer, and P. V. Prabhu, Eds., Elsevier, Amsterdam, pp. 705-715.
235. Virzi, R. A., Sorce, J. F., and Herbert, L. B. (1993), "A Comparison of Three Usability Evaluation Methods: Heuristic, Think-Aloud, and Performance Testing," in Proceedings of the Human Factors and Ergonomics Society 37th Annual Meeting, Human Factors and Ergonomics Society, Santa Monica, CA, pp. 309-313.
236. Vredenburg, K., Mao, J. Y., Smith, P. W., and Carey, T. (2002), "A Survey of User Centered Design Practice," in Proceedings of CHI 2002 , Association for Computing Machinery, Minneapolis, MN, pp. 471-478.
237. Walpole, R. E. (1976), Elementary Statistical Concepts, Macmillan, New York.
238. Wenger, M. J., and Spyridakis, J. H. (1989), "The Relevance of Reliability and Validity to Usability Testing," IEEE Transactions on Professional Communication, Vol. 32, pp. 265-271.
239. West, R., and Lehman, K. R. (2006), "Automated Summative Usability Studies: An Empirical Evaluation," in Proceedings of CHI 2006 , Association for Computing Machinery, Montreal, Canada, pp. 631-639.
240. Whiteside, J., Bennett, J., and Holtzblatt, K. (1988), "Usability Engineering: Our Experience and Evolution," in Handbook of Human-Computer Interaction, M. Helander, Ed., North-Holland, Amsterdam, pp. 791-817.
241. Wickens, C. D. (1998), "Commonsense Statistics," Ergonomics in Design, Vol. 6, No. 4, pp. 18-22.
242. Wiethoff, M., Arnold, A., and Houwing, E. (1992), Measures of Cognitive Workload, MUSiC ESPRIT Project 5429 document code TUD/M3/TD/2.
243. Wildman, D. (1995), "Getting the Most from Paired-User Testing," Interactions, Vol. 2, No. 3, pp. 21-27.
244. Williams, G. (1983), "The Lisa Computer System," Byte, Vol. 8, No. 2, pp. 33-50.
245. Winter, S., Wagner, S., and Deissenboeck, F. (2008), "A Comprehensive Model of Usability," in Engineering Interactive Systems, International Federation for Information Processing, Heidelberg, Germany, pp. 106-122.
246. Wixon, D. (2003), "Evaluating Usability Methods: Why the Current Literature Fails the Practitioner," Interactions, Vol. 10, No. 4, pp. 28-34.
247. Woolrych, A., and Cockton, G. (2001), "Why and When Five Test Users Aren't Enough," in Proceedings of IHM-HCI 2001 Conference, Vol. 2, J. Vanderdonckt, A. Blandford, and A. Derycke, Eds., Cépadèus Éditions, Toulouse, France, pp. 105-108.
248. Wright, P. C., and Monk, A. F. (1991), "A Cost-Effective Evaluation Method for Use by Designers," International Journal of Man-Machine Studies, Vol. 35, pp. 891-912.
249. Wright, R. B., and Converse, S. A. (1992), "Method Bias and Concurrent Verbal Protocol in Software Usability Testing," in Proceedings of the Human Factors and Ergonomics Society 36th Annual Meeting, Human Factors and Ergonomics Society, Santa Monica, CA, pp. 1220-1224.