A method to standardize usability metrics into a single score

Conference on Human Factors in Computing Systems - Proceedings

Volumn , Issue , 2005, Pages 401-409

  Sauro, Jeff ^a   Kindlund, Erika ^b  

^a PeopleSoft Inc ^*  (United States)

^b Intuit Inc   (United States)

Author keywords

Measurement; Principal Components Analysis; Six Sigma; Standardization; Usability

Indexed keywords

MATHEMATICAL MODELS; MEASUREMENTS; PRINCIPAL COMPONENT ANALYSIS; REGRESSION ANALYSIS; STANDARDIZATION;

SIX SIGMA; USABILITY;

USER INTERFACES;

EID: 80051471461     PISSN: 02749696     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1054972.1055028     Document Type: Conference Paper

References (46)

1. Abran, A., Surya, W., Khelifi, A., Rilling, J., Seffah, A., Robert, F. (2003). Consolidating the ISO Usability Models. Paper presented at 11th annual International Software Quality Management Conference.
2. ANSI (2001). Common industry format for usability test reports (ANSI-NCITS 354-2001). Washington, DC: American National Standards Institute.
3. th Annual International Conference on Systems documentation, (pp.95-104). ACM Press.
4. Breyfogle, F. (1999). Implementing Six Sigma: Smarter Solutions Using Statistical Methods. John Wiley and Sons.
5. Brooke, J. (1996). SUS: A "quick and dirty" usability scale. In P. Jordan, B. Thomas, and B. Weerdmeester (Eds.), Usability Evaluation in Industry (pp. 189-194). London: Taylor and Francis. See also http://www.cee.hw.ac.uk/~ph/sus.html
6. Cattell, R. B. (1966). The scree test for the number of factors. Multivariate Behavioral Research, 1, 245- 276.
7. Chin, J.P., Diehl, V. A., and Norman, K. L. (1988). Development of an instrument measuring user satisfaction of the human-computer interface. Proc. CHI '88. (pp. 213-218). Washington, D.C.: ACM Press. See also http://www.lap.umd.edu/QUIS/index.html
8. Cordes, R. E (1984). Application of Magnitude Estimation for Evaluating Software Ease of Use. In Gavriel Salvendy (Ed.) First USA-Japan Conference on Human Computer Interaction, Amsterdam: Elsevier Science Publishers.
9. Dumas, J. S. (2003). User-based evaluations. In J. A. Jacko and A. Sears (Eds.), The Human-Computer Interaction Handbook (pp. 1093-1117). Mahwah, NJ: Lawrence Erlbaum.
10. Dunteman, George H, (1989) Principal Components Analysis. In Sage University Papers Series Quantitative Applications in the Social Sciences ; No. 07-069 Newbury Park Sage Publications, Inc.
11. Dumas, J., and Redish, J. C. (1999). A practical guide to usability testing. Portland, OR: Intellect.
12. Frøkjær, E., Hertzum, M., and Hornbæk, K. (2000) Measuring usability: are effectiveness, efficiency, and satisfaction really correlated? In Proc. CHI 2000, (pp.345-352). Washington, D.C.: ACM Press.
13. Gliem, J. and Gliem, R. (2003). Calculating, Interpreting, and Reporting Cronbach's Alpha Reliability Coefficient for Likert-Type Scales. In 2003 Midwest Research to Practice Conference in Adult, Continuing and Community Education. Columbus, OH.
14. Hair, Anderson, Tatham, Black (1998) Multivariate Data Analysis Fifth Edition. NJ: Prentice Hall.
15. Hagen, E. (Ed.) (1976). Human Reliability Analysis, control and instrumentation. Nuclear Safety. 17(3), 315-326.
16. Hassenzahl, M. Sandweg, N. (2004). From Mental Effort to Perceived Usability: Transforming Experiences into Summary Assessments. In the Extended Abstracts of the 2004 conference on Human Factors and Computing Systems (pp 1283-1286).
17. Harry, M. J (1987). The Nature of Six Sigma Quality. Technical Report, Government Electronics Group, Motorola Inc. Scottsdale, AZ.
18. ISO. (1998). Ergonomic requirements for office work with visual display terminals (VDTs) - Part 11: Guidance on usability (ISO 9241-11:1998(E)). Geneva, Switzerland: Author.
19. Jolliffe, I. T. (1972). Discarding variables in a principal component analysis 1: Artificial data. Applied Statistics, 21, 160-173.
20. Jolliffe, Ian T.(2002). Principal Component Analysis. Secaucus, NJ, USA: Springer-Verlag.
21. Kaiser, H. F. (1960). The application of electronic computers to factor analysis. Educational Psychology Measurement, 20, 141-151.
22. Kirakowski, J. (1996). The Software Usability Measurement Inventory: Background and usage. In P. Jordan, B. Thomas, and B. Weerdmeester (Eds.), Usability Evaluation in Industry (pp. 169-178). London, UK: Taylor and Francis. (Also, see http://www.ucc.ie/hfrg/questionnaires/sumi/index.html)
23. Kirakowski, J., and Corbett, M. (1993). SUMI: The Software Usability Measurement Inventory. British Journal of Educational Technology, 24, 210-212.
24. Lewis, J. R (1982) "Testing Small System Customer Setup" in Proceedings of the Human Factors Society 26th Annual Meeting p. 718-720
25. Lewis, J (1991) A Rank-Based Method for the Usability Comparison of Competing Products. In Proceedings of the Human Factors and Ergonomics Society 35th Annual Meeting San Francisco California (pp 1312-1316).
26. Lewis, J. R. (1991). Psychometric evaluation of an after-scenario questionnaire for computer usability studies: The ASQ. SIGCHI Bulletin, 23, 78-81.
27. th Annual Meeting (pp. 1259-1263). Atlanta, GA: Human Factors Society.
28. Lewis, J. R. (1993). IBM computer usability satisfaction questionnaires: Psychometric evaluation and instructions for use (Tech. Report 54.786). Boca Raton, FL: IBM Corp. http://drjim.0catch.com/usabqtr.pdf
29. Lewis, J. R. (1995). IBM computer usability satisfaction questionnaires: Psychometric evaluation and instructions for use. International Journal of Human-Computer Interaction, 7, 57-78.
30. rd Edition. Gavriel Salvendy Editor. John Wiley & Sons.
31. Martin,P and Bateson, P (1993) Measuring Behaviour. (2nd Edition) Cambridge [England]; New York, NY, USA: Cambridge University Press.
32. McGee, M. (2003). Usability magnitude estimation. Proc. HFES, 47th Annual Meeting, (691-695).
33. McGee, M (2004). Master usability scaling: magnitude estimation and master scaling applied to usability measurement. In Proc. CHI 2004, (pp 335 - 342). Washington, D.C.: ACM Press.
34. McIver, J. P., & Carmines, E. G. (1981). Unidimensional scaling. Thousand Oaks, CA: Sage.
35. Molich, R., Ede, M., Kaasgaard, K., and Karyukin, B (2004). Comparative Usability Evaluation. Behaviour & Information Technology, 23(1), 65-74.
36. Morrison, D. F. (1976). Multivariate statistical methods (2nd ed.). New York: McGraw-Hill
37. Nielsen, J (1994) Usability Engineering. San Francisco: Morgan Kaufman.
38. Nielsen, J. and Levy, J. (1994) Measuring Usability: Preference vs. Performance. Communications of the ACM, 37, p. 66-76
39. rd Edition). John Wiley & Sons.
40. Nunnally, J. C. (1978). Psychometric Theory. New York, NY: McGraw-Hill.
41. rd Edition). John Wiley & Sons.
42. Sauro, J. (2004) How long should a task take? Identifying Spec Limits for Task Times in Usability Tests. Retrieved September 13, 2004, from Measuring Usability Web site : http://measuringusability.com/time_specs.htm
43. Sauro, J. (2004) How Do You Calculate a Z-Score? Retrieved September 13, 2004, from Measuring Usability Web site: http://measuringusability.eom/z_calc.htm
44. th Annual Conference of the Usability Professionals Association, Montreal, Canada
45. Stevens, S.S. (1975). Psychophysics: Introduction to its Perceptual, Neural, and Social Prospects. New York: John Wiley.
46. Whiteside, J., Bennett, J. and Holtzblatt, K. (1988) "Usability Engineering: Our Experience and Evolution" in The Handbook of Human Computer Interaction Elsevier Science Publishers, Amsterdam, pp 791-817