An improved usability measure based on novice and expert performance

International Journal of Human-Computer Interaction

Volumn 27, Issue 3, 2011, Pages 280-302

  MacDorman, Karl F ^a   Whalen, Timothy J ^a   Ho, Chin Chang ^a   Patel, Himalaya ^a  

^a Indiana University   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 79551648613     PISSN: 10447318     EISSN: 15327590     Source Type: Journal    
DOI: 10.1080/10447318.2011.540472     Document Type: Article

References (53)

1. Anderson J. R. (1976). Language, memory and thought. Hillsdale, NJ: Erlbaum.
2. Bangor, A., Kortum, P. T., & Miller, J. T. (2008). An empirical evaluation of the system usability scale. International Journal of Human-Computer Interaction, 24, 574-594.
3. Bargh, J. A., & Chartrand, T. L. (1999). The unbearable automaticity of being. American Psychologist, 54, 462-479.
4. Bessiere, K., Newhagen, J. E., Robinson, J. P., & Shneiderman, B. (2006). A model for computer frustration: The role of instrumental and dispositional factors on incident, session, and post-session frustration and mood. Computers in Human Behavior, 22, 941-961.
5. Brinkman, W.-P., Haakma, R., & Bouwhuis, D. G. (2007). Towards an empirical method of efficiency testing of system parts: A methodological study. Interacting with Computers, 19, 342-356.
6. Brooke, J. (1996). SUS: A "quick and dirty" usability scale. In P.W. Jordan, B. Thomas, B. A. Weerdmeester, & A. L. McClelland (Eds.), Usability evaluation in industry (pp. 189-194). London: Taylor and Francis.
7. Carroll, J. M., & Rosson, M. B. (1987). Paradox of the active user. In J. M. Carroll (Ed.), Interfacing thought: Cognitive aspects of human-computer interaction (pp. 80-111). Cambridge, MA: MIT Press.
8. Chin, J. P., Diehl, V. A., & Norman, K. (1988). Development of an instrument measuring user satisfaction of the human-computer interface. In Proceedings of the Sixth ACM/SIGCHI Conference on Human Factors in Computing Systems (pp. 213-218). New York: ACM Press.
9. Cohen, J. (1977). Statistical power analysis for the behavioral sciences (Rev. ed.). New York: Academic Press.
10. Desurvire, H., Lawrence, D., & Atwood, M. (1991). Empiricism versus judgement: Comparing user interface evaluation methods on a new telephone-based interface. SIGCHI Bulletin, 23(4), 58-59.
11. Dienes, Z., & Perner, J. (1999). A theory of implicit and explicit knowledge. Behavioral and Brain Sciences, 22, 735-808.
12. Dillon, A., & Song, M. (1997). An empirical comparison of the usability for novice and expert searchers of a textual and a graphical interface to an art-resource database. Journal of Digital Information, 1(1), Article No. 2.
13. Douglas, S. A., Kirkpatrick, A. E., & MacKenzie, I. S. (1999). Testing pointing device performance and user assessment with the ISO 9241, Part 9 Standard. In Proceedings of the 17th ACM/SIGCHI Conference on Human Factors in Computing Systems (pp. 215-222). New York: ACM Press.
14. Faulkner, L., & Wick, D. (2005). Cross-user analysis: Benefits of skill level comparison in usability testing. Interacting with Computers, 17, 773-786.
15. Global System for Mobile Communications Association. (2006, November 28). Strong global demand for MMS and mobile email. Retrieved June 29, 2009, from http://www.gsmworld.com/newsroom/press-releases/2118.htm
16. Goonetilleke, R. S., Shih, H. M., On, H. K., & Fritsch, J. (2001). Effects of training and representational characteristics in icon design. International Journal of Human-Computer Studies, 55, 741-760.
17. Gould, J., & Lewis, C. (1985). Designing for usability: Key principles and what designers think. Communications of the ACM, 28, 300-311.
18. Gray,W. D., & Salzman,M. C. (1998). Damaged merchandise? A review of experiments that compare usability evaluation methods. Human-Computer Interaction, 13, 203-261.
19. Greenwald, A. G., Nosek, B. A., & Banaji, M. R. (2003). Understanding and using the Implicit Association Test: I. An improved scoring algorithm. Journal of Personality and Social Psychology, 85, 197-216.
20. Hedges, L. V., & Olkin, I. (1985). Statistical methods for meta-analysis. San Diego, CA: Academic Press.
21. Henderson, R., Podda, J., Smith, M., & Varela-Alvarez, H. (1995). An examination of four user-based software evaluation methods. Interacting with Computers, 7, 412-432.
22. Hertzum, M. & Jacobsen, N. E. (2003). The evaluator effect: A chilling fact about usability evaluation methods. International Journal of Human-Computer Interaction, 15(1), 183-204.
23. Hornbæk, K., & Law, E. L.-C. (2007). Meta-analysis of correlations among usability measures. Proceedings of the ACM/SIGCHI Conference on Human Factors in Computing Systems (pp. 617-626). New York: ACM Press.
24. International Association for the Wireless Telecommunications Industry. (n.d.). Wireless quick facts. Washington, DC: Author. Retrieved on June 29, 2009, from http://www.ctia.org/media/industry_info/index.cfm/AID/10323
25. ISO 9241-11. (1997). Ergonomic requirements for office work with visual display terminals VDTs), Part 11: Guidance on usability specification and measures. Geneva, Switzerland: International Organization for Standards.
26. International Telecommunication Union. (2009). Measuring the information society: The ICT development index. Geneva, Switzerland: Author.
27. John, B. E., & Kieras, D. E. (1996). The GOMS family of user interface analysis techniques: Comparison and contrast. ACM Transactions on Computer-Human Interaction, 3, 320-351.
28. Kang, N. E., & Yoon, W. C. (2008). Age- and experience-related user behavior differences in the use of complicated electronic devices. International Journal of Human-Computer Studies, 66, 425-437.
29. Kirsh, D. (1991).When is information explicitly represented? In P. Hanson (Ed.), Information, thought, and content (pp. 340-365). Vancouver, Canada: UBC Press.
30. Kjeldskov, J., Skov, M. B., & Stage, J. (2005). Does time heal? A longitudinal study of usability. In S. Balbo & T. Bentley (Eds.), Proceedings of the Australian Computer-HumanInteraction Conference. Narrabundah: CHISIG of Australia.
31. Kuniavsky, M. (2003). Observing the user experience: A practitioner's guide to user research. San Francisco: Morgan-Kaufmann.
32. Lewis, J. R. (1995). IBM computer usability satisfaction questionnaires: Psychometric evaluation and instructions for use. International Journal of Human-Computer Interaction, 7(1), 57-78.
33. Lewis, J. R. (2002). Psychometric evaluation of the PSSUQ using data from five years of usability studies. International Journal of Human-Computer Interaction, 14, 463-488.
34. Lindgaard, G., & Dudek, C. (2003). What is this evasive beast we call user satisfaction? Interacting with Computers, 15, 429-452.
35. Luo, L., & John, B. (2005). Predicting task execution time on handheld devices using the keystroke-level model. In Extended Abstracts of the 23rd ACM/SIGCHI Conference on Human Factors in Computing Systems (pp. 1605-1608). New York: ACM Press.
36. MacDorman, K. F., Vasudevan, S. K., & Ho, C.-C. (2009). Does Japan really have robot mania? Comparing attitudes by implicit and explicit measures. AI & Society, 23, 485-510.
37. MacKenzie, I. S. (1992). Fitts' Law as a research and design tool in human-computer interaction. Human-Computer Interaction, 7(1), 91-139.
38. McGee, M., Rich, A., & Dumas, J. (2004). Understanding the usability construct: Userperceived usability. In Proceedings of the Human Factors and Ergonomics Society 48th Annual Meeting (Vol. 48, pp. 907-911). Santa Monica, CA: HFES.
39. Molich, R., & Nielsen, J. (1990). Improving a human-computer dialogue. Communications of the ACM, 33, 338-348.
40. Newell, A., & Simon, H. A. (1972). Human problem solving. Englewood Cliffs, NJ: Prentice Hall.
41. Nielsen, J. (1992). Finding usability problems through heuristic evaluation. Proceedings of the 10th ACM/SIGCHI Conference on Human Factors in Computing Systems (pp. 373-380). New York: ACM Press.
42. Nielsen, J. (1994). Usability engineering. San Francisco: Morgan Kaufmann.
43. Nielsen, J., & Molich, R. (1990). Heuristic evaluation of user interfaces. In Proceedings of the Eighth ACM/SIGCHI Conference on Human Factors in Computing Systems (pp. 249-256). New York: ACM Press.
44. Nielsen, J., & Phillips, V. L. (1993). Estimating the relative usability of two interfaces: Heuristic, formal, and empirical methods compared. In Proceedings of the INTERACT'93 and CHI'93: The 11th ACM/SIGCHI Conference on Human Factors in Computing Systems (pp. 214-221). New York: ACM Press.
45. Norman, D. A. (1983). Some observations on mental models. In D. Gentner & A. L. Stevens (Eds.), Mental models (pp. 7-14). Hillsdale, NJ: Erlbaum.
46. Norman, D. A. (1988). The psychology of everyday things. New York: Basic Books.
47. Proctor, R.W., & Van Zandt, T. (2008). Human factors in simple and complex systems (2nd ed.). Boca Raton, FL: CRC Press.
48. Ratcliff, R. (1993). Methods for dealing with reaction time outliers. Psychological Bulletin, 114, 510-532.
49. Rogers, R. D., & Monsell, S. (1995). Costs of a predictable switch between simple cognitive tasks. Journal of Experimental Psychology: General, 124, 207-231.
50. Sauro, J., & Lewis, J. R. (2009). Correlations among prototypical usability metrics: Evidence for the construct of usability. In Proceedings of the 27th International ACM/SIGCHI Conference on Human Factors in Computing Systems. New York: ACM Press.
51. Silfverberg, M., MacKenzie, I. S., & Korhonen, P. (2000). Predicting text entry speed on mobile phones. In Proceedings of the ACM/SIGCHI Conference on Human Factors in Computing Systems (pp. 9-16). New York: ACM Press.
52. Uehling, D. L., & Wolf, K. (1995). User action graphing effort (UsAGE). In Conference Companion on Human Factors in Computing Systems (pp. 290-291). New York: ACM Press.
53. Urokohara, H., Tanaka, K., Furuta, K., Honda, M., & Kurosu, M. (2000). NEM: "Novice expert ratio method" A usability evaluation method to generate a new performance measure. In Proceedings of the ACM/SIGCHI Conference on Human Factors in Computing Systems (pp. 185-186). New York: ACM Press.