A study of the evaluator effect in usability testing

Human-Computer Interaction

Volumn 23, Issue 3, 2008, Pages 251-277

  Hornbæk, Kasper ^a   Frøkjær, Erik ^a  

^a UNIVERSITY OF COPENHAGEN   (Denmark)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTER NETWORKS; KNOWLEDGE MANAGEMENT;

USABILITY PROBLEMS;

HUMAN COMPUTER INTERACTION;

EID: 51349147119     PISSN: 07370024     EISSN: None     Source Type: Journal    
DOI: 10.1080/07370020802278205     Document Type: Article

References (21)

1. Anderberg, M. A. (1973) Cluster analysis for applications, Academic, New York
2. Andre, T. S., Hartson, H. R., Belz, S. M. and McCreary, F. A. (2001) The user action framework: A reliable foundation for usability engineering support tools. International Journal of Human-Computer Studies, 54, pp. 107-136.
3. Cockton, G. and Lavery, D. (1999) A framework for usability problem extraction. Proceedings of the 7th IFIP TC. 13 International Conference on Human-Computer Interaction (Interact'99) IOS Press., Amsterdam
4. Connell, I. W. and Hammond, N. V. (1999) Comparing usability evaluation principles with heuristics: Problem instances versus problem types. Proceedings of the 7th IFIP TC. 13 International Conference on Human-Computer Interaction (Interact'99) IOS Press., Amsterdam
5. Hartson, H. R., Andre, T. S. and Williges, R. C. (2001) Criteria for evaluating usability evaluation methods. International Journal of Human-Computer Interaction, 13, pp. 373-410.
6. Hertzum, M. and Jacobsen, N. E. (1999) The evaluator effect during first-time use of the cognitive walkthrough technique. Proceedings of HCI International '99, Erlbaum., London
7. Hertzum, M. and Jacobsen, N. E. (2001) The evaluator effect: A chilling fact about usability evaluation methods. International Journal of Human-Computer Interaction, 13, pp. 421-443.
8. Hertzum, M., Jacobsen, N. E. and Molich, R. (2002) Usability inspections by groups of specialist: Perceived agreement in spite of disparate observations. Extended Abstracts of the ACM Conference on Human Factors in Computing Systems (CHI 2002) ACM Press., New York
9. Hornbæk, K. and Frøkjær, E. (2005) Comparing usability problems and redesign proposals as input to practical systems development. Proceedings of CHI'2005 Conference on Human Factors in Computing Systems ACM Press., New York
10. Jaccard, P. (1912) The distribution of the flora of the alpine zone. New Phytologist, 11, pp. 37-50.
11. Jacobsen, N., Hertzum, M. and John, B. (1998) The evaluator effect in usability studies: Problem detection and severity judgments. Proceedings of 42nd Annual Meeting of the Human Factors and Ergonomics Society HFES., Santa Monica, CA
12. Jacobsen, N. E., Hertzum, M. and John, B. E. (1998) The evaluator effect in usability tests. Conference Summary of CHI'98 Conference on Human Factors in Computing Systems ACM Press., New York
13. Jeffries, R. (1994) Usability problem reports: Helping evaluators communicate effectively with developers. Usability inspection methods, pp. 273-294. Wiley., New York
14. John, B. E. and Marks, S. J. (1997) Tracking the effectiveness of usability evaluation methods. Behaviour &Information Technology, 16, pp. 188-202.
15. Kessner, M., Wood, J., Dillon, R. F. and West, R. L. (2001) On the reliability of usability testing. Extended Abstracts of CHI 2001 Conference on Human Factors in Computing Systems. ACM Press., New York
16. Lavery, D., Cockton, G. and Atkinson, M. P. (1997) Comparison of evaluation methods using structured usability problem reports. Behaviour &Information Technology, 16, pp. 246-266.
17. Molich, R., Ede, M. R., Kaasgaard, K. and Karyukin, B. (2004) Comparative usability evaluation. Behaviour &Information Technology, 23, pp. 65-74.
18. Nielsen, J. (1992) Finding usability problems through heuristic evaluation. Proceedings of CHI'92 Conference on Human Factors in Computing Systems ACM Press., New York
19. Nielsen, J. and Molich, R. (1990) Heuristic evaluation of user interfaces. Proceedings of CHI'90 Conference on Human Factors in Computing Systems ACM Press., New York
20. Rand, W. M. (1971) Objective criteria for the evaluation of clustering methods. Journal of the American Statistical Association, 66, pp. 846-850.
21. Vermeeren, A., van Kesteren, I. and Bekker, M. (2003) Managing the evaluator effect in user testing. Proceedings of IFIP Conference on Human-Computer Interaction (Interact 2003). IOS Press., Amsterdam