Analysis of combinatorial user effect in international usability tests

Conference on Human Factors in Computing Systems - Proceedings

Volumn , Issue , 2004, Pages 9-16

  Law, Effie Lai Chong ^a   Hvannberg, Ebba Thora ^b  

^a ETH ZURICH   (Switzerland)

^b UNIVERSITY OF ICELAND   (Iceland)

Author keywords

Binomial model; Evaluator effect; International usability test; Monte Carlo simulation; User effect

Indexed keywords

COMBINATORIAL MATHEMATICS; DATA REDUCTION; MATHEMATICAL MODELS; MONTE CARLO METHODS; NATURAL FREQUENCIES; PROBLEM SOLVING; QUALITY CONTROL; SOFTWARE ENGINEERING; STANDARDS; STATISTICAL METHODS;

BINOMIAL MODEL; EVALUATOR EFFECT; INTERNATIONAL USABILITY TEST; USER EFFECT;

HUMAN ENGINEERING;

EID: 4544326494     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/985692.985694     Document Type: Conference Paper

References (27)

1. Artim. J. M. (2003). Usability problem severity ratings. Access at: http://www.primaryview.org/CommonDefinitions/
2. Bailey, R.W. (2000). Improving usability in America's voting systems. Access at: http://www.humanfactors.com/library/nov00.asp.
3. nd Annual Meeting, pp.1331-5.
4. Caulton, D.A. (2001). Relaxing the homogeneity assumption in usability testing. Behaviour & Information Technology, 20(1), 1-7.
5. Connell, I. W., & Hammond, N.V. (1999). Comparing usability evaluation principles with heuristics. In Proceedings of INTERACT'99.
6. Diaconis, P., & Efron, B. (1983). Computer intensive methods in statistics. Scientific American, 248, 116-130.
7. Dialogdesign - international usability testing. Access at: http://www.dialogdesign.dk/internationaltest.html
8. Dumas, J.S., & Redish, J.C. (1999). A practical guide to usability testing (rev. ed.). Exeter: Intellect.
9. nd Annual Meeting.
10. Greeno, J.G., Collins, A.M., & Resnick, L.B. (1996). Cognition and learning. In B.C. Berliner & R.C. Calfee (Eds.), Handbook of Educational Psychology.
11. Hertzum, M., & Jacobsen, N.E. (2001). The evaluator effect International Journal of Human Computer Interaction, 13(4), 421-444.
12. nd Annual Meeting.
13. Law, L.-C., & Hvannberg, E.T. (2002). Complementarity and convergence of usability tests and heuristics evaluation. In Proceedings of NordiCHI 2002.
14. Lewis, C. (1982). Using the thinking aloud method in cognitive interface design. Yorktown Heights, NY: IBM Thomas J. Watson Research Center. (RC 9296#40713).
15. Lewis, J.R. (1994). Sample sizes for usability studies: Additional considerations. Human Factors, 36(2), 368-378.
16. Lewis, J.R. (2001). Evaluation of procedures for adjusting problem-discovery rates estimated from small samples. International Journal of Human-Computer Interaction, 13(4), 445-480.
17. Molich, J., et al. (1998). Comparative evaluation of usability tests. In Proceedings of UPA 98.
18. Murphy, J. (2001). Modeling "Designer-Tester-Subject" relationships in international usability testing. In Proceedings of IWIPS 2001, pp. 33-44.
19. Nielsen, J. (1992). Finding usability problems through heuristic evaluation. In P. Bauersfeld, J. Bennett & G. Lynch (Eds.), Proceedings of CHI'92.
20. Nielsen, J. (1994). Estimating the number of subjects needed for a thinking aloud test. International Journal of Human-Computer Studies, 41, 385-397.
21. Nielsen, J. (2000). "Why you only need to test with 5 users", Available at http://www.useit.com/alertbox/20000319.html.
22. Nielsen, J., & Landauer, T.K. (1993). A mathematical model of the finding of usability problem. In Proceedings of InterCHI'93.
23. Spool, J., & Schroeder, W. (2001). Testing websites: Five users is nowhere near enough. In Proceedings of ACM CHI Conference on Human Factors in Computing.
24. Vermeeren, A., van Kesteren, I. & Bekker, M. (2003) Managing the 'Evaluator Effect' in User Testing, In Proceedings of Interact 2003.
25. th Annual Meeting.
26. Virzi, R.A. (1992). Refining the test phase of usability evaluation: How many subjects is enough? Human Factors.
27. Woolyrch, A., & Cockton, C. (2001). Why and when five test users aren't enough. In Proceedings of IHM-HCI 2001 Conference (Vol. 2).