What makes evaluators to find more usability problems?: A meta-analysis for individual detection rates

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Volumn 4550 LNCS, Issue PART 1, 2007, Pages 499-507

  Hwang, Wonil ^a   Salvendy, Gavriel ^b,c  

^a Soongsil University   (South Korea)

^b SCHOOL OF INDUSTRIAL ENGINEERING   (United States)

^c TSINGHUA UNIVERSITY   (China)

Author keywords

Evaluator's expertise; Heuristic evaluation; Meta analysis; Task type; Usability evaluation; User based testing

Indexed keywords

HEURISTIC METHODS; HUMAN COMPUTER INTERACTION; PROBLEM SOLVING; PRODUCT DESIGN;

HEURISTIC EVALUATION; META-ANALYSIS; USABILITY EVALUATION; USER-BASED TESTING;

USABILITY ENGINEERING;

EID: 38149013074     PISSN: 03029743     EISSN: 16113349     Source Type: Book Series    
DOI: 10.1007/978-3-540-73105-4_55     Document Type: Conference Paper

References (19)

1. Andre, T.S., Hartson, H.R., Williges, R.C.: Determining the effectiveness of the usability problem inspector: a theory-based model and tool for finding usability problems. Human Factors 45,455-482 (2003)
2. Baker, K., Greenberg, S., Gutwin, C.: Empirical development of a heuristic evaluation methodology for shared workspace groupware. In: Proceedings of the 2002 ACM Conference on Computer supported cooperative work, pp. 96-105. ACM, New York (2002)
3. Chen, C., Rada, R.: Interacting with hypertext: a meta-analysis of experimental studies. Human-Computer Interaction 11, 125-156(1996)
4. Cook, T.D., Leviton, L.C.: Reviewing the literature: a comparison of traditional methods with meta-analysis. Journal of Personality 48, 449-472 (1980)
5. De Angeli, A., Matera, M., Costabile, M.F., Garzotto, F., Paolini, P.: Validating the SUE inspection technique. In: Di Gesù, V., Levialdi, S., Tarantino, L. (eds.) Proceedings of Advanced Visual Interfaces (AVI'2000), pp. 143-150. ACM, New York (2000)
6. Glass, G.V., McGaw, B., Smith, M.L.: Meta-analysis in social research. Sage Publications, Beverly Hills CA (1981)
7. Hartson, H.R., Andre, T.S., Williges, R.C: Criteria for evaluating usability evaluation methods. International Journal of Human-Computer Interaction 15, 145-181 (2003)
8. Hedges, L.V.: Fixed effects models. In: Cooper, H., Hedges, L.V. (eds.) The handbook of research synthesis, pp. 285-299. Russell Sage Foundation, New York (1994)
9. Hedges, L.V., Olkin, I.: Statistical methods for meta-analysis. Academic Press, Orlando FL (1985)
10. Law, L.-C., Hvannberg, E.T.: Analysis of combinatorial user effect in international usability tests. In: CHI Conference on Human Factors in Computing Systems, pp. 9-16. ACM, New York (2004)
11. Lipsey, M.W., Wilson, D.B.: Practical meta-analysis. SAGE Publications, Thousand Oaks CA (2001)
12. McLeod, P.L.: An assessment of the experimental literature on electronic support of group work: results of a meta-analysis. Human-Computer Interaction 7, 257-280 (1992)
13. Nielsen, J.: Finding usability problems through heuristic evaluation. In: CHI Conference on Human Factors in Computing Systems, pp. 373-380. ACM, New York (1992)
14. Nielsen, J.: Estimating the number of subjects needed for a thinking aloud test. International Journal of Human-Computer Studies 41, 385-397 (1994)
15. Shadish, W.R., Haddock, C.K.: Combining estimates of effect size. In: Cooper, H., Hedges, L.V. (eds.) The handbook of research synthesis, pp. 261-281. Russell Sage Foundation, New York (1994)
16. Spool, J., Schroeder, W.: Testing web sites: Five users is nowhere near enough. In: CHI '01 extended abstracts on Human factors in computing systems, pp. 285-286. ACM, New York (2001)
17. Virzi, R.A.: Streamlining the design process: Running fewer subjects. In: Human Factors Society 34th Annual Meeting. Human Factors and Ergonomics Society, pp. 291-294. Human Factors and Ergonomics Society, Santa Monica CA (1990)
18. Virzi, R. A.: Refining the test phase of usability evaluation: how many subjects is enough? Human Factors 34, 457-468 (1992)
19. Zhang, Z., Basili, V., Shneiderman, B.: Perspective-based usability inspection: An empirical validation of efficacy. Empirical Software Engineering 4, 43-69 (1999)