Heterogeneity in the usability evaluation process

Proceedings of the 22nd British HCI Group Annual Conference on People and Computers: Culture, Creativity, Interaction, BCS HCI 2008

Volumn 1, Issue , 2008, Pages 89-98

  Schmettow, Martin ^a  

^a UNIVERSITY OF PASSAU   (Germany)

Author keywords

Evaluation; Five Users Debate; Overdispersion; Process Prediction; Stochastic Models; Usability Evaluation

Indexed keywords

FORECASTING; STOCHASTIC SYSTEMS;

EVALUATION; FIVE USERS DEBATE; OVERDISPERSION; PROCESS PREDICTION; USABILITY EVALUATION;

STOCHASTIC MODELS;

EID: 84923797522     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.14236/ewic/hci2008.9     Document Type: Conference Paper

References (27)

1. B. W. Boehm and V. R. Basili. Software defect reduction top 10 list. IEEE Computer, 34(1):135-137, 2001.
2. L. C. Briand, K. El Emam, and O. Freimut, B. G.and Laitenberger. A comprehensive evaluation of capture-recapture models for estimating software defect content. IEEE Transactions on Software Engineering, 26(6):518-540, 2000.
3. K. P. Burnham and D. R. Anderson. Multimodel inference. understanding AIC and BIC in model selection. Sociological Methods & Research, 33(2):261-304, 2004.
4. P. Cairns. HCI...not as it should be: Inferential statistics in HCI research. In L. J. Ball, M. A. Sasse, C. Sas, T. C. Ormerod, A. Dix, P. Bagnall, and T. McEwan, editors, Proceedings of the HCI 2007, volume 1 of People and Computers, pages 195-201. British Computing Society, 2007.
5. D. A. Caulton. Relaxing the homogeneity assumption in usability testing. Behaviour & Information Technology, 20(1):1-7, 2001.
6. A. Chao. An overview of closed capture-recapture models. Journal of Agricultural, Biological, and Environmental Statistics, 6(2):158-175, 2001.
7. L. Faulkner. Beyond the five-user assumption: Benefits of increased sample sizes in usability testing. Behavior Research Methods, Instruments & Computers, 35(3):379-383, 2003.
8. M. Hertzum and N. E. Jacobsen. The evaluator effect: A chilling fact about usability evaluation methods. International Journal of Human-Computer Interaction, 13(4):421-443, 2001.
9. J. R. Lewis. Sample sizes for usability studies: Additional considerations. Human Factors, 36:368-378, 1994.
10. J. R. Lewis. Evaluation of procedures for adjusting problemdiscovery rates estimated from small samples. International Journal of Human-Computer Interaction, 13(4):445-479, 2001.
11. J. R. Lewis. Sample sizes for usability tests: Mostly math, not magic. Interactions, 13(6):29-33, 2006.
12. G. Lindgaard and J. Chattratichart. Usability testing: What have we overlooked? In CHI '07: Proceedings of the SIGCHI conference on Human factors in computing systems, pages 1415-1424, New York, NY, USA, 2007. ACM Press.
13. A. W. Marshall and I. Olkin. Inequalities: Theory of Majorization and Its Applications. Academic Press, New York, 1979.
14. R. Molich and R. Jeffries. Comparative expert review. In Proceedings of the CHI 2003, Extended Abstracts, pages 1060-1061. ACM Press, 2003.
15. J. Nielsen and T. K. Landauer. A mathematical model of the finding of usability problems. In CHI '93: Proceedings of the SIGCHI conference on Human factors in computing systems, pages 206-213, New York, NY, USA, 1993. ACM Press.
16. J. Nielsen and R. Molich. Heuristic evaluation of user interfaces. In Proceedings of the CHI 1990, 1990.
17. R Development Core Team. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria, 2006. ISBN 3-900051-07-0.
18. L.-P. Rivest. Why a time effect often has a limited impact on capture-recapture estimates in closed populations. The Canadian Journal of Statistics, 35(4), 2007. in print.
19. M. Schmettow. Heterogenity in the usability evaluation process-Accompaning website. Website, January 2008. http://schmettow.info/Heterogeneity.
20. M. Schmettow and S. Niebuhr. A pattern-based usability inspection method: First empirical performance measures and future issues. In D. Ramduny-Ellis and D. Rachovides, editors, Proceedings of the HCI 2007, volume 2 of People and Computers, pages 99-102. BCS, September 2007.
21. M. Schmettow andW. Vietze. Introducing item response theory for measuring usability inspection processes. submitted to CHI2008, September 2007.
22. M. Schmettow and W. Vietze. Introducing item response theory for measuring usability inspection processes. In CHI 2008 Proceedings, pages 893-902. ACM SIGCHI, April 2008.
23. S. S. Shapiro and A. J. Gross. Statistical Modeling Techniques. Marcel Decker, New York, 1981.
24. R. A. Virzi. Refining the test phase of usability evaluation: How many subjects is enough? Human Factors, 34(4):457-468, 1992.
25. W. Whitt. Uniform conditional variability ordering of probability distributions. Journal of Applied Probability, 22:619-633, 1985.
26. A. Woolrych and G. Cockton. Why and when five test users aren't enough. In J. Vanderdonckt, A. Blandford, and A. Derycke, editors, Proceedings of IHM-HCI 2001 Conference, volume 2, pages 105-108. Cepadeus, Toulouse, France, 2001.
27. T.W. Yee. VGAM: Vector Generalized Linear and Additive Models, 2007. R package version 0.7-5.