The bootstrap discovery behaviour model: Why five users are not enough to test user experience

Cognitively Informed Intelligent Interfaces: Systems Design and Development

Volumn , Issue , 2012, Pages 258-279

  Borsci, Simone ^a   Federici, Stefano ^b,c   Mele, Maria Laura ^c   Polimeno, Domenico ^b   Londei, Alessandro ^c  

^a BRUNEL UNIVERSITY   (United Kingdom)

^b UNIVERSITY OF PERUGIA   (Italy)

^c SAPIENZA UNIVERSITY OF ROME   (Italy)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84887839877     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.4018/978-1-4666-1628-8.ch015     Document Type: Chapter

References (43)

1. Bohman, P. R., & Anderson, S. (2005). A conceptual framework for accessibility tools to benefit users with cognitive disabilities. Paper presented at the International Cross-Disciplinary Workshop on Web Accessibility. Chiba, Japan.
2. Borsci, S., Kurosu, M., Federici, S., & Mele, M. L. (2013). Computer systems experiences of users with and without disabilities: An evaluation guide for professionals. London, UK: CRC Press.
3. Borsci, S., Londei, A., & Federici, S. (2011). The bootstrap discovery behaviour (BDB): A new outlook on usability evaluation. Cognitive Processing, 12(1), 23-31. doi:10.1007/s10339010-0376-6
4. Caulton, D. (2001). Relaxing the homogeneity assumption in usability testing. Behaviour & Information Technology, 20(1), 1-7. doi:10.1080/01449290010020648
5. Charlton, J. I. (1998). Nothing about us without us: Disability oppression and empowerment. Berkeley, CA: University of California Press.
6. Di Giacomo, E., Didimo, W., Grilli, L., & Liotta, G. (2007). Graph visualization techniques for web clustering engines. IEEE Transactions on Visualization and Computer Graphics, 13(2), 294-304. doi:10.1109/TVCG.2007.40
7. Di Giacomo, E., Didimo, W., Grilli, L., Liotta, G., & Palladino, P. (2008). WhatsOnWeb+: An enhanced visual search clustering engine. Paper presented at the IEEE Pacific Visualization Symposium. Kyoto, Japan.
8. Efron, B. (1979). Bootstrap methods: Another look at the jackknife. Annals of Statistics, 7(1), 1-26. doi:10.1214/aos/1176344552
9. Ericsson, K. A., & Simon, H. A. (1980). Verbal reports as data. Psychological Review, 87(3), 215-251. doi:10.1037/0033-295X.87.3.215
10. Fairweather, P., & Trewin, S. (2010). Cognitive impairments and Web 2.0. Universal Access in the Information Society, 9(2), 137-146. doi:10.1007/s10209-009-0163-2
11. Federici, S., & Borsci, S. (2010). Usability evaluation: models, methods, and applications. In J. Stone & M. Blouin (Eds.), International Encyclopedia of Rehabilitation. Buffalo, NY: Center for International Rehabilitation Research Information and Exchange (CIRRIE). Retrieved from http://cirrie.buffalo.edu/encyclopedia/article.php?id=277&language=en.
12. Federici, S., Borsci, S., & Mele, M. L. (2010). Usability evaluation with screen reader users: A video presentation of the PCTA's experimental setting and rules. Cognitive Processing, 11(3), 285-288. doi:10.1007/s10339-010-0365-9
13. Federici, S., Borsci, S., Mele, M. L., & Stamerra, G. (2010). Web popularity: An illusory perception of a qualitative order in information. Universal Access in the Information Society, 9(4), 375-386. doi:10.1007/s10209-009-0179-7
14. Federici, S., Borsci, S., & Stamerra, G. (2010). Web usability evaluation with screen reader users: Implementation of the partial concurrent thinking aloud technique. Cognitive Processing, 11(3), 263-272. doi:10.1007/s10339-010-0358-8
15. Federici, S., Micangeli, A., Ruspantini, I., Borgianni, S., Corradi, F., & Pasqualotto, E. (2005). Checking an integrated model of web accessibility and usability evaluation for disabled people. Disability and Rehabilitation, 27(13), 781-790. doi:10.1080/09638280400014766
16. th International ACM SIGACCESS Conference on Computers and Accessibility. Halifax, Canada.
17. Feng, J., Lazar, J., Kumin, L., & Ozok, A. (2010). Computer usage by children with down syndrome: Challenges and future research. Transactions on Accessible Computing, 2(3), 1-44. doi:10.1145/1714458.1714460
18. Folstein, M. F., Folstein, S. E., & McHugh, P. R. (1975). Mini-mental state: A practical method for grading the cognitive state of patients for the clinician. Journal of Psychiatric Research, 12(3), 189-198. doi:10.1016/0022-3956(75)90026-6
19. Fox, J. (2002). An R and S-plus companion to applied regression. Thousand Oaks, CA: SAGE.
20. Garrett, J. J. (2003). The elements of user experience: User-centered design for the web. New York, NY: New Riders Press.
21. Goodstein, R. L. (1963). Boolean algebra. Oxford, UK: Pergamon Press.
22. Hartson, H. R., Andre, T. S., & Williges, R. C. (2001). Criteria for evaluating usability evaluation methods. International Journal of Human-Computer Interaction, 13(4), 373-410. doi:10.1207/S15327590IJHC1304_03
23. Hertzum, M., & Jacobsen, N. E. (2003). The evaluator effect: A chilling fact about usability evaluation methods. International Journal of Human-Computer Interaction, 15(4), 183-204. doi:10.1207/S15327590IJHC1501_14
24. ISO. (1998). ISO 9241-11: Ergonomic requirements for office work with visual display terminals. Geneve, Switzerland: ISO.
25. ISO. (2010). Ergonomics of human-system interaction - Part 210: Human-centred design for interactive systems. Geneve, Switzerland: ISO.
26. Lewis, C. (2005). HCI for people with cognitive disabilities. ACM SIGACCESS Accessibility and Computing, 83, 12-17. doi:10.1145/1102187.1102190
27. Lewis, J. R. (1994). Sample sizes for usability studies: Additional considerations. Human Factors, 36(2), 368-378.
28. Lewis, J. R. (2001). Evaluation of procedures for adjusting problem-discovery rates estimated from small samples. International Journal of Human-Computer Interaction, 13(4), 445-479. doi:10.1207/S15327590IJHC1304_06
29. Lewis, J. R. (2006). Sample sizes for usability tests: Mostly math, not magic. Interaction, 13(6), 29-33. doi:10.1145/1167948.1167973
30. Luckasson, R., Borthwick-Duffy, S., Buntinx, W. H. E., Coulter, D. L., Craig, E. M., & Reeve, A. (2002). Mental retardation: Definition, classification, and system of supports (10th ed.). Washington, DC: AAMR.
31. Mele, M. L., Federici, S., Borsci, S., & Liotta, G. (2010). Beyond a visuocentric way of a visual web search clustering engine: The sonification of WhatsOnWeb. In Miesenberger, K., Klaus, J., Zagler, W., & Karshmer, A. (Eds.), Computers Helping People with Special Needs (Vol. 1, pp. 351-357). Berlin, Germany: Springer. doi:10.1007/978-3642-14097-6_56
32. Nielsen, J. (1994). Heuristic evaluation. In Nielsen, J., & Mack, R. L. (Eds.), Usability inspection methods. New York, NY: John Wiley & Sons.
33. th, 2011, from http://www.useit.com/alertbox/20000319.html.
34. Nielsen, J., & Landauer, T. K. (1993). A mathematical model of the finding of usability problems. Paper presented at the Conference on Human factors in computing systems: INTERACT and CHI 1993. Amsterdam, The Netherlands.
35. Nielsen, J., & Mack, R. L. (Eds.). (1994). Usability inspection methods. New York, NY: John Wiley & Sons.
36. Norman, D. A. (1988). The psychology of everyday things. New York, NY: Basic Books.
37. Schalock, R. L., & Luckasson, R. (2004). American association on mental retardation's definition, classification, and system of supports and its relation to international trends and issues in the field of intellectual disabilities. Journal of Policy and Practice in Intellectual Disabilities, 1(3-4), 136-146. doi:10.1111/j.1741-1130.2004.04028.x
38. nd British HCI Group Annual Conference on People and Computers: Culture, Creativity. Liverpool, UK.
39. Spool, J., & Schroeder, W. (2001). Testing web sites: Five users is nowhere near enough. Paper presented at the Human Factors in Computing Systems: CHI 2001. Seattle, WA.
40. Stephanidis, C. (2001). User interfaces for all: New perspectives into human-computer interaction. In Stephanidis, C. (Ed.), User Interfaces for All: Concepts, Methods, and Tools (pp. 3-17). Mahwah, NJ: Lawrence Erlbaum Associates.
41. Virzi, R. A. (1990). Streamlining the design process: Running fewer subjects. Human Factors and Ergonomics Society Annual Meeting Proceedings, 34, 291-294.
42. Virzi, R. A. (1992). Refining the test phase of usability evaluation: How many subjects is enough? Human Factors, 34(4), 457-468.
43. Wright, P. C., & Monk, A. F. (1991). A cost-effective evaluation method for use by designers. International Journal of Man-Machine Studies, 35(6), 891-912. doi:10.1016/S0020-7373(05)80167-1