Usability testing: a review of some methodological and technical aspects of the method

International Journal of Medical Informatics

Volumn 79, Issue 4, 2010, Pages

  Bastien, J M Christian ^a  

^a UNIVERSITÉ DE LORRAINE   (France)

Author keywords

Automatic capture; Diary; Empirical user test; Log files; Mobile applications; Remote usability evaluation; Usability testing; Usability testing tools; User centered design; User events; User testing; User traces; Users

Indexed keywords

LOG FILE; MOBILE APPLICATIONS; REMOTE USABILITY; USABILITY TESTING; USER CENTERED DESIGNS; USER TESTING; USER TESTS; USER TRACE;

HEALTH CARE; TESTING;

USABILITY ENGINEERING;

ARTICLE; COMPUTER INTERFACE; MEDICAL INFORMATICS; METHODOLOGY; MOBILE PHONE; PRIORITY JOURNAL; VIDEOCONFERENCING;

FRANCE; MEDICAL INFORMATICS APPLICATIONS; SOFTWARE; SOFTWARE DESIGN; SOFTWARE VALIDATION;

EID: 77649181361     PISSN: 13865056     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.ijmedinf.2008.12.004     Document Type: Article

References (48)

1. Sainfort F., Jacko J.A., and Booske B.C. Human-computer interaction in health care. In: Jacko J.A., and Sears A. (Eds). The Human-Computer Interaction Handbook (2003), Lawrence Earlbaum Associates, London 808-822
2. Tanguay S., and Heywood P. MyFoodPhone: the start of a mobile health revolution. In: Fogg B.J., and Eckles D. (Eds). Mobile Persuasion. 20 Perspectives on the Future of Behavior Change (2007), Stanford Captology Media, Stanford, CA 21-27
3. Damen E. Simply persuasive: using mobile technology to boost physical activity. In: Fogg B.J., and Eckles D. (Eds). Mobile Persuasion. 20 Perspectives on the Future of Behavior Change (2007), Stanford Captology Media, Stanford, CA 39-44
4. Boland P. Managing chronic disease through mobile persuasion. In: Fogg B.J., and Eckles D. (Eds). Mobile Persuasion. 20 Perspectives on the Future of Behavior Change (2007), Stanford Captology Media, Stanford, CA 45-52
5. Beuscart-Zéphir M.-C., Elkin P., Pelayo S., and Beuscart R. The human factors engineering approach to biomedical informatics projects: state of the art, results, benefits and challenges. Yearbook of Medical Informatics 2007 (2007), Schattauer 109-127
6. Ginsburg G. Human factors engineering: a tool for medical device evaluation in hospital procurement decision-making. Journal of Biomedical Informatics 38 (2005) 213-219
7. Kushniruk A.W., Triola M.M., Borycki E.M., Stein B., and Kannry J.L. Technology induced error and usability: the relationship between usability problems and prescription errors when using a handheld application. International Journal of Medical Informatics 74 (2005) 519-526
8. Beuscart-Zéphir M.-C., Pelayo S., Anceaux F., Meaux J.-J., Degroisse M., and Degoulet P. Impact of CPOE on doctor-nurse cooperation for the medication ordering and administration process. International Journal of Medical Informatics 74 (2005) 629-641
9. Beuscart-Zéphir M.-C., Pelayo S., Degoulet P., Anceaux F., Guerlinger S., and Meaux J.J. A usability study of CPOE's medication administration functions: impact on physician-nurse cooperation. MedInfo Pt 2 (2004) 1018-1022
10. Horsky J., Kaufman D.R., Oppenheim M.I., and Patel V.L. A framework for analysing the cognitive complexity of computer-assisted clinical ordering. Journal of Biomedical Informatics 36 (2003) 4-22
11. Pelayo S., Leroy N., Guerlinger S., Degoulet P., Meaux J.J., and Beuscart-Zéphir M.-C. Cognitive analysis of physicians' medication ordering activity. Studies in Health Technology and Informatics 116 (2005) 929-934
12. International Standards Organisation, ISO 13407, Human centered design processes for interactive systems. International Standards Organisation, Genève, 1999.
13. International Organisation for Standardization, ISO 9241-11, Ergonomic requirements for office work with visual display terminals (VDTs)-Part 11: Guidance on usability, 1998.
14. Rosenbaum S., Rohn J.A., and Humburg J. A toolkit for strategic usability: results from workshops, panels, and surveys. In: Little R., and Nigay L. (Eds). Proceedings of the ACM CHI 2000 Conference on Human Factors in Computing Systems (2000), ACM Press, New York 337-344
15. Usability Professionals' Association, UPA 2007 Salary Survey, Usability Professionals' Association, Bloomingdale, IL, 2008.
16. Cockton G., Lavery D., and Woolrych A. Inspection-based evaluations. In: Jacko J.A., and Sears A. (Eds). The Human-Computer Interaction Handbook. Fundamentals, Evolving Technologies, and Emerging Applications (2003), Lawrence Earlbaum Associates 1118-1138
17. Dumas J.S. User-based evaluations. In: Jacko J.A., and Sears A. (Eds). The Human-Computer Interaction Handbook (2003), Lawrence Earlbaum Associates, Mahwah, New Jersey 1093-1117
18. Dumas J.S., and Redish J.C. A Practical Guide to Usability Testing (1993), Ablex, Norwood, NJ
19. Karat J. User-centered software evaluation methodologies. In: Helander M.G., Landauer T.K., and Prabhu P.V. (Eds). Handbook of Human-Computer Interaction (1997), Elsevier Science Publishers, Amsterdam, The Netherlands 689-704
20. Rubin J. Handbook of Usability Testing. How to Plan, Design and Conduct Effective Tests (1994), Wiley, New York
21. Lewis J.R. Usability testing. In: Salvendy G. (Ed). Handbook of Human Factors and Ergonomics (2006), John Wiley & Sons, New York
22. Tullis T., and Albert B. Measuring the User Experience (2008), Morgan Kauffman, Amsterdam
23. Lewis J.R. Sample sizes for usability tests: mostly math, not magic. Interactions XIII 6 (2006) 29-33
24. Turner C.W., Lewis J.R., and Nielsen J. Determining usability test sample size. In: Karwowski W. (Ed). International Encyclopedia of Ergonomics and Human Factors (2006), CRC Press, Boca Raton, FL 3084-3088
25. Nielsen J., and Landauer T.K. A mathematical model of the finding of usability problems. In: Ashlund S., Mullet K., Henderson A., Hollnagel E., and White T. (Eds). Proceedings of ACM INTERCHI'93 Conference on Human Factors in Computing Systems (1993), ACM, New York 206-213
26. Virzi R.A. Streamlining the design process: running fewer subjects. Proceedings of the Human Factors Society 34th Annual Meeting (1990), The Human Factors and Ergonomics Society, Orlando, FL 291-294
27. Virzi R.A. Refining the test phase of usability evaluation: how many subjects is enough?. Human Factors 34 (1992) 457-468
28. Spool J., and Schroeder W. Testing Web sites: five users is nowhere near enough. Proceedings of the Conference Extended Abstracts on Human Factors in Computing Systems, CHI'2001 (2001), ACM Press, New York
29. O'Malley C.E., Draper S.W., and Riley M.S. Constructive interaction: a method for studying human-computer-human interaction. In: Shackel B. (Ed). Proceedings of IFIP INTERACT'84 (1984), Elsevier Science Publishers, London, U.K. 269-274
30. Wildman D. Getting the most from paired-user testing. Interactions 2 3 (1995) 21-27
31. Westerink J.H.D.M., Rankin P.J., Majoor G.M., and Moore P.S. A new technique for early user evaluation of entertainment product interfaces. Proceedings of the Human Factors and Ergonomics Society 38th Annual Meeting (1994), Human Factors and Ergonomics Society 992
32. C. Wilson, J. Blostein, Usability techniques. Pros and cons of co-participation in usability studies. Usability Interface, 1998.
33. Grondin N., Bastien J.M.C., and Agopian B. Les tests utilisateurs: avantages et inconvénients des passations individuelles et par paires. In: Beaudoin-Lafon M. (Ed). Actes de la Conférence IHM'2002 (2002), ACM, New York, NY 121-128
34. Dray S., and Siegel D. Remote possibilities? International usability testing at a distance. Interactions march+april (2004) 10-17
35. Thompson K.E., Rozanski E.P., and Haake A.R. Here, there, anywhere: remote usability testing that works. SIGITE'04 (2004), ACM, New York 132-137
36. Hong J.I., Heer J., Waterson S., and Landay J.A. WebQuilt: a proxy-based approach to remote Web usability testing. ACM Transactions on Information Systems 19 3 (2001) 263-285
37. Atterer R., Wnuk M., and Schmidt A. Knowing the User's every move-user activity tracking for Website usability evaluation and implicit interaction. Proceedings of the 15th International Conference on World Wide Web (Edinburgh, Scotland, May 23-26) (2006), ACM, New York, NY 203-212
38. Hartson H.R., Castillo J.C., Kelso J., Kamler J., and Neale W.C. Remote evaluation: the network as an extension of the usability laboratory. In: Tauber M.J., Bellotti V., Jeffries R., Mackinlay J.D., and Nielsen J. (Eds). Proceedings of ACM CHI'96 Conference on Human Factors in Computing Systems, Companion (1996), ACM, New York, NY 228-235
39. Tullis T., Fleischman S., McNulty M., Cianchette C., and Bergel M. An empirical comparison of lab and remote usability testing of Web sites. Usability Professional Association Conference. Orlando (2002). http://home.comcast.net/~tomtullis/publications/RemoteVsLab.pdf Available at http://home.comcast.net/∼tomtullis/publications/RemoteVsLab.pdf
40. West R., and Lehman K.R. Automated summative usability studies: an empirical evaluation. CHI 2006 (2006), ACM, New York 631-639
41. Andreasen M.S., Nielsen H.V., Schrøder S.O., and Stage J. What happened to remote usability testing? An empirical study of three methods. CHI 2007 (2007), ACM, New York, NY 1405-1414
42. Noldus L.P.J.J., Trienes R.J.H., Hendriksen A.H.M., Jansen H., and Jansen R.G. The Observer Video-Pro: new software for the collection, management, and presentation of time-structured data from videotapes and digital media files. Behavior Research Methods, Instruments, & Computers 32 (2000) 197-206
43. Uehling D.L., and Wolf K. User action graphing effort (UsAGE). Conference Companion on Human Factors in Computing Systems, CHI'95 (1995), ACM Press, New York, NY 290-291
44. Al-Qaimari G., and McRostie D. KALDI: A CAUsE Tool for supporting testing and analysis of user interaction. In: Blandford A., Vanderdonckt J., and Gray P. (Eds). People and Computers XV. Joint Proceedings of HCI 2001 and IHM 2001 (2001), Springer, London, UK 153-169
45. Ivory M.Y., and Hearst M.A. The state of the art in automating usability evaluation of user interfaces. ACM Computing Surveys 33 (2001) 470-516
46. Schusteritsch R., Wei C.Y., and LaRosa M. Towards the perfect infrastructure for usability testing on mobile devices. CHI 2007 (2007), ACM, New York, NY 1839-1844
47. Rieman J. The diary study: a workplace-oriented research tool to guide laboratory efforts. Proceedings of ACM INTERCHI'93 Conference on Human Factors in Computing Systems (1993), ACM, New York 321-326
48. Palen L., and Salzman M. Voice-mail diary studies for naturalistic data capture under mobile conditions. CSCW'02 (2002), ACM, New York, NY