Measuring web usability using item response theory: Principles, features and opportunities

Interacting with Computers

Volumn 23, Issue 2, 2011, Pages 167-175

  Tezza, Rafael ^a   Bornia, Antonio Cezar ^a   Andrade, Dalton Francisco De ^a  

^a FEDERAL UNIVERSITY OF SANTA CATARINA   (Brazil)

Author keywords

E commerce; Item response theory; Usability checklist

Indexed keywords

ELECTRONIC COMMERCE; ERGONOMICS; WEB DESIGN; WEBSITES;

CRITICAL ISSUES; E-COMMERCE WEBSITES; ITEM RESPONSE THEORY; RELIABILITY AND VALIDITY; TOOLS FOR USABILITY EVALUATION; USABILITY CHECKLIST; WEB USABILITY;

USABILITY ENGINEERING;

EID: 79953065581     PISSN: 09535438     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.intcom.2011.02.004     Document Type: Article

References (75)

1. J. Alexander, and M.A. Tate Web Wisdom: How to Evaluate and Create Information Quality on the Web 1999 Lawrence Erlbaum Mahwah, New Jersey
2. D.F. Andrade, H.R. Tavares, and R.C. Valle Item Response Theory: Concepts and Applications 2000 Associaão Brasileira de Estatística São Paulo (in Portuguese)
3. D. Andrich, and G. Luo A hyperbolic cosine latent trait model for unfolding dichotomous single-stimulus responses Applied Psychological Measurement 17 3 1993 253 276
4. E. Babbie The Basics of Social Research 2005 Wadsworth Publishing Belmont, CA
5. D.L. Baker Advancing E-Government performance in the United States through enhanced usability benchmarks Government Information Quarterly 26 1 2009 82 88
6. F.B. Baker, and S.-H. Kim Item Response Theory: Parameter Estimation Techniques second ed. 2004 Marcel Dekker New York
7. E. Barriocanal, M.S. Urbán, L. González, and J. Hilera Dialogue-based design of web usability questionnaires using ontologies Computer-Aided Design of User Interfaces IV 2005 Part 4 2005 133 146
8. Bastien, J.M.C., Scapin, D.L., 1993. Ergonomic criteria for the evaluation of human-computer interfaces. Rapport technique de l'INRIA. http://www.inria.fr/rrrt/rt-0156.html > (accessed March 2009).
9. A. Birnbaum Some latent trait models and their use in inferring an examinee's ability, contributed chapters F.M. Lord, M.R. Novick, Statistical Theories of Mental Test Scores 1968 Addison Wesley Reading, MA
10. J. Brooke SUS - a "quick and dirty" usability scale P. Jordan, B. Thomas, B. Weerdmeester, A. McClelland, Usability Evaluation in Industry 1996 Taylor Francis London
11. Byrne, M.D., Anderson, J.R., DouglasS, S., Matessa, M., 1999. Eye tracking the visual search of click-down menus. In: Proceedings of CHI'99, pp. 402-409.
12. C.S. Campbell, and P.P. Maglio Facilitating navigation in information spaces: road signs on the World Wide Web International Journal of Human-Computer Studies 50 4 1999 309 327
13. Chaparro, B.S., Bernard, M.L., 2001. Finding information on the Web: does the amount of whitespace really matter? In: Proceedings of the Tenth Annual Usability Professionals' Association Conference 2001.
14. L.A. Clark, and D.B. Watson Constructing validity: basic issues in scale development Psychological Assessment 7 3 1995 309 319
15. Courville, T.G., 2004. An empirical comparison of item response theory and classical test theory item/person statistics. Doctoral dissertation, Texas A&M University.
16. R.J. de Ayala The Theory and Practice of Item Response Theory 2009 Guilford Press New York
17. R.J. de Ayala, B.G. Dodd, and W.R. Koch A computerized simulation of a flexilevel test and its comparison with a Bayesian computerized adaptive test Journal of Educational Measurement 27 4 1990 227 239
18. Desurvire, H.W., Kondziela, J.M., Atwood, M.E., 1992. What is gained and lost when using evaluation methods other than empirical testing 89-102. In: Monk, A., Diaper, D., Harrison, M.D. (Eds.), People and Computers VII. Cambridge University Press, Cambridge, pp. 89-102 (A Shorter Version of this Paper is Available in the Digest of Shot Talks Presented at CHI'92, Monterey, CA, May 7, pp. 125-126).
19. Detweiler, M.C., Omanson, R.C., 1996. Ameritech Web Page User Interface Standards and Design Guidelines. SBC.
20. Díaz, P., Sicilia M.A., Aedo, I., 2002. Evaluation of hypermedia educational systems: criteria ad imperfect measures. In: Proc. of Intl. Conference on Computers in Education 2002, pp. 621-626.
21. M. Du Toit IRT from SSI: BILOG-MG, MULTILOG, PARSCALE, TESTFACT 2003 Scientific Software International Lincolnwood, IL
22. J.S. Dumas, and J.C. Redish A Practical Guide to Usability Testing 1993 Ablex Publishing Corp. Norwood, NY
23. S. Embretson, and S.P. Reise Item Response Theory for Psychologists 2000 Lawrence Erlbaum Associates, Inc. Publishers Mahwah, New Jersey ISBN 0805828192
24. X. Fan Item response theory and classical test theory: an empirical comparison of their item/person statistics Educational and Psychological Measurement 58 3 1998 357 381
25. D. Fink, and C. Nyaga Evaluating web site quality: the value of a multi paradigm approach Benchmarking: An International Journal 16 2 2009 259 273
26. V. Flanders, and M. Willis Web Pages that Suck: Learn Good Design by Looking at Bad Design 1998 SYBEX San Francisco, CA
27. G. Gediga, K.-C. Hamborg, and I. Duntsch The IsoMetrics Usability Inventory: an operationalisation of ISO 9241-10 supporting summative and formative evaluation of software systems Behaviour and Information Technology 18 3 1999 151 164
28. K. Goto, and E. Cotler Web Redesign: Workflow that Works 2005 New Riders Berkeley, CA
29. R. Hambelton, H. Swaminathan, and H. Rogers Fundamentals of Item Response Theory 1991 Sage Newbury Park, CA
30. R.K. Hambleton Emergence of item response modeling in instrument development and data analysis Medical Care 38 9 2000 60 65
31. HHS - US Department of Health and Human Services, 2006. The Research-Based Web Design & Usability Guidelines (Guidelines) Washington, DC, US < http://www.usability.gov/guidelines/index.html > (accessed March 2009).
32. IEEE Std. 1061, 1998. Software Quality Metrics Methodology, IEEE Computer Society.
33. ISO 9241, 1998. Ergonomic requirements for office work with visual display terminals (VDTs). Part 11 Guidance on usability; Draft International Standard ISO.
34. S. Janda, P.J. Trocchia, and K. Gwinner Consumer perceptions of internet retail service quality International Journal of Service Industry Management 13 5 2002 412 431
35. Jeffries, R.J., Miller, J.R., Wharton, C., Uyeda, K.M., 1991. User interface evaluation in the real world: a comparison of four techniques. In: P. ACM CHI'91 Conference, New Orleans, LA, pp. 119-124.
36. S. Kim, and L. Stoel Dimensional hierarchy of retail website quality Information and Management 41 5 2004 619 633
37. Kirakowski, J., Cierlik, B., 1998. Measuring the usability of web sites. In: Proceeding of Human Factors and Ergonomics Society 42nd Annual Meeting, Santa Monica, CA.
38. U. Konradt, H. Wandke, B. Balazs, and T. Christophersen Usability in online shops: scale construction, validation and the influence on the buyers' intention and decision Behaviour & Information Technology 22 3 2003 165 174
39. H. Lim, and A.J. Dubinsky Consumers' perception of e-shopping characteristics: an expectancy-value approach Journal of Services Marketing 18 7 2004 500 513
40. T.H. Lin, and G. Yao Evaluating item discrimination power of WHOQOLBREF from an item response model perspectives Social Indicators Research 91 2 2008 141 153
41. H.X. Lin, Y.-Y. Choong, and G. Salvendy A proposed index of usability: a method for comparing the relative usability of different software systems Behaviour & Information Technology 16 4 1997 267 278 (Pubitemid 127623944)
42. Lord, F.M., 1952. A theory of test scores. Psychometric Monograph 7.
43. Lynch, P.J., Horton, S., 2002. Web Style Guide, second ed. Yale University, New Haven, CO. < http://www.webstyleguide.com/index.html?/ contents.html > (accessed February 2009).
44. D.J. Mislevy, and R.D. Bock BILOG: Item Analysis and Test Scoring with Binary Logistic Models [Computer Program] 1990 Scientific Software Chicago
45. E. Muraki, and R.D. Bock PARSCALE: IRT-based Test Scoring and Item Analysis for Graded Open-ended Exercises and Performance Tasks 1993 Scientific Software Chicago
46. Nielsen, J., 1992. Finding usability problems through heuristic evaluation. Paper presented at CHI '92 pp. 373-380. (Pubitemid 23609817)
47. Nielsen, J., 1996. Accessible design for users with disabilities. October, 1996. < http://www.useit.com/alertbox/9610.html > (accessed February 2009).
48. Nielsen, J., 1997. The need for speed. March, 1997. < http://www.useit.com/alertbox/9703a.html > (accessed February 2009).
49. Nielsen, J., Loranger, H., 2006. Prioritizing Web Usability. New Riders, California, US, California.
50. J. Nielsen, and M. Tahir Homepage Usability: 50 Sites Deconstructed 2002 New Riders Publishing Indianapolis, IN
51. Nielsen, J., Molich, R., Snyder, C., Farrell, S., 2001. E-Commerce User Experience: High-level Strategy. Nielsen Norman Group.
52. Nygren, E., Allard, A., 1996. Between the clicks: skilled users scanning of pages. In: Proceedings of the 2nd Conference on Human factors Web. < http://www.microsoft.com/usability/webconf/nygren.rtf > (accessed November 2008).
53. Okada, H., 2001. Usability and its evaluation technique, The Institute of Systems. Control and Information Engineers 45(5) 269-276.
54. J. Olea, F.J. Abad, V. Ponsoda, and M.C. Ximénez Un test adaptativo informatizado para evaluar el conocimiento de inglés escrito: diseño y comprobaciones psicométricas Psicothema 16 3 2004 519 525 (Pubitemid 41310389)
55. A. Piolat, J.Y. Roussey, and O. Thunin Effects of screen presentation on text reading and revising International Journal of Human Computer Studies 47 4 1998 565 589
56. R Development Core Team, 2006. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria.
57. M.D. Reckase Adaptive testing: the evolution of a good idea Educational Measurement: Issues and Practice 8 3 1989 11 15
58. M.D. Reckase The past and future of multidimensional item response theory Applied Psychological Measurement 21 1 1997 25 36
59. M.D. Reckase Multidimensional Item Response Theory 2009 Springer New York, NY
60. S.P. Reise, K.F. Widaman, and R.H. Pugh Confirmatory factor analysis and item response theory: two approaches for exploring measurement invariance Psychological Bulletin 114 3 1993 552 566
61. J.S. Roberts, J.R. Donoghue, and J.E. Laughlin A general item response theory model for unfolding unidimensional polytomous responses Applied Psychological Measurement 24 1 2000 3 32
62. D.A. Santor, J.O. Ramsay, and D.C. Zuroff Nonparametric item analyses of the Beck Depression Inventory: evaluating gender item bias and response option weights Psychological Assessment 6 3 1994 255 270 (Pubitemid 24309294)
63. Schmettow, M., Vietze, W., 2008. Introducing item response theory for measuring usability inspection processes. In: Proceeding of the Twenty-sixth Annual SIGCHI Conference on Human Factors in Computing Systems, April 05-10, Florence, Italy.
64. A. Seffah, and E. Metzker The obstacles and myths of usability and software engineering Communications of the ACM 47 12 2004 71 76
65. J. Singh Tackling measurement problems with Item Response Theory: principles, characteristics, and assessment, with an illustrative example Journal of Business Research 57 2 2004 184 208 (Pubitemid 37527907)
66. J.H. Song, and G.M. Zinkhan Features of web site design, perceptions of web site quality, and patronage behaviour ACME 2003 2003 106 114
67. J.M. Spool, T. Scanlon, W. Schroeder, C. Snyder, and T. DeAngelo Web Site Usability: A Designer's Guide 1997 User Interface Engineering North Andover, MA
68. J.H. Spyridakis Guidelines for authoring comprehensible web pages and evaluating their success Technical Communication 47 3 2000 359 382
69. W. Tan, D. Liu, and R. Bishu Web evaluation: Heuristic evaluation vs. user testing International Journal of Industrial Ergonomics 39 4 2009 621 627
70. H.R. Tavares, D.F. Andrade, and C.A.B. Pereira Detection of determinant genes and diagnostic via item response theory Genetics and Molecular Biology 27 4 2004 679 685 (Pubitemid 40074378)
71. D. Thissen Multilog: Multiple, Categorical Item Analysis and Test Scoring: Using Item Response Theory 1991 Chicago Scientific Software, Inc
72. T. Uttaro, and A. Lehman Graded response modeling of the Quality of Life Interview Evaluation and Program Planning 22 1 1999 41 52 (Pubitemid 29324293)
73. W.J. van der Linden, and C.A.W. Glas Computerized Adaptive Testing: Theory and Practice 2000 Kluwer Dordrecht, the Netherlands
74. Y. Wang, T. Tang, and J.E. Tang An instrument for measuring customer satisfaction toward web sites that market digital products and services Journal of Electronic Commerce Research 2 3 2001 89 102
75. P. Zhang, and G.M.V. Dran User expectation and rankings of quality factors in different web site domains International Journal of Electronic Commerce 6 2 2002 9 33