Measuring the User Experience: Collecting, Analyzing, and Presenting Usability Metrics: Second Edition

Measuring the User Experience: Collecting, Analyzing, and Presenting Usability Metrics: Second Edition

Volumn , Issue , 2013, Pages 1-301

  Tullis, Tom ^a   Albert, Bill ^a  

^a BENTLEY UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84902038845     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.1016/C2011-0-00016-9     Document Type: Book

References (158)

1. Bradner, E. (2010,). Recommending net promoter. Retrieved on 23.10.2011 from DUX: Designing the User Experience at Autodesk <>. http://dux.typepad.com/dux/2010/11/recommending-net-promoter.html.
2. Reichheld, F. (2003). The one number you need to grow. Harvard Business Review.
3. Owen R., Brooks L. Answering the ultimate question 2008, Jossey-Bass, San Francisco.
4. Sauro, J. (2011). Usability and net promoter benchmarks for consumer software. Retrieved on 23.10.2011 from Measuring Usability <>. http://www.measuringusability.com/software-benchmarks.php.
5. Sauro, J., & Kindlund, E. (2005). Using a single usability metric (SUM) to compare the usability of competing products. In Proceeding of the human computer interaction international conference (HCII 2005), Las Vegas, NV.
6. Guan, H., Theofanos, M., Choong, Y. Y., & Stanton, B. (2011). Real-time feedback for usable fingerprint systems. International Joint Conference on Biometrics (IJCB), pp. 1-8.
7. International Organization for Standards (ISO)(1998). 9241-11 Ergonomic requirements for office work with visual display terminals (VDTs). Part 11. Guidance on usability. Geneva, Switzerland.
8. Stanton, B., Theofanos, M., Steves, M., Chisnell, D., & Wald, H. (2012). Fingerprint scanner affordances, NISTIR (to be published), National Institute of Standards and Technology, Gaithersburg, MD.
9. Tabassi, E., Wilson, C., & Watson, C. (2004). Fingerprint image quality. NISTIR 7151, National Institute of Standards and Technology, Gaithersburg, MD <>. http://www.nist.gov/customcf/get_pdf.cfm?pub_id=905710.
10. Theofanos, M., Stanton, B., Sheppard, C., Micheals, R., Zhang, N. F., Wydler, J., et al. (2008). Usability testing of height and angles of ten-print fingerprint capture, NISTIR 7504, National Institute of Standards and Technology, Gaithersburg, MD.
11. Bergstrom J., Olmsted-Hawala E., Chen J., Murphy E. Conducting iterative usability testing on a web site: Challenges and benefits. Journal of Usability Studies 2011, 7(1):9-30.
12. ISO 9241-124. Ergonomics of human-system interaction.
13. Sauro, J., & Dumas, J. S. (2009). Comparison of three one-question, post-task usability questionnaires. Computer human interaction conference. <)>. http://www.measuringusability.com/papers/Sauro_Dumas_CHI2009.pdf.
14. Sauro J., Lewis J.R. Quantifying the user experience: practical statistics for user research 2012, Morgan Kaufmann.
15. Jarrett C., Roddis I. How to obtain maximum insight by cross-referring site statistics, focus groups and usability techniques. Web based surveys and usability testing 2002, Institute for International Research, San Francisco, CA.
16. Pruitt J., Adlin T. The persona lifecycle: Keeping people in mind throughout product design 2006, Morgan Kaufmann, San Francisco.
17. Barnum C., Palmer L. More than a feeling: Understanding the desirability factor in user experience. CHI 2010, 4703-4715.
18. Benedek J., Miner T. Measuring desirability: New methods for evaluating desirability in a usability lab setting. Proceedings of Usability Professionals Association 2002, 8-12.
19. Picard R. Emotion research by the people, for the people. Emotion Review 2010, 2:250-254.
20. Setz C., Arnrich B., Schumm J., La Marca R., Tröster G., Ehlert U. Discriminating stress from cognitive load using a wearable EDA device. IEEE Transactions on Information Technology in Biomedicine 2010, 14(2):410-417.
21. Albert, W., & Dixon, E. (2003). Is this what you expected? The use of expectation measures in usability testing. Proceedings of Usability Professionals Association 2003 Conference, Scottsdale, AZ.
22. Albert, W., Gribbons, W., & Almadas, J. (2009). Pre-conscious assessment of trust: a case study of financial and health care web sites. Human factors and ergonomics society annual meeting proceedings, 53, 449-453. Also http://www.measuringux.com/Albert_Gribbons_Preconsciousness.pdf.
23. Albert W., Tedesco D. Reliability of self-reported awareness measures based on eye tracking. Journal of Usability Studies 2010, 5(2):50-64.
24. Aldenderfer M., Blashfield R. Cluster analysis (quantitative applications in the social sciences) 1984, Sage Publications, Inc., Beverly Hills, CA.
25. American Institutes for Research. (2001). Windows XP Home Edition vs. Windows Millennium Edition (ME) public report. New England Research Center, Concord, MA. Available at http://download.microsoft.com/download/d/8/1/d810ce49-d481-4a55-ae63-3fe2800cbabd/ME_Public.doc.
26. Andre, A. (2003). When every minute counts, all automatic external defibrillators are not created equal. Published in June, 2003 by Interface Analysis Associates http://www.usernomics.com/iaa_aed_2003.pdf.
27. Bangor A., Kortum P., Miller J.A. Determining what individual SUS scores mean: adding an adjective rating scale. Journal of Usability Studies 2009, 4:3.
28. Bargas-Avila, J. A. & Hornbæk, K. (2011). Old wine in new bottles or novel challenges? a critical analysis of empirical studies of user experience, CHI '11 Proceedings of the 2011 annual conference on human factors in computing systems, 2689-2698.
29. Barnum, C., Bevan, N., Cockton, G., Nielsen, J., Spool, J., & Wixon, D. (2003). The "magic number 5": is it enough for web testing? 2003, April 5-10, Ft. Lauderdale, FL: CHI.
30. Benedek, J., & Miner, T. (2002). Measuring desirability: new methods for evaluating desirability in a usability lab setting. Usability professionals association 2002 conference, Orlando, FL, July 8-12. Also available at . Also see the appendix listing the Product Reaction Cards at http://www.microsoft.com/usability/UEPostings/ProductReactionCards.doc.
31. Bias R., Mayhew D. Cost-justifying usability, Second edition: an update for the Internet age 2005, Morgan Kaufmann, San Francisco.
32. Birns, J., Joffre, K., Leclerc, J., & Paulsen, C. A. (2002). Getting the whole picture: Collecting usability data using two methods - concurrent think aloud and retrospective probing. Proceedings of the 2002 Usability Professionals' Association Conference, Orlando, FL. Available from http://concordevaluation.com/papers/paulsen_thinkaloud_2002.pdf.
33. Breyfogle F. Implementing six sigma: smarter solutions using statistical methods 1999, John Wiley and Sons, New York.
34. Brooke J. SUS: a quick and dirty usability scale. Usability evaluation in industry 1996, Taylor & Francis, London. P.W. Jordan, B. Thomas, B.A. Weerdmeester, I.L. McClelland (Eds.).
35. Burby J., Atchison S. Actionable web analytics: using data to make smart business decisions 2007, Sybex, Indianapolis, IN.
36. Card S.K., Moran T.P., Newell A. The psychology of human-computer interaction 1983, Lawrence Erlbaum Associates, London.
37. Catani, M., & Biers, D. (1998).Usability evaluation and prototype fidelity. In Proceedings of the human factors and ergonomic society.
38. Chadwick-Dias, A., McNulty, M., & Tullis, T. (2003). Web usability and age: how design changes can improve performance. Proceedings of the 2003 ACM conference on universal usability, Vancouver, BC, Canada.
39. Chin, J. P., Diehl, V. A., & Norman, K. L. (1988). Development of an instrument measuring user satisfaction of the human-computer interface. ACM CHI'88 proceedings, 213-218.
40. Clifton B. Advanced web metrics with Google analytics 2012, Sybex, Indianapolis, IN.
41. Cockton, G., & Woolrych, A. (2001). Understanding inspection methods: lessons from an assessment of heuristic evaluation. Joint Proceedings of HCI and IHM: people and computers, XV.
42. Cox E.P. The optimal number of response alternatives for a scale: a review. Journal of Marketing Research 1980, 17(4):407-422.
43. Cunningham K. The accessibility handbook 2012, O'Reilly Media, Sebastopol, CA.
44. Dennerlein, J., Becker, T., Johnson, P., Reynolds, C. J., & Picard, R. W. (2003). Frustrating computer users increases exposure to physical factors. In Proceedings of the international ergonomics association, August 24-29, Seoul.
45. Dillman, D. A., Phelps, G., Tortora, R., Swift, K., Kohrell, J., Berck, J., et al. (2008). Response rate and measurement differences in mixed mode surveys using mail, telephone, interactive voice response, and the internet. Available at http://www.sesrc.wsu.edu/dillman/papers/2008/ResponseRateandMeasurement.pdf.
46. Ekman P., Friesen W. Unmasking the face 1975, Prentice-Hall, Englewood Cliffs, NJ.
47. Everett, S. P., Byrne, M. D., & Greene, K. K. (2006). Measuring the usability of paper ballots: efficiency, effectiveness, and satisfaction. Proceedings of the human factors and ergonomics society 50th annual meeting. Santa Monica, CA: Human Factors and Ergonomics Society.
48. Few S. Information dashboard design: the effective visual communication of data 2006, O'Reilly Media, Inc., Sebastopol, CA.
49. Few S. Now you see it: simple visualization techniques for quantitative analysis 2009, Analytics Press, Oakland, CA.
50. Few S. Show me the numbers: designing tables and graphs to enlighten 2012, Analytics Press, Oakland, CA. 2nd ed.
51. Finstad K. Response interpolation and scale sensitivity: evidence against 5-point scales. Journal of Usability Studies 2010, 5(3):104-110.
52. Fogg, B. J., Marshall, J., Laraki, O., Osipovich, A., Varma, C., Fang, N., et al. (2001). What makes web sites credible? a report on a large quantitative study. Proceedings of CHI'01, human factors in computing systems, 61-68.
53. Foraker. (2010). Usability ROI case study: discussion forums. Retrieved 4/18/2013 from http://www.usabilityfirst.com/documents/U1st_BCO_CaseStudy.pdf.
54. Foresee. (2012). ACSI e-government satisfaction index (Q4 2012). http://www.foreseeresults.com/research-white-papers/_downloads/acsi-egov-q4-2012-foresee.pdf.
55. Friedman H.H., Friedman L.W. On the danger of using too few points in a rating scale: a test of validity. Journal of Data Collection 1986, 26(2):60-63.
56. Garland R. The mid-point on a rating scale: is it desirable?. Marketing Bulletin 1991, (2):66-70. Research Note 3.
57. Guan, Z., Lee, S., Cuddihy, E., & Ramey, J. (2006). The validity of the stimulated retrospective think-aloud method as measured by eye tracking. In Proceedings of the ACM SIGCHI conference on human factors in computing systems, 2006 (pp.1253-1262). New York, New York, USA. ACM Press. Available from http://dub.washington.edu:2007/pubs/chi2006/paper285-guan.pdf.
58. Gwizdka J., Spence I. Implicit measures of lostness and success in web navigation. Interacting with Computers 2007, 19(3):357-369.
59. Hart, T. (2004). Designing "senior friendly" websites: do guidelines help? Usability News, 6.1. http://psychology.wichita.edu/surl/usabilitynews/61/older_adults-withexp.htm.
60. Henry S.L. Just ask: integrating accessibility throughout design 2007, Lulu.com, Raleigh, NC.
61. Hertzum, M., Jacobsen, N., & Molich, R. (2002). Usability inspections by groups of specialists: perceived agreement in spite of disparate observations. CHI, Minneapolis.
62. Hewett T.T. The role of iterative evaluation in designing systems for usability. People and computers: designing for usability 1986, 196-214. Cambridge University Press, Cambridge. M.D. Harrison, A.F. Monk (Eds.).
63. Holland, A. (2012a). Ecommerce button copy test: did 'Personalize Now' or 'Customize It' get 48% more revenue per visitor? Retrieved on 4/18/2013 from http://whichtestwon.com/archives/14511.
64. Holland, A. (2012b). Online newspaper layout test: should photos alternate sides or always appear to the right of stories? Retrieved on 4/18/2013 from https://whichtestwon.com/archives/18744.
65. Hornbæk K., Frøkjær E. A study of the evaluator effect in usability testing. Human-Computer Interaction 2008, 23(3):251-277.
66. Human Factors International. (2002). HFI helpsstaples.comboost repeat customers by 67%. Retrieved 4/18/2013 from http://www.humanfactors.com/downloads/documents/staples.pdf.
67. Hyman I.E., Boss S.M., Wise B.M., McKenzie K.E., Caggiano J.M. Did you see the unicycling clown? Inattentional blindness while walking and talking on a cell phone. Applied Cognitive Psychology 2010, 24:597-607.
68. ISO/IEC 25062 (2006). Software engineering - Software product Quality Requirements and Evaluation (SQuaRE) - Common Industry Format (CIF) for usability test reports.
69. Jacobsen, N., Hertzum, M., & John, B. (1998). The evaluator effect in usability studies: problem detection and severity judgments. In Proceedings of the human factors and ergonomics society.
70. Kapoor, A., Mota, S., & Picard, R. (2001). Towards a learning companion that recognizes affect. AAAI Fall Symposium, November, North Falmouth, MA.
71. Kaushik A. Web analytics 2.0: the art of online accountability and science of customer centricity 2009, Sybex, Indianapolis, IN.
72. Kirkpatrick A., Rutter R., Heilmann C., Thatcher J., Waddell C. Web accessibility: web standards and regulatory compliance 2006, Apress Media, New York, NY.
73. Kohavi, R., Crook, T., & Longbotham, R. (2009). Online experimentation at Microsoft, Third workshop on Data Mining Case Studies and Practice. Retrieved on 4/18/2013 from http://robotics.stanford.edu/~ronnyk/ExP_DMCaseStudies.pdf.
74. Kohavi, R., Deng, A., Frasca, B., Longbotham, R., Walker, T., & Xu, Y. (2012). Trustworthy online controlled experiments: five puzzling outcomes explained. In Proceedings of the 18th ACM SIGKDD international conference on knowledge discovery and data mining (KDD '12). ACM, New York, NY, USA, 786-794.
75. Kohavi, R., & Round, M. (2004). Front line internet analytics atAmazon.com. Presentation at Emetrics Summit 2004. Retrieved on 4/18/2013 from http://ai.stanford.edu/~ronnyk/emetricsAmazon.pdf.
76. Committee on quality of health care in America, institute of medicine. "To err is human: building a safer health system." 2000, National Academies Press, Washington, DC. L.T. Kohn, J.M. Corrigan, M.S. Donaldson (Eds.).
77. Kruskal J., Wish M. Multidimensional scaling (quantitative applications in the social sciences) 2006, Sage Publications, Inc., Beverly Hills, CA.
78. Kuniavsky M. Observing the user experience: a practitioner's guide to user research 2003, Morgan Kaufmann, San Francisco.
79. LeDoux, L., Mangan, E., & Tullis, T. (2005). Extreme makeover: UI edition. Presentation at Usability Professionals Association (UPA) 2005 Annual Conference, Montreal, QUE, Canada. Available from http://www.upassoc.org/usability_resources/conference/2005/ledoux-UPA2005-Extreme.pdf.
80. Lewis J. Sample sizes for usability studies: additional considerations. Human Factors 1994, 36:368-378.
81. Lewis J.R. Psychometric evaluation of an after-scenario questionnaire for computer usability studies: the ASQ. SIGCHI Bulletin 1991, 23(1):78-81. Also see .
82. Lewis J.R. IBM computer usability satisfaction questionnaires: psychometric evaluation and instructions for use. International Journal of Human-Computer Interaction 1995, 7(1):57-78. Also see http://www.acm.org/~perlman/question.cgi?form=CSUQ.
83. Lewis, J. R. & Sauro, J. (2009). The factor structure of the system usability scale. Proceedings of the human computer interaction international conference (HCII 2009), San Diego CA, USA.
84. Likert R. A technique for the measurement of attitudes. Archives of Psychology 1932, 140:55.
85. Lin, T., Hu, W., Omata, M., & Imamiya, A. (2005). Do physiological data relate to traditional usability indexes? In Proceedings of OZCHI2005, November 23-25, Canberra, Australia.
86. Lindgaard, G., & Chattratichart, J. (2007). Usability testing: what have we overlooked? In Proceedings of ACM CHI conference on human factors in computing systems.
87. Lindgaard G., Fernandes G., Dudek C., Brown J. Attention web designers: you have 50 milliseconds to make a good first impression!. Behaviour & Information Technology 2006, 25:115-126.
88. Lund, A. (2001). Measuring usability with the USE questionnaire. Usability and user experience newsletter of the STC Usability SIG. See http://www.stcsig.org/usability/newsletter/0110_measuring_with_use.html.
89. Martin P., Bateson P. Measuring behaviour 1993, Cambridge University Press, Cambridge, UK, and New York. 2nd ed.
90. Maurer, D., & Warfel, T. (2004). Card sorting: a definitive guide. Boxes and Arrows, April 2004. Retrieved on 4/18/2013 from http://boxesandarrows.com/card-sorting-a-definitive-guide/.
91. Mayhew D., Bias R. Cost-justifying usability 1994, Morgan Kaufmann, San Francisco.
92. McGee, M. (2003). Usability magnitude estimation. Proceedings of human factors and ergonomics society annual meeting, Denver, CO.
93. McLellan S., Muddimer A., Peres S.C. The effect of experience on system usability scale ratings. Journal of Usability Studies 2012, 7(2):56-67. .
94. Miner G., Elder J., Hill T., Nisbet R., Delen D., Fast A. Practical text mining and statistical analysis for non-structured text data applications 2012, Elsevier Academic Press, ISBN 978-0-12-386979-1.
95. Molich, R. (2011). CUE-9: The evaluator effect. http://www.dialogdesign.dk/CUE-9.html.
96. Molich, R., Bevan, N., Butler, S., Curson, I., Kindlund, E., Kirakowski, J., et al., (1998). Comparative evaluation of usability tests. Usability professionals association 1998 Conference, 22-26 June 1998 Washington, DC: Usability Professionals Association, pp. 189-200.
97. Molich R., Dumas J. Comparative usability evaluation (CUE-4). Behaviour & Information Technology 2008, 27:263-281.
98. Molich R., Ede M.R., Kaasgaard K., Karyukin B. Comparative usability evaluation. Behaviour & Information Technology 2004, 23(1):65-74.
99. Molich R., Jeffries R., Dumas J. Making usability recommendations useful and usable. Journal of Usability Studies 2007, 2(4):162-179. Available at .
100. Mueller J. Accessibility for everybody: understanding the Section 508 accessibility requirements 2003, Apress Media, New York, NY.
101. Nancarrow C., Brace I. Saying the "right thing": coping with social desirability bias in marketing research. Bristol Business School Teaching and Research Review 2000, (Summer):3.
102. Nielsen J. Usability engineering 1993, Morgan Kaufmann, San Francisco.
103. Nielsen, J. (2000). Why you only need to test with 5 users. AlertBox, March 19. Available at http://www.useit.com/alertbox/20000319.html.
104. Nielsen, J. (2001). Beyond accessibility: treating users with disabilities as people. AlertBox, November 11, 2001. Retrieved on 4/18/2013, from http://www.nngroup.com/articles/beyond-accessibility-treating-users-with-disabilities-as-people/.
105. Nielsen, J. (2005). Medical usability: how to kill patients through bad design, Alertbox, April 11, 2005 http://www.nngroup.com/articles/medical-usability/.
106. Nielsen J., Berger J., Gilutz S., Whitenton K. Return on Investment (ROI) for usability 2008, Nielsen Norman Group, Freemont, CA. 4th ed.
107. Nielsen, J., & Landauer, T. (1993). A mathematical model of the finding of usability problems. ACM proceedings, Interchi 93, Amsterdam.
108. Norgaard, M., & Hornbaek, K. (2006). What do usability evaluators do in practice? An explorative study of think-aloud testing. In Proceedings of designing interactive systems, pp. 209-218. University Park, PA.
109. Osgood C.E., Suci G., Tannenbaum P. The measurement of meaning 1957, University of Illinois Press, Urbana, IL.
110. Otter M., Johnson H. Lost in hyperspace: metrics and mental models. Interacting with Computers 2000, 13:1-40.
111. Petrie, H., & Precious, J. (2010). Measuring user experience of websites: think aloud protocols and an emotion word prompt list. In Proceedings of ACM CHI 2010 Conference on human factors in computing systems, 2010. pp. 3673-3678.
112. Reichheld, F. F. (2003). One number you need to grow. Harvard Business Review, December 2003.
113. Reynolds, C. (2005). Adversarial Uses of Affective Computing and Ethical Implications. Ph.D. Thesis, Massachusetts Institute of Technology, Cambridge. Available at http://affect.media.mit.edu/pdfs/05.reynolds-phd.pdf.
114. Sangster, R. L., Willits, F. K., Saltiel, J., Lorenz, F. O., & Rockwood, T. H. (2001). The effects of Numerical Labels on Response Scales. Retrieved on 3/30/2013 from http://www.bls.gov/osmr/pdf/st010120.pdf.
115. Sauro, J. (2009). Composite operators for keystroke level modeling, Proceedings of the human computer interaction international conference (HCII 2009), San Diego CA, USA.
116. Sauro, J. (2010). Does better usability increase customer loyalty? The net promoter score and the system usability scale (SUS). Retrieved on 4/1/2013 from http://www.measuringusability.com/usability-loyalty.php.
117. Sauro, J. & Dumas J. (2009). Comparison of three one-question, post-task usability questionnaires, Proceedings of the conference on human factors in computing systems (CHI 2009), Boston, MA.
118. Sauro, J., & Kindlund, E. (2005). A method to standardize usability metrics into a single score. Proceedings of the conference on human factors in computing systems (CHI 2005), Portland, OR.
119. Sauro, J., & Lewis, J. (2005). Estimating completion rates from small samples using binomial confidence intervals: comparisons and recommendations. Proceedings of the human factors and ergonomics society annual meeting, Orlando, FL.
120. Sauro, J., & Lewis, J. R. (2011). When designing usability questionnaires, does it hurt to be positive?, Proceedings of the conference on human factors in computing systems (CHI 2011), Vancouver, BC, Canada.
121. Schwarz N., Knäuper B., Hippler H.J., Noelle-Neumann E., Clark F. Rating scales: numeric values may change the meaning of scale labels. Public Opinion Quarterly 1991, 55:570-582.
122. Section 508. (1998). Workforce Investment Act of 1998, Pub. L. No. 105-220, 112 Stat. 936 (August 7). Codified at 29 U.S.C. § 794d.
123. Shaikh, A., Baker, J., & Russell, M. (2004). What's the skinny on weight loss websites? Usability News, 6.1, 2004. Available at http://psychology.wichita.edu/surl/usabilitynews/61/diet_domain.htm.
124. Smith P.A. Towards a practical measure of hypertext usability. Interacting with Computers 1996, 8(4):365-381.
125. Snyder, C. (2006). Bias in usability testing. Boston Mini-UPA Conference, March 3, Natick, MA.
126. Sostre P., LeClaire J. Web analytics for dummies 2007, Wiley, Hoboken, NJ.
127. Spencer D. Card sorting: designing usable categories 2009, Rosenfeld Media, Brooklyn, NY.
128. Spool, J., & Schroeder, W. (2001). Testing web sites: five users is nowhere near enough. CHI 2001, Seattle.
129. Stover, A., Coyne, K., & Nielsen, J. (2002). Designing usable site maps for Websites. Available from http://www.nngroup.com/reports/sitemaps/.
130. Tang, D., Agarwal, A., O'Brien, D., & Meyer, M. (2010). Overlapping experiment infrastructure: more, better, faster experimentation. In Proceedings of the 16th ACM SIGKDD international conference on Knowledge discovery and data mining (KDD '10). ACM, New York, NY, USA, 17-26.
131. Teague, R., De Jesus, K., & Nunes-Ueno, M. (2001). Concurrent vs post-task usability test ratings. CHI 2001 extended abstracts on human factors in computing systems, p. 289-290.
132. Teague, R., DeJesus, K., & Nunes-Ueno, M. (2001). Concurrent vs. post-task usability test ratings. Proceedings of CHI 2001, 289-290.
133. Tedesco, D., & Tullis, T. (2006). A comparison of methods for eliciting post-task subjective ratings in usability testing. Usability Professionals Association (UPA) 2006 annual conference, Broomfield, CO, June 12-16.
134. Trimmel M., Meixner-Pendleton M., Haring S. Stress response caused by system response time when searching for information on the internet: psychophysiology in ergonomics. Human Factors 2003, 45(4):615-621.
135. Tufte E.R. Envisioning information 1990, Graphics Press, Chesire, CT.
136. Tufte E.R. Visual explanations: images and quantities, evidence and narrative 1997, Graphics Press, Chesire, CT.
137. Tufte E.R. The visual display of quantitative information 2001, Graphics Press, Chesire, CT. 2nd ed.
138. Tufte E.R. Beautiful evidence 2006, Graphics Press, Chesire, CT.
139. Tullis, T. S. (1985). Designing a menu-based interface to an operating system. Proceedings of the CHI '85 conference on human factors in computing systems, San Francisco.
140. Tullis, T. S. (1998). A method for evaluating Web page design concepts. Proceedings of CHI '98 conference on computer-human interaction, Los Angeles, CA.
141. Tullis, T. S. (2007). Using closed card-sorting to evaluate information architectures. Usability Professionals Association (UPA) 2007 Conference, Austin, TX. Retrieved on 4/18/2013 from http://www.eastonmass.net/tullis/presentations/ClosedCardSorting.pdf.
142. Tullis, T. S. (2008a). SUS scores from 129 conditions in 50 studies. Retrieved on 3/30/2013 from http://www.measuringux.com/SUS-scores.xls.
143. Tullis, T. S. (2008b). Results of online usability study of Apollo program websites. http://www.measuringux.com/apollo/.
144. Tullis, T. S. (2011). Worst usability issue. Posted July 4, 2011. http://www.measuringux.com/WorstUsabilityIssue/.
145. Tullis, T. S., Mangan, E. C., & Rosenbaum, R. (2007). An empirical comparison of on-screen keyboards. Human factors and ergonomics society 51st annual meeting, October 1-5, Baltimore. Available from http://www.measuringux.com/OnScreenKeyboards/index.htm.
146. Tullis, T. S., & Stetson, J.. (2004). A comparison of questionnaires for assessing Website usability. Usability Professionals Association (UPA) 2004 conference, June 7-11, Minneapolis, MN. Paper available from Slides: http://www.upassoc.org/usability_resources/conference/2004/UPA-2004-TullisStetson.pdf.
147. Tullis, T. S., & Tullis, C. (2007). Statistical analyses of e-commerce websites: can a site be usable and beautiful? Proceedings of HCI international 2007 conference, Beijing, China.
148. Tullis, T. S., & Wood, L. (2004). How many users are enough for a card-sorting study? Proceedings of Usability Professionals Association Conference, June 7-11, Minneapolis, MN. Available from . http://home.comcast.net/~tomtullis/publications/UPA2004CardSorting.pdf.
149. Van den Haak M.J., de Jong M.D.T., Schellens P.J. Employing think-aloud protocols and constructive interaction to test the usability of online library catalogues: a methodological comparison. Interacting with Computers 2004, 16:1153-1170.
150. Vermeern, A., van Kesteren, I., & Bekker, M. (2003). Measuring the evaluator effect in user testing. In M. Rauterber et al. (Eds.), Human-computer interaction-INTERACT'03. pp. 647-654. Published by IOS Press, (c)IFIP.
151. Virzi R. Refining the test phase of the usability evaluation: how many subjects is enough?. Human Factors 1992, 34(4):457-468.
152. Vividence Corp. (2001). Moving on up: improves customer experience. Retrieved October 15, 2001, from http://www.vividence.com/public/solutions/our+clients/success+stories/movecom.htm.
153. Ward R., Marsden P. Physiological responses to different WEB page designs. International Journal of Human-Computer Studies 2003, 59:199-212.
154. Wilson, C., & Coyne, K. P. (2001). Tracking usability issues: to bug or not to bug? Interactions, May-June.
155. Withrow, J., Brinck, T., & Speredelozzi, A. (2000). Comparative usability evaluation for an e-government portal. Diamond Bullet Design Report, #U1-00-2, Ann Arbor, MI., December. Available at http://www.simplytom.com/research/U1-00-2-egovportal.pdf.
156. Wixon, D., & Jones, S. (1992). Usability for fun and profit: a case study of the design of DEC RALLY, Version 2. Digital Equipment Corporation.
157. Wong D. The Wall Street Journal guide to information graphics: the do's and don'ts of presenting data, facts, and figures 2010, W. W. Norton & Company, New York, NY.
158. Woolrych, A., & Cockton, G. (2001). Why and when five test users aren't enough. In Proceedings of IHM-HCI2001, 2, pp. 105-108. Toulouse, France: Cépadues-Éditions.