Quantifying the User Experience: Practical Statistics for User Research, Second Edition

Quantifying the User Experience: Practical Statistics for User Research, Second Edition

Volumn , Issue , 2016, Pages 1-350

  Sauro, Jeff ^a   Lewis, James R ^a  

^a NONE

Author keywords

[No Author keywords available]

Indexed keywords

EID: 85143456016     PISSN: None     EISSN: None     Source Type: Book    
DOI: None     Document Type: Book

References (365)

1. J.R. Lewis and J. Sauro (2016) Excel and R Companion to the Second Edition of “Quantifying the User Experience: Practical Statistics for User Research”: Rapid Answers to over 100 Examples and Exercises. Denver: Create Space Publishers
2. UXPA., 2014. The User Experience Professionals Association salary survey. Available from: https://uxpa.org/system/files/public/UXPASalarySurvey2014_Final.pdf.
3. W. Albert, T. Tullis and D. Tedesco (2010) Beyond the Usability Lab. Burlington, MA: Morgan Kaufmann
4. N.M. Aykin and T. Aykin (1991) Individual differences in human–computer interaction. Comput. Ind. Eng. 20 373-379.
5. Bisbort, A. (1999). Escaping flatland. Available from: http://www.edwardtufte.com/tufte/advocate_1099
6. J. Dickens (1987) The fresh cream cakes market: the use of qualitative research as part of a consumer research programme. U. Bradley (Eds) Applied Marketing and Social Research New York, NY: John Wiley 23-68.
7. J. Dumas and J.C. Redish (1999) A Practical Guide to Usability Testing. Portland, OR: Intellect
8. W. Gordon and R. Langmaid (1988) Qualitative Market Research: A Practitioner’s and Buyer’s Guide. Aldershot, Hampshire, England: Gower Publishing
9. ISO (1998) Ergonomic Requirements for Office Work with Visual Display Terminals (VDTs)—Part 11: Guidance on Usability (ISO 9241-11:1998(E)). Geneva, Switzerland: Author
10. J.R. Lewis (2012) Usability testing. G. Salvendy (Eds) Handbook of Human Factors and Ergonomics fourth ed. New York, NY: John Wiley 1267-1312.
11. Nielsen, J. (2001). Success rate: The simplest usability metric. Available from: http://www.useit.com/alertbox/20010218.html
12. F.F. Reichheld (2003) The one number you need to grow. Harvard Bus. Rev. 81 46-54.
13. F. Reichheld (2006) The Ultimate Question: Driving Good Profits and True Growth. Boston MA: Harvard Business School Press
14. J. Rubin (1994) Handbook of Usability Testing: How to Plan, Design, and Conduct Effective Tests. New York, NY: John Wiley
15. J. Sauro (2010) A Practical Guide to Measuring Usability. Denver, CO: Measuring Usability LLC
16. Sauro, J., 2011. How to quantify comments. Available from: http://www.measuringusability.com/blog/quantify-comments.php
17. Sauro, J., 2013. 4 experiment types for user research. Available from: http://www.measuringu.com/blog/experimenting-ux.php
18. Sauro, J., Kindlund, E., 2005. A method to standardize usability metrics into a single score. Proceedings of CHI 2005. ACM, Portland, OR, pp. 401–409.
19. Sauro, J., Lewis, J.R., 2009. Correlations among prototypical usability metrics: evidence for the construct of usability. Proceedings of CHI 2009. ACM, Boston, MA, pp. 1609–1618.
20. R. Schumacher (2010) The Handbook of Global User Research. Burlington, MA: Morgan Kaufmann
21. M. Scriven (1967) The methodology of evaluation. R.W. Tyler, R.M. Gagne, M. Scriven (Eds) Perspectives of Curriculum Evaluation Chicago, IL: Rand McNally 39-83.
22. T. Tullis and B. Albert (2013) Measuring the User Experience: Collecting, Analyzing, and Presenting Usability Metrics. second ed. Waltham, MA: Morgan Kaufmann
23. A. Agresti and B. Coull (1998) Approximate is better than ‘exact’ for interval estimation of binomial proportions. Am. Stat. 52 119-126.
24. H.M. Blalock (1972) Social Statistics. New York, NY: McGraw-Hill
25. C.J. Clopper and E. Pearson (1934) The use of confidence intervals for fiducial limits illustrated in the case of the binomial. Biometrika 26 404-413.
26. R. Cordes (1993) The effects of running fewer subjects on time-on-task measures. Int. J. Hum. Comput. Int. 5(4), 393-403.
27. P.S. Laplace (1812) Theorie Analytique Des Probabilitites. Paris, France: Courcier
28. J.R. Lewis (1996) Binomial confidence intervals for small sample usability studies. G. Salvendy, A. Ozok (Eds) Advances in Applied Ergonomics: Proceedings of the 1st International Conference on Applied Ergonomics—ICAE ’96 Istanbul, Turkey: USA Publishing 732-737.
29. J.R. Lewis and J. Sauro (2006) When 100% really isn’t 100%: Improving the accuracy of small-sample estimates of completion rates. J. Usability Stud. 3(1), 136-150.
30. Sauro, J. (2004). Restoring confidence in usability results. From Measuring Usability, Available from: http://www.measuringu.com/conf_intervals.htm.
31. Sauro, J., Lewis, J.R. (2005). Estimating completion rates from small samples using binomial confidence intervals: Comparisons and recommendations. In: Proceedings of the Human Factors and Ergonomics Society Forty-Ninth Annual Meeting (pp. 2100–2104). Santa Monica, CA: HFES.
32. Sauro, J., Lewis, J.R. (2010). Average task times in usability tests: What to report? In: Proceedings of CHI 2010 (pp. 2347–2350). Atlanta, GA: ACM.
33. Smithson, M. (2003). Confidence intervals. Sage University Papers Series on Quantitative Applications in the Social Science, 07-140. Thousand Oaks, CA: Sage.
34. E.B. Wilson (1927) Probable inference, the law of succession, and statistical inference. J. Am. Stat. Assoc. 22 209-212.
35. A. Agresti and C.A. Franklin (2007) Statistics: The Art and Science of Learning from Data. New York, NY: Pearson
36. P. Armitage, G. Berry and J.N.S. Matthews (2002) Statistical Methods in Medical Research. fourth ed. Oxford, UK: Blackwell Science
37. A. Bangor, P.T. Kortum and J.T. Miller (2008) An empirical evaluation of the System Usability Scale. Int. J. Hum.-Comput. Interact. 6 574-594.
38. A. Chapanis (1988) Some generalizations about generalization. Hum. Factors 30 253-267.
39. L. Gonick and W. Smith (1993) Cartoon Guide to Statistics. New York, NY: Collins Reference
40. Lah, U., & Lewis, J.R., 2016. How expertise affects a digital-rights-management-sharing application’s usability. IEEE Software 33(3), 76–82.
41. Lewis, J.R., 1982. Testing small system customer set-up. Proceedings of the Human Factors Society 26th Annual Meeting. Human Factors Society, Santa Monica, CA, pp. 718–720.
42. J.R. Lewis and J. Sauro (2009) The factor structure of the System Usability Scale. M. Kurosu (Eds) Human Centered Design, HCII 2009 94-103 Berlin, Germany: Springer-Verlag
43. Molich, R., Kirakowski, J., Sauro, J., Tullis, T., 2009. Comparative usability task measurement workshop (CUE-8). At the UPA 2009 Conference. UPA, Portland, OR.
44. S. Rosenbaum (1989) Usability evaluations versus usability testing: When and why? IEEE Trans. Prof. Commun. 32 210-216.
45. J. Sauro (2009) Estimating productivity: composite operators for keystroke level modeling. J.A. Jacko (Eds) Proceedings of the 13th International Conference on Human-Computer Interaction, HCII 2009 Springer-Verlag, Berlin, Germany 352-361.
46. Sauro, J., 2010. Does better usability increase customer loyalty? From Measuring U. Available from: http://www.measuringu.com/usability-loyalty.php
47. Sauro, J. (2011). What is a good task-completion rate? From Measuring U. Available from: http://www.measuringu.com/blog/task-completion.php
48. Sauro, J., Kindlund E., 2005. How long should a task take? Identifying specification limits for task times in usability tests. Proceedings of the Human Computer Interaction International Conference (HCII 2005), Las Vegas, USA.
49. Sauro, J., Lewis, J.R., 2010. Average task times in usability tests: What to report? Proceedings of CHI 2010. ACM, Atlanta, GA, 2347–2350.
50. A. Agresti and B. Caffo (2000) Simple and effective confidence intervals for proportions and differences of proportions result from adding two successes and two failures. Am. Stat. 54(4), 280-288.
51. A. Agresti and B. Coull (1998) Approximate is better than ‘exact’ for interval estimation of binomial proportions. Am. Stat. 52 119-126.
52. A. Agresti and C.A. Franklin (2007) Statistics: The Art and Science of Learning from Data. Upper Saddle River, NJ: Prentice Hall
53. A. Agresti and Y. Min (2005) Simple improved confidence intervals for comparing matched proportions. Stat. Med. 24 729-740.
54. M. Bland (2000) An Introduction to Medical Statistics. third ed. Oxford, UK: Oxford University Press
55. G.E.P. Box (1953) Non-normality and test on variance. Biometrika 40 318-355.
56. I. Campbell (2007) Chi-squared and Fisher–Irwin tests of two-by-two tables with small sample recommendations. Stat. Med. 26 3661-3675.
57. W.G. Cochran (1952) The χ 2 test of goodness of fit. Ann. Math. Stat. 23 315-345.
58. W.G. Cochran (1954) Some methods for strengthening the common χ 2 tests. Biometrics 10 417-451.
59. D. Howell (2002) Statistical Methods for Psychology. fifth ed. Andover, UK: Thomson Learning
60. Q. McNemar (1969) Psychological Statistics. fourth ed. New York: Wiley
61. K. Pearson (1900) On the criterion that a given system of deviations from the probable in the case of a correlated system of variables is such that it can be reasonably supposed to have arisen from random sampling. Phil. Mag. Ser. 5(50), 157-175.
62. F.E. Satterthwaite (1946) An approximate distribution of estimates of variances components. Biometrics Bull. 2 110-114.
63. J. Sauro (2011) A Practical Guide to the System Usability Scale. Denver, CO: CreateSpace
64. Sauro, J., 2011b. If you could only ask one question, use this one. Available from: http://www.measuringu.com/blog/single-question.php
65. B.L. Welch (1938) The significance of the difference between two means when the population variances are unequal. Biometrika 29 350-362.
66. A. Agresti and B. Coull (1998) Approximate is better than ‘exact’ for interval estimation of binomial proportions. Am. Stat. 52 119-126.
67. A. Agresti and Y. Min (2005) Simple improved confidence intervals for comparing matched proportions. Stat. Med. 24 729-740.
68. P.L. Alreck and R.B. Settle (1985) The Survey Research Handbook. Homewood, IL: Richard D. Irwin
69. J.V. Bradley (1976) Probability; Decision; Statistics. Englewood Cliffs, NJ: Prentice-Hall
70. F.E. Brown (1980) Marketing Research: A Structure for Decision Making. Reading, MA: Addison-Wesley
71. I. Campbell (2007) Chi-squared and Fisher–Irwin tests of two-by-two tables with small sample recommendations. Stat. Med. 26 3661-3675.
72. D.T. Campbell and J.C. Stanley (1963) Experimental and Quasi-Experimental Designs for Research. Chicago, IL: Rand McNally
73. G.A. Churchill Jr (1991) Marketing Research: Methodological Foundations. Fort Worth, TX: Dryden Press
74. J. Cohen (1988) Statistical Power Analysis for the Behavioral Sciences. second ed. Hillsdale, NJ: Lawrence Erlbaum
75. M. Cowles (1989) Statistics in Psychology: An Historical Perspective. Hillsdale, NJ: Lawrence Erlbaum
76. W.J. Diamond (1981) Practical Experiment Designs for Engineers and Scientists. Belmont, CA: Lifetime Learning Publications
77. J.S. Dumas and M.C. Salzman (2006) Usability assessment methods. R.C. Williges (Eds) Reviews of Human Factors and Ergonomics vol. 2 Santa Monica, CA: HFES 109-140.
78. J. Kirakowski (2005) Summative usability testing: measurement and sample size. R.G. Bias, D.J. Mayhew (Eds) Cost-Justifying Usability: An Update for the Internet Age Amsterdam, The Netherlands: Elsevier 519-554.
79. H.C. Kraemer and S. Thiemann (1987) How Many Subjects? Statistical Power Analysis in Research. Newbury Park, CA: Sage Publications
80. T.K. Landauer (1997) Behavioral research methods in human–computer interaction. M. Helander, T.K. Landauer, P. Prabhu (Eds) Handbook of Human–Computer Interaction second ed. Amsterdam, The Netherlands: Elsevier 203-227.
81. J.R. Lewis (2012) Usability testing. G. Salvendy (Eds) Handbook of Human Factors and Ergonomics fourth ed. New York, NY: John Wiley 1267-1312.
82. R.E. Mayer (1997) From novice to expert. M.G. Helander, T.K. Landauer, P.V. Prabhu (Eds) Handbook of Human–Computer Interaction second ed. Amsterdam, The Netherlands: Elsevier 781-795.
83. E.W. Minium, B.M. King and G. Bear (1993) Statistical Reasoning in Psychology and Education. third ed. New York, NY: John Wiley
84. J.L. Myers (1979) Fundamentals of Experimental Design. Boston, MA: Allyn & Bacon
85. J. Nielsen (1997) Usability testing. G. Salvendy (Eds) Handbook of Human Factors and Ergonomics second ed. New York, NY: John Wiley 1543-1568.
86. A. Parasuraman (1986) Nonprobability sampling methods. Reading, MA: Addison-Wesley pp. 498-516.
87. D. Salsburg (2001) The Lady Tasting Tea: How Statistics Revolutionized Science in the Twentieth Century. New York, NY: Henry Holt
88. Sauro, J., Lewis, J.R., 2010. Average task times in usability tests: what to report? In: Proceedings of CHI. ACM, Atlanta, GA, pp. 2347–2350.
89. M. Scriven (1967) The methodology of evaluation. R.W. Tyler, R.M. Gagne, M. Scriven (Eds) Perspectives of Curriculum Evaluation Chicago, IL: Rand McNally 39-83.
90. S.M. Stigler (1986) The History of Statistics: The Measurement of Uncertainty Before 1900. Cambridge, MA: The Belknap Press of Harvard University Press
91. R.E. Walpole (1976) Elementary Statistical Concepts. New York, NY: Macmillan
92. C.D. Wickens (1998) Commonsense statistics. Ergon. Des. 6(4), 18-22.
93. B.J. Winer, D.R. Brown and K.M. Michels (1991) Statistical Principles in Experimental Design. third ed. New York, NY: McGraw-Hill
94. A. Agresti (1994) Simple capture-recapture models permitting unequal catchability and variable sampling effort. Biometrics 50 494-500.
95. J. Al-Awar, A. Chapanis and R. Ford (1981) Tutorials for the first-time computer user. IEEE Trans. Prof. Commun. 24 30-37.
96. R.M. Baecker (2008) Themes in the early history of HCI—some unanswered questions. Interactions 15(2), 22-27.
97. Barnum, C., Bevan, N., Cockton, G., Nielsen, J., Spool, J., Wixon, D., 2003. The “Magic Number 5”: Is it enough for web testing? In: Proceedings of CHI 2003. Ft. Lauderdale, FL: ACM, pp. 698–699.
98. B.W. Boehm (1981) Software Engineering Economics. Englewood Cliffs, NJ: Prentice-Hall
99. S. Borsci, A. Londei and S. Federici (2011) The bootstrap discovery behaviour (BDB): a new outlook on usability evaluation. Cogn. Process. 12 23-31.
100. S. Borsci, R.D. MacRedie, J. Barnett, J. Martin, J. Kuljis and T. Young (2013) Reviewing and extending the five-user assumption: A grounded procedure for interaction evaluation. ACM Trans. Comput. Hum. Interact. 20(5), 1-23. Article 29
101. J.V. Bradley (1976) Probability; Decision; Statistics. Englewood Cliffs, NJ: Prentice-Hall
102. J.V. Bradley (1978) Robustness? Br. J. Math. Stat. Psychol. 31 144-152.
103. L.C. Briand, K. El Emam, B.G. Freimut and O. Laitenberger (2000) A comprehensive evaluation of capture-recapture models for estimating software defect content. IEEE Trans. Softw. Eng. 26(6), 518-540.
104. K.P. Burnham and W.S. Overton (1979) Estimation of the size of a closed population when capture probabilities vary among animals. Biometrika 65 625-633.
105. D.A. Caulton (2001) Relaxing the homogeneity assumption in usability testing. Behav. Inf. Technol. 20 1-7.
106. A. Chapanis (1988) Some generalizations about generalization. Hum. Factors 30 253-267.
107. B.A. Coull and A. Agresti (1999) The use of mixed logit models to reflect heterogeneity in capture-recapture studies. Biometrics 55 294-301.
108. M. Cowles (1989) Statistics in Psychology: An Historical Perspective. Hillsdale, NJ: Lawrence Erlbaum
109. S.R. Dalal and C.L. Mallows (1990) Some graphical aids for deciding when to stop testing software. IEEE J. Sel. Area. Comm. 8(2), 169-175.
110. R.M. Dorazio (2009) On selecting a prior for the precision parameter of Dirichlet process mixture models. J. Stat. Plan. Inference. 139 3384-3390.
111. R.M. Dorazio and J.A. Royle (2003) Mixture models for estimating the size of a closed population when capture rates vary among individuals. Biometrics 59 351-364.
112. J. Dumas (2007) The great leap forward: The birth of the usability profession (1988-1993). J. Usability Stud. 2(2), 54-60.
113. Dumas, J., Sorce, J., Virzi, R., 1995. Expert reviews: How many experts is enough? In: Proceedings of the Human Factors and Ergonomics Society Thirty-Ninth Annual Meeting. Santa Monica, CA: Human Factors and Ergonomics Society, pp. 228–232.
114. Eick, S.G., Loader, C.R., Vander Wiel, S.A., Votta, L.G., 1993. How many errors remain in a software design document after inspection? In: Proceedings of the Twenty-Fifth Symposium on the Interface. Fairfax Station, V.A.: Interface Foundation of North America, pp. 195–202.
115. D.M. Ennis and J. Bi (1998) The beta-binomial model: accounting for inter-trial variation in replicated difference and preference tests. J. Sens. Stud. 13 389-412.
116. L. Faulkner (2003) Beyond the five-user assumption: benefits of increased sample sizes in usability testing. Behav. Res. Methods Instrum. Comput. 35 379-383.
117. J.D. Gould (1988) How to design usable systems. M. Helander (Eds) Handbook of Human–Computer Interaction Amsterdam, Netherlands: North-Holland 757-789.
118. J.D. Gould and S.J. Boies (1983) Human factors challenges in creating a principal support office system: The Speech Filing System approach. ACM Trans. Inf. Syst. 1 273-298.
119. J.D. Gould and C. Lewis (1984) Designing for Usability: Key Principles and What Designers Think. Yorktown Heights, NY: IBM Corporation (Tech. Report RC-10317)
120. J.D. Gould, S.J. Boies, S. Levy, J.T. Richards and J. Schoonard (1987) The 1984 Olympic message system: a test of behavioral principles of system design. Commun. ACM 30 758-769.
121. G. Guest, A. Bunce and L. Johnson (2006) How many interviews are enough? An experiment with data saturation and variability. Field Methods 18(1.), 59-82.
122. M. Hertzum and N.J. Jacobsen (2001) The evaluator effect: A chilling fact about usability evaluation methods. Int. J. Hum. Comput. Interact. 13 421-443.
123. K. Hornbæk (2010) Dogmas in the assessment of usability evaluation methods. Behav. Inf. Technol. 29(1), 97-111.
124. W. Hwang and G. Salvendy (2007) What makes evaluators to find more usability problems?: A meta-analysis for individual detection rates. J. Jacko (Eds) Human-Computer Interaction, Part I, HCII 2007 Heidelberg, Germany: Springer-Verlag 499-507.
125. W. Hwang and G. Salvendy (2009) Integration of usability evaluation studies via a novel meta-analytic approach: What are significant attributes for effective evaluation. Int. J. Hum. Comput. Interact. 25(4), 282-306.
126. W. Hwang and G. Salvendy (2010) Number of people required for usability evaluation: The 10 ± 2 rule. Commun. ACM 53(5), 130-133.
127. F. Jelinek (1997) Statistical Methods for Speech Recognition. Cambridge, MA: MIT Press
128. H. Kanis (2011) Estimating the number of usability problems. Appl. Ergon. 42 337-347.
129. Kennedy, P. J., 1982. Development and testing of the operator training package for a small computer system. In: Proceedings of the Human Factors Society Twenty-Sixth Annual Meeting. Santa Monica, CA: Human Factors Society, pp. 715–717.
130. Law, E. L., Hvannberg, E. T., 2004. Analysis of combinatorial user effect in international usability tests. In: Proceedings of CHI 2004. Vienna, Austria: ACM, pp. 9–16.
131. Lewis, J. R., 1982. Testing small system customer set-up. In: Proceedings of the Human Factors Society Twenty-Sixth Annual Meeting. Santa Monica, CA: Human Factors Society.
132. J.R. Lewis (1994) Sample sizes for usability studies: additional considerations. Hum. Factors 36 368-378.
133. Lewis, J. R., 2000. Evaluation of problem discovery rate adjustment procedures for sample sizes from two to ten (Tech. Report 29.3362). Raleigh, NC: IBM Corp. Available from: http://drjim.0catch.com/pcarlo5-ral.pdf.
134. J.R. Lewis (2001) Evaluation of procedures for adjusting problem-discovery rates estimated from small samples. Int. J. Hum. Comput. Interact. 13 445-479.
135. Lewis, J. R., 2006a. Effect of level of problem description on problem discovery rates: Two case studies. In: Proceedings of the Human Factors, Ergonomics Fiftieth Annual Meeting. Santa Monica, C.A.: HFES, pp. 2567–2571.
136. J.R. Lewis (2006) Sample sizes for usability tests: mostly math, not magic. Interactions 13(6), 29-33.
137. J.R. Lewis (2008) Usability evaluation of a speech recognition IVR. T. Tullis, B. Albert (Eds) Measuring the User Experience, Chapter 10: Case Studies Amsterdam, Netherlands: Morgan-Kaufman 244-252.
138. J.R. Lewis (2012) Usability testing. G. Salvendy (Eds) Handbook of Human Factors and Ergonomics 4th ed. New York, NY: John Wiley 1267-1312.
139. J.R. Lewis and J. Sauro (2006) When 100% really isn’t 100%: Improving the accuracy of small-sample estimates of completion rates. J. Usability Test. 3(1), 136-150.
140. Lewis, J. R., Henry, S. C., Mack, R. L., 1990. Integrated office software benchmarks: a case study. In: Proceedings of the Third IFIP Conference on Human-Computer Interaction–INTERACT ’90. Cambridge, UK: Elsevier Science Publishers, pp. 337–343.
141. Lindgaard, G., Chattratichart, J., 2007. Usability testing: What have we overlooked?” In: Proceedings of CHI 2007. San Jose, CA: ACM, pp. 1415–1424.
142. C.D. Manning and H. Schütze (1999) Foundations of Statistical Natural Language Processing. Cambridge, MA: MIT Press
143. M.C. Medlock, D. Wixon, M. McGee and D. Welsh (2005) The rapid iterative test and evaluation method: Better products in less time. R.G. Bias, D.J. Mayhew (Eds) Cost-Justifying Usability: An Update for the Internet Age Amsterdam, Netherlands: Elsevier 489-517.
144. Nielsen, J., 1992. Finding usability problems through heuristic evaluation. In: Proceedings of CHI ‘92. Monterey, CA: ACM, pp. 373–380.
145. Nielsen, J., 2000. Why you only need to test with 5 users. Alertbox, www.useit.com/alertbox/20000319.html. (Downloaded January 26, 2011.).
146. Nielsen, J., Landauer, T. K., 1993. A mathematical model of the finding of usability problems. In: Proceedings of INTERCHI’93. Amsterdam, Netherlands: ACM, pp. 206–213.
147. Nielsen, J., Molich, R., 1990. Heuristic evaluation of user interfaces. In: Proceedings of CHI ’90. New York, NY: ACM, pp. 249–256.
148. Perfetti, C., Landesman, L., 2001. Eight is not enough. Available from: http://www.uie.com/articles/eight_is_not_enough/.
149. J. Sauro (2014) The relationship between problem frequency and problem severity in usability evaluations. J. Usability Stud. 10(1), 17-25.
150. Schmettow, M., 2008. Heterogeneity in the usability evaluation process. In: Proceedings of the Twenty-Second British HCI Group Annual Conference on HCI 2008: People and Computers XXII: Culture, Creativity, Interaction—Volume 1. Liverpool, UK: ACM, pp. 89–98.
151. Schmettow, M., 2009. Controlling the usability evaluation process under varying defect visibility. In: Proceedings of the 2009 British Computer Society Conference on Human-Computer Interaction. Cambridge, UK: ACM, pp. 188–197.
152. M. Schmettow (2012) Sample size in usability tests. Commun. ACM 55(4), 64-70.
153. Z.E. Schnabel (1938) The estimation of the total fish population of a lake. Amer. Math. Monthly 45 348-352.
154. D.C. Smith, C. Irby, R. Kimball, B. Verplank and E. Harlem (1982) Designing the star user interface. Byte 7(4), 242-282.
155. Spool, J., Schroeder, W., 2001. Testing websites: five users is nowhere near enough. In: CHI 2001 Extended Abstracts. New York, N.Y., AC.M., pp. 285–286.
156. C.W. Turner, J.R. Lewis and J. Nielsen (2006) Determining usability test sample size. W. Karwowski (Eds) The International Encyclopedia of Ergonomics and Human Factors Boca Raton, FL: CRC Press 3084-3088.
157. Virzi, R. A., 1990. Streamlining the design process: Running fewer subjects. In: Proceedings of the Human Factors Society Thirty-Fourth Annual Meeting. Santa Monica, CA: Human Factors Society, pp. 291–294.
158. R.A. Virzi (1992) Refining the test phase of usability evaluation: how many subjects is enough? Hum. Factors 34 457-468.
159. R.A. Virzi (1997) Usability inspection methods. M.G. Helander, T.K. Landauer, P.V. Prabhu (Eds) Handbook of Human–Computer Interaction second ed. Amsterdam, Netherlands: Elsevier 705-715.
160. Walia, G. S., Carver, J. C., 2008. Evaluation of capture-recapture models for estimating the abundance of naturally-occurring defects. In: Proceedings of ESEM ’08. Kaiserslautern, Germany: ACM, pp. 158–167.
161. Walia, G. S., Carver, J. C., Nagappan, N., 2008. The effect of the number of inspectors on the defect estimates produced by capture-recapture models. In: Proceedings of ICSE ’08. Leipzig, Germany: ACM, pp. 331–340.
162. J. Whiteside, J. Bennett and K. Holtzblatt (1988) Usability engineering: Our experience and evolution. M. Helander (Eds) Handbook of Human–Computer Interaction Amsterdam, Netherlands: North-Holland 791-817.
163. G. Williams (1983) The Lisa computer system. Byte 8(2), 33-50.
164. E.B. Wilson (1927) Probable inference, the law of succession, and statistical inference. J. Am. Stat. Assoc. 22 209-212.
165. Woolrych, A., Cockton, G., 2001. Why and when five test users aren’t enough. In: Vanderdonckt, J., Blandford, A., Derycke, A. (Eds.), Proceedings of IHM–HCI 2001 Conference, vol. 2. Toulouse, France: Cépadèus Éditions, pp. 105–108.
166. P.C. Wright and A.F. Monk (1991) A cost-effective evaluation method for use by designers. Int. J. Man Mach. Stud. 35 891-912.
167. A.M. Aladwani and P.C. Palvia (2002) Developing and validating an instrument for measuring user perceived Web quality. Inform. Manag. 39 467-476.
168. Albert, W., Dixon, E., 2003. Is this what you expected? The use of expectation measures in usability testing. Paper presented at the Usability Professionals Association Annual Conference, UPA, Scottsdale, AZ.
169. A. Anastasi (1976) Psychological Testing. New York, NY: Macmillan
170. D. Andrich (1978) Application of a psychometric rating model to ordered categories which are scored with successive integers. Appl. Psychol. Meas. 2(4), 581-594.
171. ANSI, 2001. Common Industry Format for Usability Test Reports (ANSI-NCITS 354-2001). Washington, DC: Author.
172. C.V. Azzara (2010) Questionnaire Design for Business Research. Mustang, OK: Tate Publishing
173. A. Bangor, P.T. Kortum and J.T. Miller (2008) An empirical evaluation of the System Usability Scale. Int. J. Hum.-Comput. Interact. 24 574-594.
174. A. Bangor, P.T. Kortum and J.T. Miller (2009) Determining what individual SUS scores mean: adding an adjective rating scale. J. Usability Stud. 4(3), 114-123.
175. Bangor, A., Joseph, K., Sweeney-Dillon, M., Stettler, G., Pratt, J., 2013. Using the SUS to help demonstrate usability’s value to business goals. In: Proceedings of the Human Factors Society and Ergonomics Society Annual Meeting, HFES, Santa Monica, CA, pp. 202–205.
176. J.A. Bargas-Avila, J. Lötscher, S. Orsini and K. Opwis (2009) Intranet Satisfaction Questionnaire: development and validation of a questionnaire to measure user satisfaction with the Intranet. Comput. Hum. Behav. 25 1241-1250.
177. J.J. Barnette (2000) Effects of stem and Likert response option reversals on survey internal consistency: if you feel the need, there is a better alternative to using those negatively worded stems. Educ. Psychol. Meas. 60 361-370.
178. Benedek, J., Miner, T., 2002. Measuring desirability: new methods for evaluating desirability in a usability lab setting. Paper presented at the Usability Professionals Association Annual Conference, UPA, Orlando, FL.
179. B. Blažica and J.R. Lewis (2015) A Slovene translation of the System Usability Scale (SUS). Int. J. Hum.-Comput. Interact. 31 112-117.
180. T.G. Bond and C.M. Fox (2001) Applying the Rasch Model: Fundamental Measurement in the Human Sciences. Mahwah, NJ: Lawrence Erlbaum
181. S. Borsci, S. Federici and M. Lauriola (2009) On the dimensionality of the System Usability Scale: a test of alternative measurement models. Cogn. Process. 10 193-197.
182. S. Borsci, S. Federici, S. Bacci, M. Gnaldi and F. Bartolucci (2015) Assessing user satisfaction in the era of user experience: comparison of the SUS, UMUX and UMUX-LITE as a function of product experience. Int. J. Hum.-Comput. Interact. 31 484-495.
183. I. Brace (2008) Questionnaire Design: How to Plan, Structure, and Write Survey Material for Effective Market Research. second ed. London, UK: Kogan Page Limited
184. J. Brooke (1996) SUS: A ‘quick and dirty’ usability scale. P. Jordan, B. Thomas, B. Weerdmeester (Eds) Usability Evaluation in Industry London, UK: Taylor & Francis 189-194.
185. J. Brooke (2013) SUS: a retrospective. J. Usability Stud. 8(2), 29-40.
186. H. Cavallin, W.M. Martin and A. Heylighen (2007) How relative absolute can be: SUMI and the impact of the nature of the task in measuring perceived software usability. Artif. Intell. Soc. 22 227-235.
187. G.W. Cheung and R.B. Rensvold (2000) Assessing extreme and acquiescence response sets in cross-cultural research using structural equations modeling. J. Cross-Cult. Psychol. 31 187-212.
188. Chin, J.P., Diehl, V.A., Norman, K.L., 1988. Development of an instrument measuring user satisfaction of the human–computer interface. In: Proceedings of CHI 1988, ACM, Washington, DC, pp. 213–218.
189. Cordes, R.E., 1984a. Software ease of use evaluation using magnitude estimation. In: Proceedings of the Human Factors Society, HFS, Santa Monica, CA, pp. 157–160.
190. Cordes, R.E., 1984b. Use of magnitude estimation for evaluating product ease of use (Tech. Report 82.0135), IBM, Tucson, AZ.
191. J.M. Cortina (1993) What is coefficient alpha? An examination of theory and applications. J. Exp. Psychol. 78(1), 98-104.
192. C. Courage and K. Baxter (2005) Understanding Your Users: A Practical Guide to User Requirements. San Francisco, CA: Morgan Kaufmann
193. L.J. Cronbach (1946) Response sets and test validity. Educ. Psychol. Meas. 6 475-494.
194. D. Davis (1989) Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quart. 13(3), 319-339.
195. S.E. Embretson and S.P. Reise (2000) Item Response Theory for Psychologists. Mahwah, NJ: Lawrence Erlbaum Associates, Inc
196. O. Erdinç and J.R. Lewis (2013) Psychometric evaluation of the T-CSUQ: the Turkish version of the Computer System Usability Questionnaire. Int. J. Hum.-Comput. Interact. 29 319-323.
197. K. Finstad (2006) The System Usability Scale and non-native English speakers. J. Usability Stud. 1(4), 185-188.
198. K. Finstad (2010) Response interpolation and scale sensitivity: evidence against 5-point scales. J. Usability Stud. 5(3), 104-110.
199. K. Finstad (2010) The usability metric for user experience. Interact. Comput. 22 323-327.
200. K. Finstad (2013) Response to commentaries on “The Usability Metric for User Experience”. Interact. Comput. 25 327-330.
201. Grier, R.A., Bangor, A., Kortum, P., Peres, S.C., 2013. The System Usability Scale: beyond standard usability testing. In: Proceedings of the Human Factors Society and Ergonomics Society Annual Meeting, HFES, Santa Monica, CA, pp. 187–191.
202. S.D. Grimm and A.T. Church (1999) A cross-cultural study of response biases in personality measures. J. Res. Pers. 33 415-441.
203. M. Hassenzahl (2001) The effect of perceived hedonic quality on product appealingness. Int. J. Hum.-Comput. Interact. 13(4), 481-499.
204. M. Hassenzahl (2004) The interplay of beauty, goodness, and usability in interactive products. Hum.-Comput. Interact. 19 319-349.
205. Hassenzahl, M., Platz, A., Burmester, M., Lehner, K., 2000. Hedonic and ergonomic quality aspects determine a software’s appeal. In: Proceedings of CHI 2000, ACM, The Hague, Amsterdam, pp. 201–208.
206. M. Hassenzahl, A. Wiklund-Engblom, A. Bengs, S. Hägglund and S. Diefenbach (2015) Experience-oriented and product-oriented evaluation: psychological need fulfillment, positive affect, and product perception. Int. J. Hum.-Comput. Interact. 31 530-544.
207. Hollemans, G., 1999. User satisfaction measurement methodologies: extending the user satisfaction questionnaire. In: Proceedings of HCI International 1999, Lawrence Erlbaum, Mahwah, NJ, pp. 1008–1012.
208. K. Hornbæk (2006) Current practice in measuring usability: challenges to usability studies and research. Int. J. Hum.-Comput. Stud. 64(2), 79-102.
209. Hornbæk, K., Law, E.L., 2007. Meta-analysis of correlations among usability measures. In: Proceedings of CHI 2007, ACM, San Jose, CA, pp. 617–626.
210. A.M. Ibrahim (2001) Differential responding to positive and negative items: the case of a negative item in a questionnaire for course and faculty evaluation. Psychol. Rep. 88 497-500.
211. ISO. (1998). Ergonomic requirements for office work with visual display terminals (VDTs), Part 11, Guidance on usability (ISO 9241-11:1998E). Geneva, Switzerland: Author.
212. M. Joyce and J. Kirakowski (2015) Measuring attitudes towards the Internet: the General Internet Attitude Scale. Int. J. Hum.-Comput. Interact. 31 506-517.
213. Karn, K., Little, A., Nelson, G., Sauro, J., Kirakowski, J., Albert, W., Norman, K., 2008. Subjective ratings of usability: reliable or ridiculous? Panel Presentation at the Usability Professionals Association Annual Conference, UPA, Baltimore, MD.
214. T.L. Keiningham, B. Cooil, T.W. Andreassen and L. Aksoy (2007) A longitudinal examination of Net Promoter and firm revenue growth. J. Market. 71 39-51.
215. J. Kirakowski (1996) The software usability measurement inventory: background and usage. P. Jordan, B. Thomas, B. Weerdmeester (Eds) Usability Evaluation in Industry London, UK: Taylor & Francis 169-178. Available from: www.ucc.ie/hfrg/questionnaires/sumi/index.html
216. Kirakowski, J., Cierlik, B., 1998. Measuring the usability of websites. In: Proceedings of the Human Factors and Ergonomics Society 42nd Annual Meeting, HFES, Santa Monica, CA, pp. 424–428. Available from: www.wammi.com
217. J. Kirakowski and M. Corbett (1993) SUMI: the Software Usability Measurement Inventory. Br. J. Educ. Technol. 24 210-212.
218. J. Kirakowski and A. Dillon (1988) The Computer User Satisfaction Inventory (CUSI): Manual and Scoring Key. Cork, Ireland: Human Factors Research Group, University College of Cork
219. P. Kortum and A. Bangor (2013) Usability ratings for everyday products measured with the System Usability Scale. Int. J. Hum.-Comput. Interact. 29 67-76.
220. Kortum, P., Johnson, M., 2013. The relationship between levels of user experience with a product and perceived system usability. In: Proceedings of the Human Factors Society and Ergonomics Society Annual Meeting, HFES, Santa Monica, CA, pp. 197–201.
221. P. Kortum and S.C. Peres (2014) The relationship between system effectiveness and subjective usability scores using the System Usability Scale. Int. J. Hum.-Comput. Interact. 30 575-584.
222. P. Kortum and M. Sorber (2015) Measuring the usability of mobile applications for phones and tablets. Int. J. Hum.-Comput. Interact. 31 518-529.
223. M. Kuniavsky (2003) Observing the User Experience: A Practitioner’s Guide to User Research. San Francisco, CA: Morgan Kaufmann
224. M.J. LaLomia and J.B. Sidowski (1990) Measurements of computer satisfaction, literacy, and aptitudes: a review. Int. J. Hum.–Comput. Interact. 2 231-253.
225. T.K. Landauer (1997) Behavioral research methods in human–computer interaction. M. Helander, T.K. Landauer, P. Prabhu (Eds) Handbook of Human–Computer Interaction second ed. Amsterdam, Netherlands: Elsevier 203-227.
226. D. Lascu and K.E. Clow (2008) Web site interaction satisfaction: scale development consideration. J. Internet Commer. 7(3), 359-378.
227. D. Lascu and K.E. Clow (2013) Website interaction satisfaction: a reassessment. Interact. Comput. 25 307-311.
228. Lewis, J.R., 1990a. Psychometric evaluation of a post-study system usability questionnaire: The PSSUQ (Tech. Report 54.535), International Business Machines Corp, Boca Raton, FL.
229. Lewis, J. R., 1990b. Psychometric evaluation of an after-scenario questionnaire for computer usability studies: The ASQ (Tech. Report 54.541), International Business Machines Corp., Boca Raton, FL.
230. J.R. Lewis (1991) Psychometric evaluation of an after-scenario questionnaire for computer usability studies: the ASQ. SIGCHI Bull. 23 78-81.
231. Lewis, J.R., 1992. Psychometric evaluation of the Post-Study System Usability Questionnaire: the PSSUQ. In: Proceedings of the Human Factors Society 36th Annual Meeting, Human Factors Society, Santa Monica, CA, pp. 1259–1263.
232. J.R. Lewis (1993) Multipoint scales: mean and median differences and observed significance levels. Int. J. Hum.-Comput. Interact. 5 382-392.
233. J.R. Lewis (1995) IBM computer usability satisfaction questionnaires: psychometric evaluation and instructions for use. Int. J. Hum.-Comput. Interact. 7 57-78.
234. Lewis, J.R., 1999. Tradeoffs in the design of the IBM computer usability satisfaction questionnaires. In: Proceedings of HCI International 1999, Lawrence Erlbaum, Mahwah, NJ, pp. 1023–1027.
235. J.R. Lewis (2002) Psychometric evaluation of the PSSUQ using data from five years of usability studies. Int. J. Hum.–Comput. Interact. 14 463-488.
236. J.R. Lewis (2011) Practical Speech User Interface Design. Boca Raton, FL: Taylor & Francis
237. Lewis, J.R., 2012a. Predicting net promoter scores from system usability scale scores. Available from: www.measuringu.com/blog/nps-sus.php.
238. J.R. Lewis (2012) Usability testing. G. Salvendy (Eds) Handbook of Human Factors and Ergonomics fourth ed. New York, NY: John Wiley 1267-1312.
239. J.R. Lewis (2013) Critical review of “Intranet Satisfaction Questionnaire: development and validation of a questionnaire to measure user satisfaction with the intranet”. Interact. Comput. 25 299-301.
240. J.R. Lewis (2013) Critical review of “The Usability Metric for User Experience”. Interact. Comput. 25 320-324.
241. J.R. Lewis and D.K. Mayes (2014) Development and psychometric evaluation of the Emotional Metric Outcomes (EMO) questionnaire. Int. J. Hum.-Comput. Interact. 30 685-702.
242. J.R. Lewis and J. Sauro (2009) The factor structure of the System Usability Scale. M. Kurosu (Eds) Human Centered Design, HCII 2009 Heidelberg, Germany: Springer-Verlag 94-103.
243. J.R. Lewis, S.C. Henry, R.L. Mack (1990) Integrated office software benchmarks: a case study. D. Diaper, et al., (Eds) Proceedings of the 3rd IFIP Conference on Human–Computer Interaction, INTERACT ’90 Cambridge, UK: Elsevier Science 337-343.
244. Lewis, J.R., Utesch, B.S., Maher, D.E., 2013. UMUX-LITE—when there’s no time for the SUS. In: Proceedings of CHI 2013, Association for Computing Machinery, Paris, France, pp. 2099–2102.
245. J.R. Lewis, J. Brown and D.K. Mayes (2015) Psychometric evaluation of the EMO and the SUS in the context of a large-sample unmoderated usability study. Int. J. Hum.-Comput. Interact. 31 545-553.
246. J.R. Lewis, B.S. Utesch and D.E. Maher (2015) Measuring perceived usability: the SUS, UMUX-LITE, and AltUsability. Int. J. Hum.-Comput. Int. 31 496-505.
247. E.T. Loiacono, R.T. Watson and D.L. Goodhue (2002) WEBQUAL: a measure of website quality. Market. Theory Appl. 13(3), 432-438.
248. Lucey, N.M., 1991. More than meets the I: User-satisfaction of computer systems (Unpublished thesis for Diploma in Applied Psychology). University College Cork, Cork, Ireland.
249. Lund, A., 1998. USE Questionnaire Resource Page, Available from: http://usesurvey.com.
250. A. Lund (2001) Measuring usability with the USE questionnaire. Usability User Exp. Newslett. STC Usability SIG 8(2), 1-4. Available from: www.stcsig.org/usability/newsletter/0110_measuring_with_use.html
251. D.W. Massaro (1975) Experimental Psychology and Information Processing. Chicago, IL: Rand McNally College Publishing Company
252. McGee, M., 2003. Usability magnitude estimation. In: Proceedings of the Human Factors and Ergonomics Society 47th Annual Meeting, HFES, Santa Monica, CA, pp. 691–695.
253. McGee, M., 2004. Master usability scaling: magnitude estimation and master scaling applied to usability measurement. In: Proceedings of CHI 2004, ACM, Vienna, Austria, pp. 335–342.
254. S. McLellan, A. Muddimer and S.C. Peres (2012) The effect of experience on System Usability Scale ratings. J. Usability Stud. 7(2), 56-67.
255. McSweeney, R., 1992. SUMI: A psychometric approach to software evaluation (unpublished M.A. (Qual.) thesis in applied psychology). University College of Cork, Cork, Ireland. Available from: http://sumi.ucc.ie/sumipapp.html.
256. Molich, R., Kirakowski, J, Sauro, J., Tullis, T., 2009. Comparative usability task measurement workshop (CUE-8). Workshop conducted at the UPA 2009 Conference in Portland, OR.
257. P. Mussen, M.R. Rosenzweig, E. Aronson, D. Elkind, S. Feshbach, J. Geiwitz, S.E. Glickman, B.B. Murdock, M. Wertheimer and L.O. Harvey (1977) Psychology: An Introduction. second ed. Lexington, MA: D.C. Heath and Company
258. J.C. Nunnally (1978) Psychometric Theory. New York, NY: McGraw-Hill
259. S. Orsini, K. Opwis and J.A. Bargas-Avila (2013) Response to the reviews on Bargas-Avila et al (2009) ’Intranet Satisfaction Questionnaire: Development and validation of a questionnaire to measure user satisfaction with the intranet’. Interact. Comput. 25 304-306.
260. A. Parasuraman (1986) Marketing Research. Reading, MA: Addison-Wesley
261. Peres, S.C., Pham, T., Phillips, R., 2013. Validation of the System Usability Scale (SUS): SUS in the wild. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, HFES, Santa Monica, CA, pp. 192–196.
262. S. Presser and H. Schuman (1980) The measurement of a middle position in attitude surveys. Pub. Opin. Quart. 44(1), 70-85.
263. C.C. Preston and A.M. Colman (2000) Optimal number of response categories in rating scales: reliability, validity, discriminating power, and respondent preferences. Acta Psychol. 104 1-15.
264. L.C. Quilty, J.M. Oakman and E. Risko (2006) Correlates of the Rosenberg Self-Esteem Scale method effects. Struct. Eq. Model. 13(1), 99-117.
265. M. Rauschenberger, M. Schrepp, M.P. Cota, S. Olschner and J. Thomaschewski (2013) Efficient measurement of the user experience of interactive products: How to use the User Experience Questionnaire (UEQ). Int. J. Artif. Intell. Interact. Multimed. 2(1), 39-45.
266. F.F. Reichheld (2003) The one number you need to grow. Harvard Bus. Rev. 81 46-54.
267. F. Reichheld (2006) The Ultimate Question: Driving Good Profits and True Growth. Boston MA: Harvard Business School Press
268. S.P. Reise, A.T. Ainsworth and M.G. Haviland (2005) Item response theory: fundamentals, applications, and promise in psychological research. Curr. Dir. Psychol. Sci. 14(2), 95-101.
269. Safar, J.A., Turner, C.W.,2005. Validation of a two factor structure of system trust. In: Proceedings of the Human Factors and Ergonomics Society 49th Annual Meeting, HFES, Santa Monica, CA, pp. 497–501.
270. Sauro, J., 2010a. Does better usability increase customer loyalty? Available from: www.measuringu.com/usability-loyalty.php.
271. Sauro, J., 2010b. If you could only ask one question, use this one. Available from: www.measuringu.com/blog/single-question.php.
272. Sauro, J., 2010c. That’s the worst website ever! Effects of extreme survey items. Available from: www.measuringu.com/blog/extreme-items.php.
273. Sauro, J., 2010d. Top-box scoring of rating scale data. Available from: www.measuringu.com/blog/top-box.php.
274. J. Sauro (2011) A Practical Guide to the System Usability Scale (SUS): Background, Benchmarks & Best Practices. Denver, CO: Measuring Usability LLC
275. Sauro, J., 2011b. The Standardized User Experience Percentile Rank Questionnaire (SUPR-Q). Available from: www.suprq.com/
276. J. Sauro (2015) SUPR-Q: A comprehensive measure of the quality of the website user experience. J. Usability Stud. 10(2), 68-86.
277. Sauro, J., Dumas, J.S., 2009. Comparison of three one-question, post-task usability questionnaires. In: Proceedings of CHI 2009, ACM, Boston, MA, pp. 1599–1608.
278. Sauro, J., Lewis, J.R., 2009. Correlations among prototypical usability metrics: evidence for the construct of usability. In: Proceedings of CHI 2009, ACM, Boston, MA, pp. 1609–1618.
279. Sauro, J., Lewis, J.R., 2011. When designing usability questionnaires, does it hurt to be positive? In: Proceedings of CHI 2011, ACM, Vancouver, Canada, pp. 2215–2223.
280. Schmettow, M., Vietze, W., 2008. Introducing item response theory for measuring usability inspection processes. In: Proceedings of CHI 2008, ACM, Florence, Italy, pp. 893–902.
281. Slaughter, L., Harper, B., Norman, K., 1994. Assessing the equivalence of the paper and on-line formats of the QUIS 5.5. In: Proceedings of the 2nd Annual Mid-Atlantic Human Factors Conference, HFES, Washington, DC, pp. 87–91.
282. P. Spector, P. Van Katwyk, M. Brannick and P. Chen (1997) When two factors don’t reflect two constructs: how item characteristics can produce artifactual factors. J. Manag. 23(5), 659-677.
283. H. Sun and P. Zhang (2011) Causal relationships between perceived enjoyment and perceived ease of use: an alternative approach. J. Assoc. Inform. Syst. 7(9), 618-645.
284. Tedesco, D.P., Tullis, T.S., 2006. A comparison of methods for eliciting post-task subjective ratings in usability testing. Paper presented at the Usability Professionals Association Annual Conference, UPA, Broomfield, CO.
285. T. Teo and J. Noyes (2011) An assessment of the influence of perceived enjoyment and attitude on the intention to use technology among pre-service teachers: a structural equation modeling approach. Comput. Educ. 57(2), 1645-1653.
286. L.L. Thurstone (1928) Attitudes can be measured. Am. J. Sociol. 33 529-554.
287. Travis, D. (2008). Measuring satisfaction: beyond the usability questionnaire. Available from: www.userfocus.co.uk/articles/satisfaction.html
288. T. Tullis and B. Albert (2008) Measuring the User Experience: Collecting, Analyzing, and Presenting Usability Metrics. Burlington, MA: Morgan Kaufmann
289. Tullis, T.S., Stetson, J.N., 2004. A comparison of questionnaires for assessing website usability. Paper presented at the Usability Professionals Association Annual Conference, UPA, Minneapolis, MN(home.comcast.net/∼tomtullis/publications/UPA2004TullisStetson.pdf).
290. F.J.R. van de Vijver and K. Leung (2001) Personality in cultural context: methodological issues. J. Pers. 69 1007-1031.
291. J. Wang and S. Senecal (2007) Measuring perceived website usability. J. Internet Commer. 6(4), 97-112.
292. J. Whiteside, J. Bennett and K. Holtzblatt (1988) Usability engineering: Our experience and evolution. M. Helander (Eds) Handbook of Human–Computer Interaction Amsterdam, Netherlands: North-Holland, Amsterdam 791-817.
293. J. Wu, Y. Chen and L. Lin (2007) Empirical evaluation of the revised end user computing acceptance model. Comput. Hum. Behav. 23 162-174.
294. M.J. Zickar (1998) Modeling item-level data with item response theory. Curr. Dir. Psychol. Sci. 7 104-109.
295. Zijlstra, R., van Doorn, L., 1985. The construction of a scale to measure subjective effort (Tech. Rep.). Delft, Netherlands: Delft University of Technology, Department of Philosophy and Social Sciences.
296. M. Zviran, C. Glezer and I. Avni (2006) User satisfaction from commercial web sites: the effect of design and use. Inform. Manag. 43 157-178.
297. R.P. Abelson (1995) Statistics as Principled Argument. Hillsdale, NJ: Lawrence Erlbaum
298. T. Baguley (2004) Understanding statistical power in the context of applied research. Appl. Ergon. 35 73-80.
299. Y. Benjamini and Y. Hochberg (1995) Controlling the false discovery rate: a practical and powerful approach to multiple testing. J. R. Soc. Ser. B 57(1), 289-300.
300. J.F. Box (1978) Fisher, the Life of a Scientist. New York, NY: John Wiley
301. G.E.P. Box (1984) The importance of practice in the development of statistics. Technometrics 26(1), 1-8.
302. J.V. Bradley (1976) Probability; Decision; Statistics. Englewood Cliffs, NJ: Prentice-Hall
303. J.V. Bradley (1978) Robustness? Br. J. Math. Stat. Psychol. 31 144-152.
304. S.L. Braver (1975) On splitting the tails unequally: a new perspective on one- versus two-tailed tests. Educ. Psychol. Meas. 35 283-301.
305. N. Cliff (1987) Analyzing Multivariate Data. San Diego, CA: Harcourt, Brace, Jovanovich
306. J. Cohen (1990) Things I have learned (so far). Am. Psychol. 45(12), 1304-1312.
307. M. Cowles (1989) Statistics in Psychology: An Historical Perspective. Hillsdale, NJ: Lawrence Erlbaum
308. P. Diaconis (1985) Theories of data analysis: from magical thinking through classical statistics. D.C. Hoaglin, F. Mosteller, J.W. Tukey (Eds) Exploring Data Tables, Trends, and Shapes New York: John Wiley 1-36.
309. W.J. Diamond (1981) Practical Experiment Designs for Engineers and Scientists. Belmont, CA: Lifetime Learning Publications
310. R.A. Fisher (1929) The statistical method in psychical research. Proc. Soc. Psychical Res. 39 189-192.
311. J. Gaito (1980) Measurement scales and statistics: resurgence of an old misconception. Psychol. Bull. 87(3), 564-567.
312. J.W. Grove (1989) In Defence of Science: Science, Technology, and Politics in Modern Society. Toronto, Canada: University of Toronto Press
313. R.J. Harris (1985) A Primer of Multivariate Statistics. Orlando, FL: Academic Press
314. R.J. Harris (1997) Significance tests have their place. Psychol. Sci. 8(1), 8-11.
315. Hornbæk, K., Law, E., 2007. Meta-analysis of correlations among usability measures. In: Proceedings of CHI 2007, ACM, San Jose, CA pp. 617–626.
316. Lewis, J.R., 1991. A rank-based method for the usability comparison of competing products. In: Proceedings of the Human Factors Society 35th Annual Meeting, HFS, Santa Monica pp. 1312–1316.
317. J.R. Lewis (1993) Multipoint scales: mean and median differences and observed significance levels. Int. J. Hum.-Comput. Interact. 5(4), 383-392.
318. Lewis, J.R., Henry, S.C., Mack, R.L., 1990. Integrated office software benchmarks: a case study. In: Diaper, D., et al. (Eds.), Proceedings of the third IFIP Conference on Human–Computer Interaction, INTERACT’90, Elsevier Science, Cambridge, UK, pp. 337–343.
319. F.M. Lord (1953) On the statistical treatment of football numbers. Am. Psychol. 8 750-751.
320. F.M. Lord (1954) Further comment on “football numbers.”. Am. Psychol. 9( 6 ), 264-265.
321. J. Mitchell (1986) Measurement scales and statistics: a clash of paradigms. Psychol. Bull. 100 398-407.
322. J.L. Myers (1979) Fundamentals of Experimental Design. third ed. Boston, MA: Allyn and Bacon
323. J.C. Nunnally (1978) Psychometric Theory. New York, NY: McGraw-Hill
324. T.V. Perneger (1998) What’s wrong with Bonferroni adjustments? Br. Med. J. 316 1236-1238.
325. R.L. Rosnow and R. Rosenthal (1989) Statistical procedures and the justification of knowledge in psychological science. Am. Psychol. 44 1276-1284.
326. D. Salsburg (2001) The Lady Tasting Tea: How Statistics Revolutionized Science in the Twentieth Century. New York, NY: W. H. Freeman
327. Sauro, J., Kindlund, E., 2005. A method to standardize usability metrics into a single score. In: Proceedings of CHI 2005, ACM, Portland, OR, pp. 401–409.
328. Sauro, J., Lewis, J.R., 2009. Correlations among prototypical usability metrics: evidence for the construct of usability. In: Proceedings of CHI 2009, ACM, Boston, MA, pp. 1609–1618.
329. A.Z. Scholten and D. Borsboom (2009) A reanalysis of Lord’s statistical treatment of football numbers. J. Math. Psychol. 53(2), 69-75.
330. S.S. Stevens (1946) On the theory of scales of measurement. Science 103(2684), 677-680.
331. S.S. Stevens (1959) Measurement, psychophysics, and utility. C.W. Churchman, P. Ratoosh (Eds) Measurement: Definitions and Theories New York, NY: John Wiley 18-82.
332. S.M. Stigler (1986) The History of Statistics: The Measurement of Uncertainty Before 1900. Cambridge, MA: Harvard University Press
333. S.M. Stigler (1999) Statistics on the Table: The History of Statistical Concepts and Methods. Cambridge, MA: Harvard University Press
334. J.T. Townsend and F.G. Ashby (1984) Measurement scales and statistics: the misconception misconceived. Psychol. Bull. 96(2), 394-401.
335. P.F. Velleman and L. Wilkinson (1993) Nominal, ordinal, interval, and ratio typologies are misleading. Am. Stat. 47(1), 65-72.
336. C.D. Wickens (1998) Commonsense statistics. Ergon. Des. 6(4), 18-22.
337. B.J. Winer, D.R. Brown and K.M. Michels (1991) Statistical Principles in Experimental Design. third ed. New York, NY: McGraw-Hill
338. R.P. Abelson (1995) Statistics as Principled Argument. Hillsdale, NJ: Lawrence Erlbaum
339. Y. Benjamini and Y. Hochberg (1995) Controlling the false discovery rate: a practical and powerful approach to multiple testing. J. R. Soc. Ser. B 57(1), 289-300.
340. P. Bobko (2001) Correlation and Regression: Applications for Industrial Organizational Psychology and Management. second ed. Thousand Oaks, CA: Sage Publications
341. J. Cohen (1988) Statistical Power Analysis for the Behavioral Sciences. second ed. Hillsdale, NJ: Lawrence Erlbaum
342. L. Cohen, L. Manion and K. Morrison (2011) Research Methods in Education. seventh ed. New York, NY: Routledge
343. M. Cowles (1989) Statistics in Psychology: An Historical Perspective. Hillsdale, NJ: Lawrence Erlbaum
344. M. Gladwell (2008) Outliers: The Story of Success. New York, NY: Little, Brown, and Company
345. J.R. Lewis (2011) Practical Speech User Interface Design. Boca Raton, FL: Taylor & Francis
346. J.R. Lewis and J. Sauro (2016) Excel and R Companion to “Quantifying the User Experience”: Rapid Answers to Over 100 Examples and Exercises. second ed. Denver, CO: Create Space Publishing
347. Lewis, J.R., Utesch, B.S., Maher, D.E., 2013. UMUX-LITE—when there’s no time for the SUS. In: Proceedings of CHI 2013, Association for Computing Machinery, Paris, France, pp. 2099–2102.
348. M. Moinester and R. Gottfried (2014) Sample size estimation for correlations with pre-specified confidence interval. Quant. Methods Psychol. 10(2), 124-130.
349. C. Nass and S. Brave (2005) Wired for Speech: How Voice Activates and Advances the Human–Computer Relationship. Cambridge, MA: MIT Press
350. M.D. Polkosky and J.R. Lewis (2003) Expanding the MOS: development and psychometric evaluation of the MOS-R and MOS-X. Int. J. Speech Technol. 6 161-182.
351. A. Rutherford (2011) ANOVA and ANCOVA: A GLM Approach. Hoboken, NJ: John Wiley
352. Sauro, J., 2010. Were most software millionaires born around 1955? Available from: http://www.measuringu.com/1955.php
353. Sauro, J., 2011. Are women paid less than men in UX? Available from: http://www.measuringu.com/ux-gender.php
354. Sauro, J., 2013. Usability & Net Promoter benchmark report for retail websites. Available from: http://www.measuringu.com/products/retailReport
355. J. Sauro (2015) Customer Analytics for Dummies. Hoboken, NJ: John Wiley
356. Sauro, J., Lewis, J.R., 2009. Correlations among prototypical usability metrics: evidence for the construct of usability. In: Proceedings of CHI 2009, ACM, Boston, MA, pp. 1609–1618.
357. B.G. Tabachnick and L.S. Fidell (2012) Using Multivariate Statistics. sixth ed. New York, NY: Pearson
358. M. Thomson, D.J. MacInnis and C. Whan Park (2005) The ties that bind: measuring the strength of consumers’ emotional attachments to brands. J. Consum. Psychol. 15(1), 77-91.
359. J.R. Bergstrom and A.J. Schall (2014) Eye Tracking in User Experience Design. Boston, MA/Amsterdam: Elsevier
360. J.R. Lewis and J. Sauro (2016) Excel and R Companion to “Quantifying the User Experience: Practical Statistics for User Research”: Rapid Answers to Over 100 Examples and Exercises. second ed. Denver: Create Space Publishers
361. J. Sauro (2010) A Practical Guide to Measuring Usability. Denver, CO: Create Space Publishers
362. J. Sauro (2011) A Practical Guide to the System Usability Scale. Denver, CO: Create Space Publishers
363. D. Searles (1978) PSI burn: a study of physiological deterioration in parapsychological experimentation. Omni Mag. 1(3), 108-110.
364. T. Tullis and B. Albert (2013) Measuring the User Experience: Collecting, Analyzing, and Presenting Usability Metrics. second ed. Waltham, MA: Morgan Kaufmann
365. A. Vickers (2010) What is a p-value Anyway? Boston, MA: Addison-Wesley