When designing usability questionnaires, does it hurt to be positive?

Conference on Human Factors in Computing Systems - Proceedings

Volumn , Issue , 2011, Pages 2215-2223

  Sauro, Jeff ^a   Lewis, James R ^b  

^a ORACLE CORPORATION   (United States)

^b IBM   (United States)

Author keywords

Acquiescent bias; Satisfaction measures; Standardized questionnaires; System Usability Scale (SUS); Usability evaluation

Indexed keywords

USABILITY ENGINEERING;

ACQUIESCENT BIAS; EXTREME RESPONSE; ORIGINAL SYSTEMS; RESPONSE BIAS; SATISFACTION MEASURE; STANDARD SYSTEM; STANDARDIZED QUESTIONNAIRE; SYSTEM USABILITY; SYSTEM USABILITY SCALE; USABILITY EVALUATION;

SURVEYS;

EID: 79958129752     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1978942.1979266     Document Type: Conference Paper

References (30)

1. Anastasi, A. (1982). Psychological testing (5th ed.). New York: Macmillan.
2. Anderson, A. B., Basilevsky, A., & Hum, D. P. J. (1983). Measurement: Theory and techniques. In P. H. Rossi, J. D. Wright, & A. B. Anderson (Eds.), Handbook of Survey Research (pp. 231-187). New York, NY: Academic Press.
3. Bangor, A., Kortum, P. T., & Miller, J. T. (2008). An empirical evaluation of the System Usability Scale. International Journal of Human-Computer Interaction, 6, 574-594.
4. Barnette, J. J. (2000). Effects of stem and Likert response option reversals on survey internal consistency: If you feel the need, there is a better alternative to using those negatively worded stems. Educational and Psychological Measurement, 60(3), 361-370.
5. Borsci, S., Federici, S., & Lauriola, M. (2009). On the dimensionality of the System Usability Scale: A test of alternative measurement models. Cognitive Processes, 10, 193-197.
6. Brooke, J. (1996). SUS: A quick and dirty usability scale. In P.W. Jordan, B. Thomas, B.A. Weerdmeester & I.L. McClelland (Eds.), Usability Evaluation in Industry (pp. 189-194). London, UK: Taylor & Francis.
7. Chin, J. P, Diehl, V. A., & Norman, K. (1988). Development of an instrument measuring user satisfaction of the human-computer interface. In Conference on Human Factors in Computing Systems (pp. 213-218). New York, NY: Association for Computing Machinery.
8. Finstad, K. (2006). The System Usability Scale and non-native English speakers. Journal of Usability Studies, 1, 185-188.
9. Finstad, K. (2010). Response interpolation and scale sensitivity: Evidence against 5-point scales. Journal of Usability Studies, 5(3), 104-110.
10. Karn, K., Little, A., Nelson, G., Sauro, J. Kirakowski, J., Albert, W. & Norman, K., (2008) Subjective Ratings of Usability: Reliable or Ridiculous? Panel Presentation at the Usability Professionals Association (UPA 2008) Conference Baltimore, MD.
11. Kirakowski, J., & Corbett, M. (1993). SUMI: The Software Usability Measurement Inventory. British Journal of Educational Technology, 24, 210-212.
12. Lewis, J. R. (1995). IBM computer usability satisfaction questionnaires: Psychometric evaluation and instructions for use. International Journal of Human-Computer Interaction, 7, 57-78.
13. Lewis, J. R. (2002). Psychometric evaluation of the PSSUQ using data from five years of usability studies. International Journal of Human-Computer Interaction, 14(3&4), 463-488. (Pubitemid 36611301)
14. Lewis, J. R (2006). Usability testing. In G. Salvendy (Ed.), Handbook of Human Factors and Ergonomics (3rd ed.) (pp. 1275-1316). New York, NY: John Wiley.
15. Lewis, J. R., & Sauro, J. (2009). The factor structure of the System Usability Scale. In M. Kurosu (Ed.), Human Centered Design, HCII 2009 (pp. 94-103). Berlin, Germany: Springer-Verlag.
16. Lewis, J. R. (2011). Practical speech user interface design. Boca Raton, FL: Taylor & Francis.
17. Nunnally, J. C. (1978). Psychometric theory (2nd ed.). New York, NY: McGraw-Hill.
18. Molich, R., Kirakowski, J., Sauro, J., & Tullis, T., (2009) Comparative Usability Task Measurement Workshop (CUE-8) at the UPA 2009 Conference in Portland, OR.
19. Pilotte, W. J., & Gable, R. K. (1990). The impact of positive and negative item stems on the validity of a computer anxiety scale. Educational and Psychological Measurement, 50, 603-610.
20. Ramos, L. (1993). The effects of on-hold telephone music on the number of premature disconnections to a statewide protective services abuse hot line. Journal of Music Therapy, 30(2), 119-129.
21. Sauro, J. (2010). That's the worst website ever!: Effects of extreme survey items. www.measuringusability.com/blog/extreme-items.php (last viewed 9/23/2010).
22. Schmitt N., & Stuits, D. (1985) Factors defined by negatively keyed items: The result of careless respondents? Applied Psychological Measurement, 9(4), 367-373.
23. Schriesheim, C.A., & Hill, K.D. (1981). Controlling acquiescence response bias by item reversals: the effect on questionnaire validity. Educational and Psychological Measurement, 41(4), 1101-1114.
24. Spector, P., Van Katwyk, P., Brannick, M., & Chen, P. (1997). When two factors don't reflect two constructs: How item characteristics can produce artifactual factors Journal of Management, 23(5), 659-677. (Pubitemid 127173456)
25. Stewart, T. J., & Frye, A. W. (2004). Investigating the use of negatively-phrased survey items in medical education settings: Common wisdom or common mistake? Academic Medicine, 79(10 Supplement), S1-S3.
26. SPSS. (2003). SPSS for Windows, Rel. 12.0.1. (2003). Chicago, IL: SPSS Inc.
27. Thurstone, L. L. (1928). Attitudes can be measured. American Journal of Sociology, 33, 529-554.
28. Weijters, B., Cabooter, E., & Schillewaert, N. (2010). The effect of rating scale format on response styles: the number of response categories and response category labels. International Journal of Research in Marketing, 27, 236-247.
29. Wong, N., Rindfleisch, A., & Burroughs, J. (2003). Do reverse-worded items confound measures in cross-cultural consumer research? The case of the material values scale. Journal of Consumer Research, 30, 72-91.
30. Zviran, M., Glezer, C., & Avni, I. (2006). User satisfaction from commercial web sites: The effect of design and use. Information & Management. 43, 157-178. (Pubitemid 43112420)