Discriminant validity where there should be none: Positioning same-scale items in separated blocks of a questionnaire

Applied Psychological Measurement

Volumn 38, Issue 6, 2014, Pages 450-463

  Weijters, Bert ^a   De Beuckelaer, Alain ^a,b,c   Baumgartner, Hans ^d  

^a GHENT UNIVERSITY   (Belgium)

^b RENMIN UNIVERSITY OF CHINA   (China)

^c RADBOUD UNIVERSITY NIJMEGEN   (Netherlands)

^d PENNSYLVANIA STATE UNIVERSITY   (United States)

Author keywords

Confirmatory factor analysis; Discriminant validity; Grouping (survey) items; Inter item correlation; Randomizing (survey) items; Response styles; Survey methods

Indexed keywords

EID: 84923817509     PISSN: 01466216     EISSN: 15523497     Source Type: Journal    
DOI: 10.1177/0146621614531850     Document Type: Article

References (27)

1. Artino, A. R., Jr., Gehlbach, H., & Durning, S. J. (2011). AM last page: Avoiding five common pitfalls of survey design. Academic Medicine: Journal of the Association of American Medical Colleges, 86, 1327-1327.
2. Billiet, J., & Davidov, E. (2008). Testing the stability of an acquiescence style factor behind two interrelated substantive variables in a panel design. Sociological Methods & Research, 36, 542-562.
3. Conte, J. M., Dean, M. A., Ringenbach, K. L., Moran, S. K., & Landy, F. J. (2005). The relationship between work attitudes and job analysis ratings: Do rating scale type and task discretion matter? Human Performance, 18, 1-21.
4. Converse, J. M., & Presser, S. (1986). Survey questions: Handcrafting the standardized questionnaire. Beverly Hills, CA: Sage.
5. Emons, W. H. M. (2009). Detection and diagnosis of person misfit from patterns of summed polytomous item scores. Applied Psychological Measurement, 33, 599-619.
6. Fornell, C, & Larcker, D. F. (1981). Evaluating structural equation models with unobserved variables and measurement errors. Journal of Marketing Research, 18, 39-50.
7. Fowler, F. J. (2009). Survey research methods (4th ed.). Thousand Oaks, CA: Sage.
8. Franke, G. H. (1997). "The whole is more than the sum of its parts'': The effects of grouping and randomizing items on the reliability and validity of questionnaires. European Journal of Psychological Assessment, 13, 67-74.
9. Gehlbach, H., & Barge, S. (2012). Anchoring and adjusting in questionnaire responses. Basic and Applied Social Psychology, 34, 417-433.
10. Harrison, D. A., & McLaughlin, M. E. (1996). Structural properties and psychometric qualities of organizational self-reports: Field tests of connections predicted by cognitive theory. Journal of Management, 22, 313-338.
11. Hu, L.-T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6, 1-55.
12. Lastovicka, J. L., Bettencourt, L. A., Hughner, R. S., & Kuntze, R. J. (1999). Lifestyle of the tight and frugal: Theory and measurement. Journal of Consumer Research, 26, 85-98.
13. Lehmann, D. R. (1988). An alternative procedure for assessing convergent and discriminant validity. Applied Psychological Measurement, 12, 411-423.
14. McFarland, L. A., Ryan, A. M., & Ellis, A. (2002). Item placement on a personality measure: Effects on faking behavior and test measurement properties. Journal of Personality Assessment, 78, 348-369.
15. Rick, S. I., Cryder, C. E., & Loewenstein, G. (2008). Tightwads and spendthrifts. Journal of Consumer Research, 34, 767-782.
16. Schriesheim, C. A. (1981a). The effect of grouping or randomizing items on leniency response bias. Educational and Psychological Measurement, 41, 401-411.
17. Schriesheim, C. A. (1981b). Leniency effects on convergent and discriminant validity for grouped questionnaire items: A further investigations. Educational and Psychological Measurement, 41, 1093-1099.
18. Schriesheim, C. A., & DeNisi, A. S. (1980). Item presentation as an influence on questionnaire validity: A field experiment. Educational and Psychological Measurement, 40, 175-182.
19. Schriesheim, C. A., Solomon, E., & Kopelman, R. E. (1989a). The effect of grouped versus randomized questionnaire format on scale reliability and validity: A three-study investigation. Educational and Psychological Measurement, 49, 487-508.
20. Schriesheim, C. A., Solomon, E., & Kopelman, R. E. (1989b). Grouped versus randomized format: An investigation of scale convergent and discriminant validity using LISREL confirmatory factor analysis. Applied Psychological Measurement, 13, 19-31.
21. Shipp, A. J., Edwards, J. R., & Lambert, L. S. (2009). Conceptualization and measurement of temporal focus: The subjective experience of the past, present, and future. Organizational Behavior and Human Decision Processes, 110, 1-22.
22. Tourangeau, R., Couper, M. P., & Conrad, F. (2004). Spacing, position, and order: Interpretive heuristics for visual features of survey questions. Public Opinion Quarterly, 68, 368-393.
23. Tourangeau, R., Rips, L. J., & Rasinski, K. A. (2000). The psychology of survey response. Cambridge, England: Cambridge University Press.
24. Weijters, B., Cabooter, E., & Schillewaert, N. (2010). The effect of rating scale format on response styles: The number of response categories and response category labels. International Journal of Research in Marketing, 27, 236-247.
25. Weijters, B., Geuens, M., & Schillewaert, N. (2009). The proximity effect: The role of interitem distance on reverse-item bias. International Journal of Research in Marketing, 26, 2-12.
26. Weijters, B., Geuens, M., & Schillewaert, N. (2010a). The individual consistency of acquiescence and extreme response style in self-report questionnaires. Applied Psychological Measurement, 34, 105-121.
27. Weijters, B., Geuens, M., & Schillewaert, N. (2010b). The stability of individual response styles. Psychological Methods, 15, 96-110.