Blocked versus randomized format of questionnaires: A confirmatory multigroup analysis

Educational and Psychological Measurement

Volumn 66, Issue 6, 2006, Pages 961-974

  Sparfeldt, Jörn R ^a   Schilling, Susanne R ^a   Rost, Detlef H ^a   Thiel, Alexandra ^a  

^a PHILIPPS UNIVERSITY MARBURG   (Germany)

Author keywords

Academic self concept; Factorial structure; Item blocking; Reliability; Validity

Indexed keywords

EID: 33750733887     PISSN: 00131644     EISSN: 15523888     Source Type: Journal    
DOI: 10.1177/0013164405285906     Document Type: Article

References (42)

1. Alterman, A. I., McDermott, P. A., Mulvaney, F. D., Caccioly, J. S., Rutherford, M. J., Searles, J. S., et al. (2003). Comparison of embedded and isolated administrations of the California Psychological Inventory's Socialization subscale. Psychological Assessment, 15, 64 - 70.
2. American Educational Research Association. (1999). Standards for educational and psychological testing. Washington, DC: Author.
3. Arbuckle, J. L. (1999). Amos 4.01 [Computer software]. Chicago: Smallwaters.
4. Baehr, M. E. (1953). A simplified procedure for the measurement of employee attitudes. Journal of Applied Psychology, 37, 163 - 167.
5. Byrne, B. M. (1996). Academic self-concept: Its structure, measurement, and relation to academic achievement. In B. A. Bracken (Ed.), Handbook of self-concept: Development, social, and clinical considerations (pp. 287 - 316). New York: John Wiley.
6. Byrne, B. M. (2001). Structural equationmodellingwith AMOS: Basic concepts,applicationsand programming. Mahwah, NJ: Lawrence Erlbaum.
7. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). New York: Lawrence Erlbaum.
8. Costa, P. T., & McCrae, R. R. (1992). Revised NEO Personality Inventory (NEO-PI) and NEO Five Factor Inventory: Professional manual. Odessa, FL: Psychological Assessment Resources.
9. Derogatis, L. R. (1986). SCL-90-R: Self-Report Symptom Inventory. In Collegium Internationale Psychiatriae Scalarum (Eds.), Internationale Skalen für Psychiatrie [International psychiatric scales]. Weinheim, Germany: Beltz.
10. Eysenck, H.-J. (1995). The Eysenck Personality Profiler and Eysenck's theory of personality. London: Corporate Assessment Network, Ltd.
11. Franke, G. H. (1996). Effekte der Typographieund Itempositionierungin der Fragebogendiagnostik[Effects of typography and item position in diagnostic inventories]. Zeitschrift für Differentielle und Diagnostische Psychologie, 17, 187 - 200.
12. Franke, G. H. (1997). "The whole is more than the sum of its parts": The effect of grouping and randomizing items on the reliability and validity of questionnaires. European Journal of Psychological Assessment, 13, 67 - 74.
13. Franke, G. H. (1998). Auswirkungen der PC-gestützten Anwendung des Freiburger Persönlichkeitsinventars unter besonderer Berücksichtigung der Itemreihenfolge: Eine experimentelle Studie [Effects of computer administration on the Freiburger Personality Inventory with a special focus on the item sequence: An experimental study]. Zeitschrift für Differentielle und Diagnostische Psychologie, 19, 187-199.
14. Franke, G. H. (1999). Effekte der Computeradministration bei der Symptom-Checkliste (SCL-90-R) unter besonderer Berücksichtigung der Itemreihenfolge [Effects of computer administration on the Symptom Checklist (SCL-90-R) with a special focus on the item sequence]. Diagnostica, 45, 147 - 153.
15. Franke, G. H., & Stäcker, K.-H. (1995). Reliabilität und Validität der Symptom-Checkliste (SCL-90-R; Derogatis, 1986) bei Standardreihenfolge versus inhaltshomogener Itemblockbildung [Reliability and validity of the Symptom Checklist (SCL-90-R; Derogatis, 1986) in standardized versus homogenous item-blocked sequence]. Diagnostica, 41, 349 - 373.
16. Gough, H. G., & Bradley, P. (1996). The California Personality Inventory manual (3rd ed.). Palo Alto, CA: Consulting Psychologists Press.
17. Knowles, E. S. (1988). Item context effects on personality scales: Measuring changes the measure. Journal of Personality and Social Psychology, 55, 312 - 320.
18. Knowles, E. S., & Byers, B. (1996). Reliability shifts in measurement reactivity: Driven by content engagement or self-engagement? Journal of Personality and Social Psychology, 70, 1080 - 1090.
19. Krampen, G. (1993). Effekte von Bewerbungsinstruktion und Subskalenextraktion in der Fragebogendiagnostik [Effects of a application instruction and extraction of subscales on questionnaire assessment]. Diagnostica, 39, 97 - 108.
20. Krampen, G., Hense, H., & Schneider, J. F. (1992). Reliabilität und Validität von Fragebogenskalen bei Standardreihenfolge versus inhaltshomogener Blockbildung ihrer Items [Reliability and validity of questionnaire subscales in standardized versus homogenous item-blocked sequence]. Zeitschrift für Experimentelle und Angewandte Psychologie, 39, 229 - 248.
21. Lyman, H. B. (1949). A comparison of the use of scrambled and blocked items in a multi-scale school attitude inventory. Journal of Educational Research, 43, 287 - 292.
22. Marsh, H. W. (1986). Verbal and math self-concepts: An internal/external frame of reference model. American Educational Research Journal, 23, 129 - 149.
23. Marsh, H. W. (1990). Self-Description Questionnaire II: Manual. Sydney, Australia: University of Western Sidney.
24. Marsh, H. W. (1993). The multidimensional structure of academic self-concept: Invariance over gender and age. American Educational Research Journal, 30, 841 - 860.
25. Marsh, H. W., Balla, J. R., & Hau, K. T. (1996). An evaluation of incremental fit indices: A clarification of mathematical and empirical process. In G. A. Marcoulides & R. E. Schumacker (Eds.), Advanced structural equation modeling techniques (pp. 315-353). Hillsdale, NJ: Lawrence Erlbaum.
26. Marsh, H. W., Balla, J. R., & McDonald, R. P. (1988). Goodness-of-fit indexes in confirmatory factor analysis: The effect of sample size. Psychological Bulletin, 103, 391 - 410.
27. Marsh, H. W., Craven, R., & Debus, R. (1999). Separation of competency and affect components of multiple dimensions of academic self-concept: A developmental perspective. Merrill-Palmer Quarterly, 45, 567 - 601.
28. Marsh, H. W., & Yeung, A. S. (1999). The lability of psychological ratings: The chameleon effect in global self-esteem. Personality and Social Psychology Bulletin, 25, 49 - 64.
29. Möller, J., & Köller, O. (2004). Die Genese akademischer Selbstkonzepte: Effekte dimensionaler und sozialer Vergleiche [On the development of academic self-concepts: The impact of social and dimensional comparisons]. Psychologische Rundschau, 55, 19-27.
30. Ortner, T. M. (2004). On changing the position of items in personality questionnaires: Analyzing effects of item sequence using IRT. Psychology Science, 46, 466 - 476.
31. Rost, D. H., & Hoberg, K. (1997). Itempositionsveränderungen in Persönlichkeitsfragebogen: Methodischer Kunstfehler oder tolerierbare Praxis? [Changing the position of items in personality questionnaires: Methodological malpractice or tolerable practice?]. Diagnostica, 43, 97 - 112.
32. Rost, D. H., & Sparfeldt, J. R. (2002). Facetten des schulischen Selbstkonzepts. Ein Verfahren zur Messung des differentiellen Selbstkonzepts schulischer Leistungen und Fähigkeiten (DISK-Gitter) [Facets of academic self-concept. Development of a self-concept grid: Psychometric properties and some validity data]. Diagnostica, 48, 130-140.
33. Rost, D. H., Sparfeldt, J. R., & Schilling, S. R. (2006). Differentielles Schulisches Selbstkonzept-Gitter [Differential Academic Self-Concept Grid]. Göttingen, Germany: Hogrefe.
34. Schilling, S. R., Sparfeldt, J. R., Rost, D. H., & Nickels, G. (2005). Schulische Selbstkonzepte: Zur Validität einer erweiterten Version des Differentiellen Selbstkonzept Gitters (DISK-Gitter) [Facets of academic self-concept: Validity of the Differential Self-Concept Grid (DISC-Grid)]. Diagnostica, 51, 21 - 28.
35. Schriesheim, C. A., & DeNisi, A. S. (1980). Item presentation as an influence on questionnaire validity: A field experiment. Educational and Psychological Measurement, 40, 175 - 182.
36. Schriesheim, C. A., Kopelman, R. E., & Solomon, E. (1989). The effect of grouped versus randomizedquestionnaire format on scale reliability and validity: A three-study investigation. Educational and Psychological Measurement, 49, 487 - 508.
37. Schwarz, N. (1999). Self-reports: How the questions shape the answers. American Psychologist, 54, 93 - 105.
38. Solomon,E., & Kopelman, R. E. (1984). Questionnaire format and scale reliability: An examinationof three modes of item presentation. Psychological Reports, 54, 447 - 452.
39. Spielberger, C. D., Gorsuch, R. L., & Lushene, R. E. (1970). Manual for the State-Trait Anxiety Inventory. Palo Alto, CA: Consulting Psychologists Press.
40. Steinberg, L. (1994). Context and serial-order effects in personality measurement: Limits on the generality of measuring changes the measure. Journal of Personality and Social Psychology, 66, 341 - 349.
41. Tourangeau, R., & Rasinski, K. A. (1988). Cognitive processes underlying context effects in attitude measurement. Psychological Bulletin, 103, 299 - 314.
42. Tucker, L. R., & Lewis, C. (1973). A reliability coefficient for maximum likelihood factor analysis. Psychometrika, 38, 1 - 10.