Examining the attitude-achievement paradox in PISA using a multilevel multidimensional IRT model for extreme response style

Large-Scale Assessments in Education

Volumn 3, Issue 1, 2015, Pages

  Lu, Yi ^a   Bolt, Daniel M ^b  

^a American College Testing Incorporated   (United States)

^b UNIVERSITY OF WISCONSIN MADISON   (United States)

Author keywords

Extreme response style (ERS); Multidimensional IRT; Multilevel models; PISA

Indexed keywords

EID: 84971356649     PISSN: None     EISSN: 21960739     Source Type: Journal    
DOI: 10.1186/s40536-015-0012-0     Document Type: Article

References (28)

1. Baumgartner, H, & Steenkamp, J-BEM. (2001). Response styles in marketing research: A cross-national investigation. Journal of Marketing Research, 38, 143–156.
2. Bock, RD. (1972). Estimating item parameters and latent ability when the responses are scored in two or more nominal categories. Psychometrika, 37, 29–51.
3. Böckenholt, U. (2012). Modeling multiple response processes in judgment and choice. Psychological methods, 17(4), 665–678.
4. Bolt, DM, & Johnson, TR. (2009). Addressing score bias and differential item functioning due to individual differences in response style. Applied Psychological Measurement, 33, 335–352.
5. Bolt, DM, & Newton, J. (2011). Multiscale measurement of extreme response style. Educational and Psychological Measurement, 71, 814–833.
6. Bolt, DM, Lu, Y, & Kim, J-S. (2014). Measurement and control of response styles using anchoring vignettes: A model-based approach. Psychological Methods, 19, 528–541.
7. Buckley, J (2009). Cross-national response styles in international educational assessment: Evidence from PISA 2006. NCES Conference on the Program for International Student Assessment: What we can learn from PISA (Downloaded from https://edsurveys.rti.org/PISA/documents/Buckley_PISAresponsestyle.pdf on May 16, 2010).
8. Bybee, R, & McCrae, B. (2007). Scientific literacy and student attitudes: Perspectives from PISA 2006 science. International Journal of Science Education, 33, 7–26.
9. Clarke, I. (2000). Extreme response style in cross-cultural research. Journal of Social Behavior and Personality, 15, 137–152.
10. Greenleaf, EA. (1992). Measuring extreme response style. Public Opinion Quarterly, 56, 328–361.
11. Johnson, TR, & Bolt, DM. (2010). On the use of factor-analytic multinomial logit item response models to account for individual differences in response style. Journal of Educational and Behavioral Statistics, 35(1), 92–114.
12. Johnson, T, Kulesa, P, Cho, YI, & Shavitt, S. (2005). The relation between culture and response styles: Evidence from 19 countries. Journal of Cross-Cultural Psychology, 36, 264–277.
13. Khorramdel, L, & von Davier, M. (2014). Measuring response styles across the Big Five: A multiscale extension of an approach using multinomial processing trees. Multivariate Behavioral Research, 49(2), 161–177.
14. King, G, Murray, CJL, Salomon, JA, & Tandon, A. (2004). Enhancing the validity and cross-cultural comparability of survey research. American Political Science Review, 98, 191–207.
15. Loveless, T. (2006). The 2006 Brown Center report on American education: How well are American students learning? Washington, D. C.: Brookings Institution.
16. Lu, Y. (2012). A multilevel multidimensional item response theory model to address the role of response style on measurement of attitudes in PISA 2006. University of Wisconsin, Madison: Doctoral dissertation, 164 pages.
17. Marsh, HW, Seaton, M, Trautwein, U, Ludtke, O, Hau, K-T, O’Mara, AJ, & Craven, RG. (2008). The big fish little pond effect stands up to critical scrutiny: Implications for theory, methodology, and future research. Educational Psychology Review, 20, 319–350.
18. Meisenberg, G, & Williams, A. (2008). Are acquiescent and extreme response styles related to low intelligence and education? Personality and Individual Differences, 44, 1539–1550.
19. Moors, G. (2003). Diagnosing response style behavior by means of a latent-class factor approach. Socio-demographic correlates of gender role attitudes and perceptions of ethnic discrimination reexamined. Quality and Quantity, 37(3), 277–302.
20. Rost, J, Carstensen, C, & von Davier, M. (1997). Applying the mixed Rasch model to personality questionnaires. In J Rost & R Langeheine (Eds.), Applications of latent trait and latent class models in the social sciences (pp. 324–332). Munster: Germany: Waxmann.
21. Spiegelhalter, D, Thomas, A, & Best, N. (2004). WinBUGS version 1.4. Cambridge, UK: MRC Biostatistics Unit, Institute of Public Health [Computer program].
22. Van de Gaer, E, & Adams, R. (2010). The modeling of response style bias: An answer to the attitude-achievement paradox? Denver, CO: Paper presented at the annual meeting of the American Educational Research Association. http://www.acer.edu.au/files/vandegaer_paper_aera2010.pdf
23. Van Herk, H, Poortinga, YH, & Verhallen, TMM. (2004). Response styles in rating scales: Evidence of method bias in data from six EU countries. Journal of Cross-Cultural Psychology, 35, 346–360.
24. Vermunt, JK, & Magidson, J. (2005). Technical Guide for Latent Gold 4.0: Basic and Advanced. Belmont, MA: Statistical Innovations, Inc.
25. Vermunt, JK, & Magidson, J. (2008). LG-Syntax User’s Guide: Manual for Latent Gold 4.5 Syntax Module. Belmont, MA: Statistical Innovations, Inc.
26. von Davier, M, Gonzalez, E, & Mislevy, RJ. (2009). What are plausible values and why are they useful? In M Von Davier & D Hastedt (Eds.), IERI Monograph Series: Issues and methodologies in large scale assessments (Vol. 2, pp. 9–36). Hamburg, Germany: IERI Institute.
27. Weijters, B, Geuens, M, & Schillewaert, N. (2010). The stability of individual response styles. Psychological Methods, 15, 96–110.
28. Wetzel, E, Carstensen, CH, & Böhnke, JR. (2013). Consistency of extreme response style and non-extreme response style across traits. Journal of Research in Personality, 47(2), 178–189.