Modeling Instructional Sensitivity Using a Longitudinal Multilevel Differential Item Functioning Approach

Journal of Educational Measurement

Volumn 51, Issue 4, 2014, Pages 381-399

  Naumann, Alexander ^a   Hochweber, Jan ^a,b   Hartig, Johannes ^a  

^a GERMAN INSTITUTE FOR INTERNATIONAL EDUCATIONAL RESEARCH   (Germany)

^b University of Teacher Education StGallen   (Switzerland)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84915809460     PISSN: 00220655     EISSN: 17453984     Source Type: Journal    
DOI: 10.1111/jedm.12051     Document Type: Article

References (51)

1. Agresti, A. (1990). Categorical data analysis. New York, NY: Wiley.
2. Airasian, P. W., & Madaus, G. F. (1983). Linking testing and instruction: Policy issues. Journal of Educational Measurement, 20, 103-118.
3. Bafumi, J., Gelman, A., Park, D. K., & Kaplan, N. (2005). Practical issues in implementing and understanding Bayesian ideal point estimation. Political Analysis, 13, 171-187.
4. Baker, E. L. (1994). Making performance assessment work: The road ahead. Educational Leadership, 51(6), 58-62.
5. Burstein, L. (1989, March). Conceptual considerations in instructionally sensitive assessment. Paper presented at the Annual Meeting of the American Educational Research Association, San Francisco, CA.
6. Clauser, B. E., Nungester, R. J., & Swaminathan, H. (1996). Improving the matching for DIF analysis by conditioning on both test score and an educational background variable. Journal of Educational Measurement, 33, 453-464.
7. Cox, R. C., & Vargas, J. S. (1966, April). A comparison of item-selection techniques for norm referenced and criterion referenced tests. Paper presented at the Annual Conference of the National Council on Measurement in Education, Chicago, IL.
8. Creemers, B. P. M., & Kyriakides, L. (2008). The dynamics of educational effectiveness: A contribution to policy, practice and theory in contemporary schools. London: Routledge.
9. Crehan, K. D. (1974). Item analysis for teacher-made mastery tests. Journal of Educational Measurement, 11, 255-261.
10. Cronbach, L. J., & Meehl, P. E. (1955). Construct validity in psychological tests. Psychological Bulletin, 52, 281-302.
11. D'Agostino, J. V., Welsh, M. E., & Corson, N. M. (2007). Instructional sensitivity of a state standards-based assessment. Educational Assessment, 12, 1-22.
12. DESI-Konsortium (Ed.). (2008). Unterricht und Kompetenzerwerb in Deutsch und Englisch: Ergebnisse der DESI-Studie [Learning and instruction of German and English. Results from the DESI study]. Weinheim, Germany: Beltz.
13. Eichler, W. (2007). Sprachbewusstheit [Language awareness]. In B. Beck & E. Klieme (Eds.), Sprachliche Kompetenzen: Konzepte und Messung (pp. 147-157). Weinheim, Germany: Beltz.
14. Geisinger, K. F., & McCormick, C. M. (2010). Adopting cut scores: Post-standard-setting panel considerations for decision makers. Educational Measurement: Issues and Practice, 29, 38-44.
15. Gelman, A. B., & Hill, J. (2006). Data analysis using regression and multilevel/hierarchical models. Cambridge, UK: Cambridge University Press.
16. Glaser, R. (1963). Instructional technology and the measurement of learning outcomes: Some questions. American Psychologist, 18, 519-521.
17. Haladyna, T. M. (2004). Developing and validating multiple-choice test items. Mahwah, NJ: Lawrence Erlbaum.
18. Haladyna, T. M., & Roid, G. H. (1981). The role of instructional sensitivity in the empirical review of criterion-referenced test items. Journal of Educational Measurement, 18, 39-53.
19. Hardy, I., Hertel, S., Kunter, M., Klieme, E., Warwas, J., Büttner, G., & Lühken, A. (2011). Adaptive Lerngelegenheiten in der Grundschule: Merkmale, methodisch-didaktische Schwerpunktsetzungen und erforderliche Lehrerkompetenzen [Adaptive learning environments in primary school]. Zeitschrift für Pädagogik, 57, 819-833.
20. Holland, P. W., & Wainer, H. (Eds.) (1993). Differential item functioning: Theory and practice. Hillsdale, NJ: Lawrence Erlbaum.
21. Ing, M. (2008). Using instructional sensitivity and instructional opportunities to interpret students' mathematics performance. Journal of Educational Research and Policy Studies, 8(1), 23-43.
22. Kamata, A. (2001). Item analysis by the hierarchical generalized linear model. Journal of Educational Measurement, 38, 79-93.
23. Kane, M. T. (2001). Current concerns in validity theory. Journal of Educational Measurement, 38, 319-342.
24. Kane, M. T. (2013). Validating the interpretations and uses of test scores. Journal of Educational Measurement, 50, 1-73.
25. Kleickmann, T., Hardy, I., Möller, K., Pollmeier, J., Tröbst, S., & Beinbrech, C. (2010). Die Modellierung naturwissenschaftliche Kompetenz im Grundschulalter: Theoretische Konzeption und Testkonstruktion [Modeling scientific competence of primary school children]. Zeitschrift für Didaktik der Naturwissenschaften, 16, 263-281.
26. Kolen, M. J., & Brennan, R. L. (2004). Test equating, scaling, and linking: Methods and practices. New York, NY: Springer Science.
27. Linn, R. L. (1983). Testing and instruction: Links and distinctions. Journal of Educational Measurement, 20, 179-189.
28. Linn, R. L., & Harnisch, D. L. (1981). Interactions between item content and group membership on achievement test items. Journal of Educational Measurement, 18, 109-118.
29. Lunn, D., Jackson, C., Best, N., Thomas, A., & Spiegelhalter, D. (2010). The BUGS Book: A practical introduction to Bayesian analysis. London, UK: Chapman & Hall.
30. Lunn, D., Thomas, A., Best, N., & Spiegelhalter, D. (2000). WinBUGS - A Bayesian modelling framework: concepts, structure, and extensibility. Statistics and Computing, 10, 325-337.
31. Masters, G. N. (1982). A Rasch model for partial credit scoring. Psychometrika, 47, 149-174.
32. Messick, S. (1989). Validity. In R. L. Linn (Ed.), Educational measurement (3rd ed., pp. 13-103). Phoenix, AZ: Oryx Press.
33. Meulders, M., & Xie, Y. (2004). Person-by-item predictors. In P. De Boeck & M. Wilson (Eds.), Explanatory item response models: A generalized linear and nonlinear approach (pp. 213-240). New York, NY: Springer.
34. Muthén, B. O. (1989). Using item-specific instructional information in achievement modeling. Psychometrika, 54, 385-396.
35. Muthén, B. O., Kao, C.-F., & Burstein, L. (1991). Instructionally sensitive psychometrics: Application of a new IRT-based detection technique to mathematics achievement test items. Journal of Educational Measurement, 28, 1-22.
36. Pellegrino, J. W. (2002). Knowing what students know. Issues in Science and Technology, 19(2), 48-52.
37. Polikoff, M. S. (2010). Instructional sensitivity as a psychometric property of assessments. Educational Measurement: Issues and Practice, 29(4), 3-14.
38. Popham, J. W. (1971). Indices of adequacy for criterion-reference test items. In J. W. Popham (Ed.), Criterion-referenced measurement (an introduction) (pp. 79-98). Englewood Cliffs, NJ: Educational Technology Publications.
39. Popham, J. W. (2007). Instructional insensitivity of tests: Accountability's dire drawback. Phi Delta Kappan, 89, 146-155.
40. R Development Core Team. (2008). R: A language and environment for statistical computing. Vienna, Austria. Retrieved from http://www.R-project.org
41. Robitzsch, A. (2009). Methodische Herausforderungen bei der Kalibrierung von Leistungstests [Methodical challenges in the calibration of performance tests]. In D. Granzer, O. Köller, & A. Bremerich-Vos (Eds.), Bildungsstandards Deutsch und Mathematik (pp. 42-106). Weinheim, Germany: Beltz.
42. Ruiz-Primo, M. A., Li, M., Wills, K., Giamellaro, M., Lan, M.-C., Mason, H., & Sands, D. (2012). Developing and evaluating instructionally sensitive assessments in science. Journal of Research in Science Teaching, 49, 691-712.
43. Ruiz-Primo, M. A., Shavelson, R. J., Hamilton, L., & Klein, S. P. (2002). On the evaluation of systematic science education reform: Search for instructional sensitivity. Journal of Research in Science Teaching, 39, 369-393.
44. Spiegelhalter, D. J., Best, N. G., Carlin, B. P., & van der Linde, A. (2002). Bayesian measures of model complexity and fit. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 64, 583-639.
45. Sturtz, S., Ligges, U., & Gelman, A. (2005). R2WinBUGS: A package for running WinBUGS from R. Journal of Statistical Software, 12(3), 1-16.
46. Taylor, J., Stecher, B., O'Day, J., Naftel, S., & Le Floch, K. C. (2010). State and local implementation of the No Child Left Behind Act: Volume IX-Accountability under NCLB. Washington, DC: U.S. Department of Education.
47. Travers, K. J., & Westbury, I. (1989). The IEA study of mathematics I: Analysis of mathematics curricula (Vol. 1). Elmsford, NY: Pergamon Press.
48. Vosniadou, S. (2007). The cognitive-situative divide and the problem of conceptual change. Educational Psychologist, 42, 55-66.
49. Warm, T. A. (1989). Weighted likelihood estimation of ability in item response theory. Psychometrika, 54, 427-450.
50. Wright, B. D., & Linacre, J. M. (1994). Reasonable mean-square fit values. Rasch Measurement Transactions, 8, 370.
51. Yoon, B., & Resnick, L. B. (1998). Instructional validity, opportunity to learn, and equity: New standards examinations for the California Mathematics Renaissance. Los Angeles, CA: Center for the Study of Evaluation.