Validating measurement of knowledge integration in science using multiple-choice and explanation items

Applied Measurement in Education

Volumn 24, Issue 2, 2011, Pages 115-136

  Lee, Hee Sun ^a   Liu, Ou Lydia ^b   Linn, Marcia C ^a  

^a UNIVERSITY OF CALIFORNIA   (United States)

^b EDUCATIONAL TESTING SERVICE   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 79953136183     PISSN: 08957347     EISSN: None     Source Type: Journal    
DOI: 10.1080/08957347.2011.554604     Document Type: Article

References (53)

1. American Association for the Advancement of Science (AAAS). (1993). Benchmarks for science literacy. New York: Oxford University Press.
2. Baxter, G. P., & Glaser, R. (1998). Investigating the cognitive complexity of science assessments. Educational Measurement: Issues and Practice, 17(3), 37-45.
3. Bennett, R. E., Rock, D. A., & Wang, M. (1991). Equivalence of free-response and multiple-choice items. Journal of Educational Measurement, 28(1), 77-92.
4. Bransford, J. D., Brown, A. L., & Cocking, K. R. (2000). How people learn: Brain, mind, experience, and school. Washington, DC: National Academy Press.
5. Bransford, J. D., & Schwartz, D. L. (1999). Rethinking transfer: A simple proposal with multiple implications. Review of Research in Education, 24, 61-100.
6. Briggs, D. C., Alonzo, A. C., Schwab, C., & Wilson, M. (2006). Diagnostic assessment with ordered multiple-choice items. Educational Assessment 11(1), 33-63.
7. Bridgeman, B., & Rock, D. A. (1993). Relationships among multiple-choice and open-ended analytical questions. Journal of Educational Measurement, 30(4), 313-329.
8. Chi, M. T. H., De Leeuw, N., Chiu, M., & Lavancher, C. (1994). Eliciting self-explanations improves understanding. Cognitive Science, 18, 439-477.
9. Clark, D. B., & Linn, M. C. (2003).Scaffolding knowledge integration through curricular depth. Journal of Learning Sciences, 12, 451-494.
10. Delandshere, G., & Petrosky, A. R. (1998). Assessment of complex performances: Limitations of key measurement assumptions. Educational Researcher, 27, 14-24.
11. Ebel, R. L. (1954). Procedures for the analysis of classroom tests. Educational and Psychological Measurement, 14, 352-364.
12. Ercikan, K., Schwarz, R. D., Julian, M. W., Burket, G. R., Weber, M. M., & Link, V. (1998). Calibration and scoring of tests with multiple-choice and constructed-response item types. Journal of Educational Measurement, 35, 137-154.
13. Frederiksen, N. (1984). The real test bias: Influence of testing on teaching and learning. American Psychologist, 39, 193-202.
14. Hempel, C. (1966). Philosophy of natural science. Upper Saddle River, NJ: Prentice Hall.
15. Hunt, E., & Minstrell, J. (1994). A cognitive approach to the teaching of physics. In K. McGilly (Ed.), Classroom lessons: integrating cognitive theory and classroom practice (pp. 51-74). Cambridge, MA: MIT Press.
16. Hurd, A. W. (1932). Comparisons of short answer and multiple choice tests covering identical subject content. Journal of Educational Research, 26(1), 28-30.
17. International Association for the Evaluation of Educational Achievement (IEA). (1996). TIMSS science items: Released set for population 2 (seventh and eighth grades). Retrieved April 1, 2004 from timss.bc.edu/timss1995i/TIMSSPDF/BSItems.pdf
18. Kauertz, A. & Fischer, H. E. (2006). Assessing students' level of knowledge and analysing the reasons for learning difficulties in physics by Rasch analysis. In X. Liu & W. J. Boone (Eds.), Applications of Rasch measurement in science education (pp. 212-246). Maple Grove, MN: JAM press.
19. Kelley, T. L. (1939). The selection of upper and lower groups for the validation of test items. Journal of Educational Psychology, 30, 17-24.
20. Kennedy, M. M. (1999). Approximations to indicators of student outcomes. Educational Evaluation and Policy Analysis, 21(4), 345-363.
21. Lane, S. (2004). Validity of high-stakes assessment: Are students engaged in complex thinking? Educational Measurement: Issues and Practice, 23(3), 6-14.
22. Lee, H.-S., Linn, M. C., Varma, K., & Liu, O. L. (2010). How does technology-enhanced inquiry instruction with visualizations impact classroom learning? Journal of Research in Science Teaching, 47(1), 71-90.
23. Linder, C. J. (1993). A challenge to conceptual change. Science Education, 77(3), 293-300.
24. Linn, M. C. (2006). The knowledge integration perspective on learning and instruction. In R. K. Sawyer (Ed.), The Cambridge handbook of the learning sciences (pp. 243-264). New York: Cambridge University Press.
25. Linn, M. C., Davis, E. A., & Bell, P. (Eds.). (2004). Internet environments for science education. Mahwah, NJ: Lawrence Erlbaum Associates.
26. Linn, M. C., & Hsi, S. (2000). Computers, teachers, peers: Science learning partners. Mahwah, NJ: Lawrence Erlbaum Associates.
27. Linn, M. C., Lee, H. S., Tinker, R., Husic, F., & Chiu, J. L. (2006). Teaching and assessing knowledge integration in science. Science, 313, 1049-1050.
28. Liu, O. L., Lee, H. S., Hofstetter, C., & Linn, M. C. (2008). Assessing knowledge integration in science: Construct, measures, and evidence. Educational Assessment, 13(1), 1-23.
29. Masters, G. N. (1982). A Rasch model for partial credit scoring. Psychometrika, 47, 149-173.
30. McCarthy, C. B. (2005). Effects of thematic-based, hands-on science teaching versus a textbook approach for students with disabilities. Journal of Research in Science Teaching, 42(3), 245-263.
31. Messick, S. (1989). Validity. In R. L. Linn (Ed.), Educational measurement (3rd ed.) (pp. 13-103). New York: Macmillan.
32. Millar, R., & Driver, R. (1987). Beyond processes. Studies in Science Education, 14, 33-62.
33. National Assessment Governing Board. (2004).Science framework for the 2005 National Assessment of Educational Progress. Washington, DC: U.S. Department of Education & Office of Educational Research and Improvement.
34. National Research Council. (1996). The national science education standards. Washington, DC: National Academy Press.
35. Nichols, P., & Sugrue, B. (1999). The lack of fidelity between cognitively complex constructs and conventional test development practice. Educational Measurement: Issues and Practice, 18(2), 18-29.
36. Pellegrino, J. W., Chudowsky, N., & Glaser, R. (Eds.). (2001). Knowing what students know: The science and design of educational assessment. Washington, DC: National Academy Press.
37. Phelps, R. P. (1998). The demand for standardized student testing. Educational Measurement: Issues and Practice, 17(3), 5-23.
38. Rodriguez, M. C. (1993). Choosing an item format. In R. E. Bennett & W. C. Ward (Eds.), Construction versus choice in cognitive measurement (pp. 213-231). Hillsdale, NJ: Lawrence Erlbaum Associates.
39. Rodriguez, M. C. (2003). Construct equivalence of multiple-choice and constructed-response items: A random effects synthesis of correlations. Journal of Educational Measurement, 40(2), 163-184.
40. Ruiz-Primo, M. A., & Shavelson, R. J. (1996). Problems and issues in the use of concept maps in science assessment. Journal of Research in Science Teaching, 33(6), 569-600.
41. Sadler, P. M. (1998). Psychometric models of student conceptions in science: Reconciling qualitative studies and distractor-driven assessment instruments. Journal of Research in Science Teaching, 35(3), 265-296.
42. Sandoval, W. A. (2003). Conceptual and epistemic aspects of students' scientific explanations. The Journal of the Learning Sciences, 12(1), 5-51.
43. Schmidt, W. H., Raizen, S. A., Britton, E. D., Bianchi, L. J., & Wolfe, R. G. (1997). Many visions, many aims: A cross-national investigation of curricular intentions in school science. Boston: Kluwer Academic Publishers.
44. Shepard, L. A. (2000). The role of assessment in a learning culture. Educational Researcher, 29(7), 4-14.
45. Shymansky, J. A., Kyle, J. W. C., & Alport, J. M. (1983). The effects of new science curricula on student performance. Journal of Research in Science Teaching, 20, 387-404.
46. Slavin, R. E. (2008). Perspectives on evidence-based research in education-what works? Issues in synthesizing educational program evaluations. Educational Researcher, 37(1), 5-14.
47. Taylor, A. B., West, S. G., & Aiken, L. S. (2006). Loss of power in logistic, ordinal logistic, and probit regression when an outcome variable is coarsely categorized. Educational Psychological Measurement, 66(2), 228-239.
48. Thornton, R., & Sokoloff, D. (1998). Assessing student learning of Newton's laws: The force and motion conceptual evaluation and the evaluation of active learning laboratory and lecture curricula. American Journal of Physics, 66(4), 338-352.
49. Wainer, H., & Thissen, D. (1993). Combining multiple-choice and constructed-response test scores: Toward a Marxist theory of test construction. Applied Measurement in Education, 6(2), 103-118.
50. Wilson, M. (2005). Constructing measures: An item response modeling approach. Mahwah, NJ: Lawrence Erlbaum Associates.
51. Wilson, M., & Wang, W.-C. (1995). Complex composites: Issues that arise in combining different modes of assessment. Applied Psychological Measurement, 19(1), 51-71.
52. Wright, B. D., & Masters, G. N. (1982). Rating scale analysis. Chicago: Mesa Press.
53. Wu, M., Adams, R., & Wilson, M. (1997). ConQuest: Generalized item response modeling software. Melbourne: ACER Press.