Cognitive foundations for science assessment design: Knowing what students know about evolution

Journal of Research in Science Teaching

Volumn 49, Issue 6, 2012, Pages 744-777

  Opfer, John E ^a   Nehm, Ross H ^a   Ha, Minsu ^a  

^a OHIO STATE UNIVERSITY   (United States)

Author keywords

assessment; cognition; constructed response; explanations; misconceptions; natural selection

Indexed keywords

ASSESSMENT; COGNITION; CONSTRUCTED RESPONSE; EXPLANATIONS; MISCONCEPTIONS; NATURAL SELECTION;

COGNITIVE SYSTEMS; DESIGN; PLANTS (BOTANY);

STUDENTS;

EID: 84863823365     PISSN: 00224308     EISSN: 10982736     Source Type: Journal    
DOI: 10.1002/tea.21028     Document Type: Article

References (109)

1. Ahn, W., Gelman, S. A., Amsterlaw, J. A., Hohenstein, J., &, Kalish, C. W., (2000). Causal status effect in children's categorization. Cognition, 76, 35-43.
2. Alonzo, A. C., &, Gotwals, A. W., (2012). Learning progressions in science: Current challenges and future directions. Rotterdam, The Netherlands: Sense Publishers.
3. Alonzo, A. C., &, Steedle, J. T., (2009). Developing and assessing a force and motion learning progression. Science Education, 93, 389-421.
4. American Association for the Advancement of Science. (1993). Benchmarks for science literacy. New York: Oxford University Press.
5. Anderson, D. L., Fisher, K. M., &, Norman, G. J., (2002). Development and evaluation of the conceptual inventory of natural science. Journal of Research in Science Teaching, 39, 952-978.
6. Anderson, J. R., Reder, L. M., &, Simon, H. A., (2000). Applications and misapplications of cognitive psychology to mathematics education. Texas Education Review, 1, 29-49.
7. Anglin, J. M., (1977). Word, object, and conceptual development. New York: W.W. Norton & Company Inc.
8. Au, T. K., Chan, C. K. K., Chan, T., Cheung, M. W. L., Ho, J., &, Ip, G. W. M., (2008). Folkbiology meets microbiology: A study of conceptual and behavioral change. Cognitive Psychology, 57, 1-19.
9. Au, T. K., Romo, L. F., &, DeWitt, J. E., (1999). Considering children's folkbiology in health education. In, M. Siegal, &, C. C. Peterson, (Eds.), Children's understanding of biology and health (pp. 209-234). Cambridge, MA: MIT Press.
10. Barnett, S. M., &, Ceci, S. J., (2002). When and where do we apply what we learn? A taxonomy for far transfer. Psychological Bulletin, 128, 612-637.
11. Bartsch, K., &, Wellman, H. M., (1995). Children talk about the mind. New York: Oxford University Press.
12. Beggrow, E. P., &, Nehm, R. H., (2012). Exploring the role of non-adaptive reasoning in students' evolutionary explanations. Proceedings of the National Association for Research in Science Teaching (NARST) annual conference, Indianapolis, IN.
13. Berland, L. K., &, McNeill, K. L., (2010). A learning progression for scientific argumentation: Understanding student work and designing supportive instructional contexts. Science Education, 94 (5), 765-793.
14. Bishop, B., &, Anderson, C., (1990). Student conceptions of natural selection and its role in evolution. Journal of Research in Science Teaching, 27, 415-427.
15. Booth, J. L., &, Siegler, R. S., (2006). Developmental and individual differences in pure numerical estimation. Developmental Psychology, 41, 189-201. (Pubitemid 43172338)
16. Bower, G. H., Karlin, M. B., &, Dueck, A., (1975). Comprehension and memory for pictures. Memory & Cognition, 3 (2), 216-220.
17. Brainerd, C. J., &, Gordon, L., (1994). Development of verbatim and gist memory for numbers. Developmental Psychology, 30 (2), 163-177.
18. Bransford, J. D., &, Johnson, M. K., (1972). Contextual prerequisites for understanding: Some investigations of comprehension and recall1. Journal of Verbal Learning and Verbal Behavior, 11 (6), 717-726.
19. Briggs, D., Alonzo, A., Schwab, C., &, Wilson, M., (2006). Diagnostic assessment with ordered multiple-choice items. Educational Assessment, 11, 33-64.
20. Brumby, M. N., (1984). Misconceptions about the concept of natural selection by medical biology students. Science Education, 68, 493-503.
21. Bulloch, M. J., &, Opfer, J. E., (2009). What makes relational reasoning smart? Revisiting the relational shift in cognitive development. Developmental Science, 12, 114-122.
22. Callanan, M. A., &, Oakes, L. M., (1992). Preschoolers' questions and parents' explanations: Causal thinking in everyday activity. Cognitive Development, 7, 213-233.
23. Carey, S., (1985). Conceptual change in childhood. Cambridge, MA: MIT Press.
24. Case, R., &, Okamoto, Y., (1996). The role of central conceptual structures in the development of children's thought. Monographs of the Society for Research in Child Development, 61, 1-2.
25. Chi, M. T. H., (2006). Two approaches to the study of experts' characteristics. In, K. A. Ericsson, N. Charness, P. J. Feltovich, &, R. R. Hoffman, (Eds.), The Cambridge handbook of expertise and expert performance (pp. 21-30). New York: Cambridge University Press.
26. Chi, M. T. H., Feltovich, P. J., &, Glaser, R., (1981). Categorization and representation of physics problems by experts and novices. Cognitive Science, 5, 121-152.
27. Clough, E. E., &, Driver, R., (1986). A study of consistency in the use of students' conceptual frameworks across different task contexts. Science Education, 70, 473-496.
28. Clough, E. E., &, Wood-Robinson, C., (1985). How secondary students interpret instances of biological adaptation. Journal of Biological Education, 19, 125-130.
29. Cozby, P. C., (1997). Methods in behavioral research (6th ed.) Mountain View, CA: Mayfield Publishing Company.
30. Craik, F. I. M., &, Lockhart, R. S., (1972). Levels of processing. A framework for memory research. Journal of Verbal Learning and Verbal Behaviour, 11, 671-684.
31. Dagher, Z. R., &, BouJaoude, S., (1997). Scientific views and religious beliefs of college students: The case of biological evolution. Journal of Research in Science Teaching, 34, 429-445. (Pubitemid 127458958)
32. Dawkins, R., (1996). The blind watchmaker: Why the evidence of evolution reveals a universe without design. New York: W.W. Norton & Company.
33. diSessa, A. A., (1993). Toward an epistemology of physics. Cognition and Instruction, 10 (2-3), 105-225.
34. Dobzhansky, T., (1973). Nothing in biology makes sense except in the light of evolution. The American Biology Teacher, 35, 125-129.
35. Endler, J. A., (1992). Natural selection: Current uses. In, E. F. Keller, &, E. A. Lloyd, (Eds.), Keywords in evolutionary biology (pp. 220-224). Cambridge, MA: Harvard University Press.
36. Ericsson, K. A., &, Smith, J., (1991). Toward a general theory of expertise: Prospects and limits. Cambridge, MA: Cambridge University Press.
37. Evans, E. M., (2001). Cognitive and contextual factors in the emergence of diverse belief systems: Creation versus evolution. Cognitive Psychology, 42, 217-266. (Pubitemid 33651068)
38. Evans, E. M., Spiegel, A. N., Gram, W., Frazier, B. N., Tare, M., &, Thompson, S., (2010). A conceptual guide to natural history museum visitors' understanding of evolution. Journal of Research in Science Teaching, 47, 326-353.
39. Fenker, D. B., Waldmann, M. R., &, Holyoak, K. J., (2005). Accessing causal relations in semantic memory. Memory & Cognition, 33, 1036-1046. (Pubitemid 46196665)
40. Frisbie, D. A., (1992). The multiple true-false item format: A status review. Educational Measurement: Issues and Practice, 11 (4), 21-26.
41. Gelman, S. A., (2003). The essential child: Origins of essentialism in everyday thought. Oxford, UK: Oxford University Press.
42. Gelman, S. A., (2004). Psychological essentialism in children. Trends in Cognitive Sciences, 8, 404-409. (Pubitemid 39179903)
43. Gentner, D., (1988). Metaphor as structure mapping: The relational shift. Child Development, 59, 47-59.
44. Gentner, D., &, Colhoun, J., (2010). Analogical processes in human thinking and learning. In, A. von Müller, &, E. Pöppel, (Series Eds.), B. Glatzeder, V. Goel, &, A. von Müller, (Vol. Eds.), On thinking: Vol. 2 towards a theory of thinking. Berlin, Heidelberg, New York: Springer-Verlag.
45. Gentner, D., &, Markman, A. B., (1997). Structure mapping in analogy and similarity. American Psychologist, 52, 45-56. (Pubitemid 127458598)
46. Gick, M. L., &, Holyoak, K. J., (1983). Schema induction and analogical transfer. Cognitive Psychology, 15, 1-38.
47. Goldberg, R. F., &, Thompson-Schill, S. L., (2009). Developmental "roots" in mature biological knowledge. Psychological Science, 20 (4), 480-487.
48. Gotwals, A. W., &, Songer, N. B., (2010). Reasoning up and down a food chain: Using an assessment framework to investigate students' middle knowledge. Science Education, 94, 259-281.
49. Griffin, S. A., Case, R., &, Siegler, R. S., (1994). Rightstart: Providing the central conceptual prerequisites for first formal learning of arithmetic to students at risk for school failure. In, K. McGilly, (Ed.), Classroom lessons: Integrating cognitive theory and classroom practice (pp. 25-49). Cambridge, MA: MIT Press.
50. Griffiths, T. L., Steyvers, M., &, Firl, A., (2007). Google and the mind: Predicting fluency with PageRank. Psychological Science, 18, 1069-1076.
51. Ha, M., Nehm, R. H., Urban-Lurain, M., &, Merrill, J. E., (2011). Applying computerized scoring models of written biological explanations across courses and colleges: Prospects and limitations. CBE-Life Science Education, 10, 379-393.
52. Hartshorn, K., Rovee-Collier, C., Gerhardstein, P., Bhatt, R. S., Klein, P. J., &, Aaron, F., (1998). Developmental changes in the specificity of memory over the first year of life. Developmental Psychobiology, 33, 61-78. (Pubitemid 28455940)
53. Hunt, E., &, Minstrell, J., (1994). A cognitive approach to the teaching of physics. In, K. McGilly, (Ed.), Classroom lessons: Integrating cognitive theory and classroom practice (pp. 51-74). Cambridge, MA: MIT Press.
54. Inagaki, K., &, Hatano, G., (2002). Young children's naive thinking about the biological world. New York: Psychology Press.
55. Inagaki, K., &, Hatano, G., (2004). Vitalistic causality in young children's naive biology. Trends in Cognitive Sciences, 8, 356-362. (Pubitemid 39099461)
56. Kampourakis, K., &, Zogza, V., (2009). Preliminary evolutionary explanations: A basic framework for conceptual change and explanatory coherence in evolution. Science & Education, 18, 1313-1340.
57. Kane, M. T., (2001). Current concerns in validity theory. Journal of Educational Measurement, 38, 319-342. (Pubitemid 33591226)
58. Katona, G., (1940). Organizing and memorizing: Studies in the psychology of learning and teaching. New York: Columbia University Press.
59. Keil, F. C., (2006). Explanation and understanding. Annual Review of Psychology, 57, 227-254. (Pubitemid 43237233)
60. Keil, F. C., &, Batterman, N., (1984). A characteristic-to-defining shift in the development of word meaning. Journal of Verbal Learning and Verbal Behavior, 23, 221-236.
61. Kelemen, D., (2003). British and American children's preferences for teleo-functional explanations of the natural world. Cognition, 88, 201-221. (Pubitemid 36579314)
62. Kelemen, D., &, Rosset, E., (2009). The human function compunction: Teleological explanation in adults. Cognition, 111, 138-143.
63. Kirsh, D., (2009). Problem solving and situated cognition. In, P. Robbins, &, M. Aydede, (Eds.), The Cambridge handbook of situated cognition (pp. 264-306). Cambridge, MA: Cambridge University Press.
64. Krascum, R. M., &, Andrews, S., (1998). The effects of theories on children's acquisition of family-resemblance categories. Child Development, 69 (2), 333-346. (Pubitemid 128453712)
65. Legare, C. H., Evans, E. M., Rosengren, K. S., &, Harris, P. L., (2012). The coexistence of natural and supernatural explanations across cultures and development. Child Development, 83, 779-793.
66. Lewontin, R. C., (1970). The units of selection. Annual Review of Ecology and Systematics, 1 (1), 1-18.
67. Lewontin, R. C., (1978). Adaptation. Scientific American, 239, 212-228.
68. Linn, R. L., Baker, E. L., &, Dunbar, S. B., (1991). Complex, performance-based assessment: Expectations and validation criteria. Educational Researcher, 20, 15-21.
69. Lombrozo, T., (2006). The structure and function of explanations. Trends in Cognitive Sciences, 10 (10), 464-470. (Pubitemid 44415570)
70. Lombrozo, T., (2012). Explanation and abductive inference. In, K. J. Holyoak, &, R. G. Morrison, (Eds.), Oxford handbook of thinking and reasoning (pp. 260-276), Oxford, UK: Oxford University Press.
71. Lombrozo, T., &, Carey, S., (2006). Functional explanation and the function of explanation. Cognition, 99, 167-204.
72. Marion, S. F., &, Pellegrino, J. W., (2006). A validity framework for evaluating the technical quality of alternate assessment. Educational Measurement: Issues and Practice, 25, 47-57.
73. Marshall, S. P., (1995). Schemas in problem-solving. New York: Cambridge University Press.
74. McCloskey, M., (1983). Intuitive physics. Scientific American, 248, 114-122.
75. National Research Council. (2007). Taking science to school: Learning and teaching science in grades K-8. Washington, DC: National Academy Press.
76. National Research Council. (1996). National science education standards. Washington, DC: National Academy Press.
77. National Research Council. (2001). Knowing what students know: The science and design of educational assessment. Washington, DC: National Academy Press.
78. Nehm, R. H., (2006). Faith-based evolution education ? BioScience, 56 (8), 638-639.
79. Nehm, R. H., Beggrow, E. P., Opfer, J. E., &, Ha, M., (2012). Reasoning about natural selection: Diagnosing contextual competency using the ACORNS instrument. The American Biology Teacher, 74 (2), 92-98.
80. Nehm, R. H., &, Ha, M., (2011). Item feature effects in evolution assessment. Journal of Research in Science Teaching, 48, 237-256.
81. Nehm, R. H., Ha, M., &, Mayfield, E., (2012). Transforming biology assessment with machine learning: Automated scoring of written evolutionary explanations. Journal of Science Education and Technology, 21 (1), 183-196.
82. Nehm, R. H., Ha, M., Rector, M., Opfer, J. E., Perrin, L., Ridgway, J., &, Mollohan, K., (2010). Scoring guide for the open response instrument (ORI) and evolutionary gain and loss test (ACORNS). Technical Report of National Science Foundation REESE Project 0909999.
83. Nehm, R. H., &, Reilly, L., (2007). Biology majors' knowledge and misconceptions of natural selection. BioScience, 57 (3), 263-272. (Pubitemid 351837761)
84. Nehm, R. H., &, Ridgway, J., (2011). What do experts and novices "see" in evolutionary problems ? Evolution: Education and Outreach, 4 (4), 666-679.
85. Nehm, R. H., &, Schonfeld, I. S., (2008). Measuring knowledge of natural selection: A comparison of the CINS, an open-response instrument, and an oral interview. Journal of Research in Science Teaching, 45 (10), 1131-1160.
86. Nehm, R. H., &, Schonfeld, I. S., (2010). The future of natural selection knowledge measurement: A reply to Anderson et al. Journal of Research in Science Teaching, 47 (3), 358-362.
87. Newell, A., &, Simon, H. A., (1972). Human problem solving. Englewood Cliffs, NJ: Prentice-Hall.
88. Opfer, J. E., (2002). Identifying living and sentient kinds from dynamic information: The case of goal-directed versus aimless autonomous movement in conceptual change. Cognition, 86, 97-122. (Pubitemid 35346379)
89. Opfer, J. E., &, Bulloch, M. J., (2007). Causal relations drive young children's induction, naming, and categorization. Cognition, 105, 206-217. (Pubitemid 47223170)
90. Opfer, J. E., &, Gelman, S. A., (2001). Children's and adults' models for predicting teleological action: The development of a biology-based model. Child Development, 72, 1367-1381. (Pubitemid 33596531)
91. Opfer, J. E., &, Gelman, S. A., (2010). Development of the animate-inanimate distinction. In, U. Goswami, (Ed.), The Wiley-Blackwell handbook of childhood cognitive development. Cambridge, UK: Blackwell.
92. Opfer, J. E., &, Siegler, R. S., (2004). Revisiting preschoolers' living things concept: A microgenetic analysis of conceptual change in basic biology. Cognitive Psychology, 49, 301-332. (Pubitemid 39196027)
93. Opfer, J. E., &, Siegler, R. S., (2012). Development of quantitative thinking. In, K. Holyoak, &, R. Morrison, (Eds.), Oxford handbook of thinking and reasoning. Cambridge, UK: Oxford University Press.
94. Page, L., Brin, S., Motwani, R., &, Winograd, T., (1998). The PageRank citation ranking: Bringing order to the web (Tech. Rep.). Stanford, CA: Stanford Digital Library Technologies Project.
95. Patterson, C., (1978). Evolution. Ithaca: Cornell University Press.
96. Perkins, D. N., &, Grotzer, T. A., (2005). Dimensions of causal understanding: The role of complex causal models in students understanding of science. Studies in Science Education, 41 (1), 117-165.
97. Pigliucci, M., &, Kaplan, J., (2006). Making sense of evolution: The conceptual foundations of evolutionary biology. Chicago, IL: University of Chicago Press.
98. Poling, D. A., &, Evans, E. M., (2002). Why do birds of a feather flock together? Developmental change in the use of multiple explanations: Intention, teleology and essentialism. British Journal of Developmental Psychology, 20, 89-112. (Pubitemid 38845588)
99. Prasada, S., &, Dillingham, E. M., (2006). Principled and statistical connections in common sense conception. Cognition, 99, 73-112. (Pubitemid 43145922)
100. Russ, R. S., Scherr, R. E., Hammer, D., &, Mikeska, J., (2008). Recognizing mechanistic reasoning in student scientific inquiry: A framework for discourse analysis developed from philosophy of science. Science Education, 92 (3), 499-525.
101. Sabella, M. S., &, Redish, E. F., (2007). Knowledge organization and activation in physics problem solving. American Journal of Physics, 75, 1017-1029.
102. Shtulman, A., (2006). Qualitative differences between naïve and scientific theories of evolution. Cognitive Psychology, 52, 170-194. (Pubitemid 43160193)
103. Shtulman, A., &, Schulz, L., (2008). The relation between essentialist beliefs and evolutionary reasoning. Cognitive Science, 32 (6), 1049-1062.
104. Sinatra, G. M., Brem, S. K., &, Evans, E. M., (2008). Changing minds? Implications of conceptual change for teaching and learning about biological evolution. Evolution: Education Outreach, 1, 189-195.
105. Sinatra, G. M., Southerland, S. A., McConaughy, F., &, Demastes, J. W., (2003). Intentions and beliefs in students' understanding and acceptance of biological evolution. Journal of Research in Science Teaching, 40, 510-528.
106. Sloutsky, V. M., &, Fisher, A. V., (2004). When learning and development decrease memory: Evidence against category-based induction. Psychological Science, 15, 553-558. (Pubitemid 39080420)
107. Vosniadou, S., &, Brewer, W. F., (1992). Mental models of the earth: A study of conceptual change in childhood. Cognitive Psychology, 24, 535-585.
108. White, B. Y., &, Frederiksen, J. R., (1998). Inquiry, modeling, and metacognition: Making science accessible to all students. Cognition & Instruction, 16, 3-118. (Pubitemid 128343737)
109. Woods, N. N., Brooks, L. R., &, Norman, G. R., (2007). It all makes sense: Biomedical knowledge, causal connections and memory in the novice diagnostician. Advances in Health Sciences Education, 12, 405-415.