Designing automated guidance for concept diagrams in inquiry instruction

Journal of Research in Science Teaching

Volumn 53, Issue 7, 2016, Pages 1003-1035

  Ryoo, Kihyun ^a   Linn, Marcia C ^b  

^a UNIVERSITY OF NORTH CAROLINA   (United States)

^b UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

adaptive guidance; automated guidance; concept diagrams; inquiry learning; knowledge integration; science education; technology education

Indexed keywords

AUTOMATION; DATA VISUALIZATION; INFORMATION USE; VISUALIZATION;

ADAPTIVE GUIDANCE; AUTOMATED GUIDANCE; CONCEPT DIAGRAMS; INQUIRY LEARNING; KNOWLEDGE INTEGRATION; SCIENCE EDUCATION; TECHNOLOGY EDUCATION;

STUDENTS;

EID: 84963556526     PISSN: 00224308     EISSN: 10982736     Source Type: Journal    
DOI: 10.1002/tea.21321     Document Type: Article

References (93)

1. Anderson, J. R., Corbett, A. T., Koedinger, K. R., & Pelletier, R. (1995). Cognitive Tutors: Lessons learned. The Journal of the Learning Sciences, 4(2), 167–207.
2. Andre, T., & Thieman, A. (1988). Level of adjunct question, type of feedback, and learning concepts by reading. Contemporary Educational Psychology, 13(3), 296–307.
3. Attali, Y., & Powers, D. (2010). Immediate feedback and opportunity to revise answers to open-ended questions. Educational and Psychological Measurement, 70(1), 22–35.
4. Azevedo, R., & Bernard, R. M. (1995). A meta-analysis of the effects of feedback in computer-based instruction. Journal of Educational Computing Research, 13(2), 111–127.
5. Bell, B., & Cowie, B. (2001). The characteristics of formative assessment in science education. Science Education, 85(5), 536–553.
6. Black, P., & William, D. (1998). Assessment and classroom learning. Assessment in Education, 5(1), 7–74.
7. Boramy, S. (2010). Using directive and facilitative feedback to improve student writing: A case study of a higher education setting in Cambodia. Language Education in Asia, 1(1), 23–47.
8. Brown, A. L., & Campione, J. C. (1994). Guided discovery in a community of learners. In K. McGilly (Ed.), Classroom lessons: Integrating cognitive theory and classroom practice (pp. 229–270). Cambridge, MA: MIT Press.
9. Burstein, J., Chodorow, M., & Leacock, C. (2003). Criterion online essay evaluation: An application for automated evaluation of student essays. Proceedings of the Fifteenth Annual Conference on Innovative Applications of Artificial Intelligence, Acapulco, Mexico. Available online at: http://www.ets.org/Media/Research/pdf/erater_iaai03_burstein.pdf
10. Butler, A. C., Karpicke, J. D., & Roediger, H. L, III (2007). The effect of type and timing of feedback on learning from multiple-choice tests. Journal of Experimental Psychology: Applied, 13(4), 273.
11. Chase, J. A., & Houmanfar, R. (2009). The differential effects of elaborate feedback. Journal of Behavioral Education, 18(3), 245–265.
12. Chen, C. F. E., & Cheng, W. Y. E. (2008). Beyond the design of automated writing evaluation: Pedagogical practices and perceived learning effectiveness in EFL writing classes. Language Learning & Technology, 12(2), 94–112.
13. Chi, M., Siler, S., Jeong, H., Yamauchi, T., & Hausmann, R. (2001). Learning from human tutoring. Cognitive Science, 25(4), 471–533.
14. Chinn, C. A., & Malhotra, B. A. (2002). Epistemologically authentic inquiry in schools: A theoretical framework for evaluating inquiry tasks. Science Education, 86(2), 175–218.
15. Clariana, R. B., Ross, S. M., & Morrison, G. R. (1991). The effects of different feedback strategies using computer-administered multiple-choice questions as instruction. Educational Technology Research and Development, 39(2), 5–17.
16. Conati, C., & VanLehn, K. (2000). Toward computer-based support of meta-cognitive skills: A computational framework to coach self-explanation. International Journal of Artificial, 11, 389–415.
17. Conlon, T. (2004). But is our concept map any good?: Classroom experiences with the reasonable fallible analyser. In Concept maps: Theory, methodology, technology. Proceedings of the first international conference on concept mapping. Pamplona, Spain: Universidad Pública de Navarra. Intelligence in Education, 11, 389–415.
18. Corbalan, G., Paas, F., & Cuypers, H. (2010). Computer-based feedback in linear algebra: Effects on transfer performance and motivation. Computers & Education, 55(2), 692–703.
19. Corbett, A. (2001). Cognitive computer tutors: Solving the two-sigma problem. In M. Bauer, P. J. Gmytrasiewicz, & J. Vassileva (Eds.), User modeling: Proceedings of the Eighth International Conference, UM 2001 (pp. 137–147). Berlin, Germany: Springer-Verlag BerlinHeidelberg.
20. Corbett, A., & Trask, H. (2000, April). Instructional interventions in computer-based tutoring: Differential impact on learning time and accuracy. In Proceedings of the SIGCHI conference on human factors in computing systems (pp. 97–104). ACM.
21. Dam, G., & Kaufmann, S. (2008). Computer assessment of interview data using latent semantic analysis. Behavior Research Methods, 40(1), 8–20.
22. Davis, E. A. (2003). Prompting middle school science students for productive reflection: Generic and directed prompts. The Journal of the Learning Sciences, 12(1), 91–142.
23. Debuse, J. C., & Lawley, M. (2016). Benefits and drawbacks of computer-based assessment and feedback systems: Student and educator perspectives. British Journal of Educational Technology, 47(2), 294–301.
24. diSessa, A. A. (2006). A history of conceptual change research: Threads and fault lines. In K. Sawyer (Ed.), Cambridge handbook of the learning sciences (pp. 265–281). Cambridge, UK: Cambridge University Press.
25. Economides, A. A. (2009). Conative feedback in computer-based assessment. Computers in the Schools, 26(3), 207–223.
26. Edelson, D. C. (2001). Learning-for-use: A framework for the design of technology-supported inquiry activities. Journal of Research in Science Teaching, 38(3), 355–385.
27. Furtak, E. M., Seidel, T., Iverson, H., & Briggs, D. C. (2012). Experimental and quasi-experimental studies of inquiry-based science teaching: A meta-analysis. Review of Educational Research, 82(3), 300–329.
28. Gerard, L. F., Ryoo, K., McElhaney, K. W., Liu, O. L., Rafferty, A. N., & Linn, M. C. (2016). Automated guidance for student inquiry. Journal of Educational Psychology, 108(1), 60–81.
29. Gouli, E., Gogoulou, A., & Grigoriadou, M. (2009). Working with COMPASS: Analyzing and evaluating students’ concept maps’. In Proceedings of IADIS International Conference on Cognition and Exploratory Learning in Digital Age, Italy (pp. 85–92).
30. Graesser, A., & McNamara, D. (2010). Self-regulated learning in learning environments with pedagogical agents that interact in natural language. Educational Psychologist, 45(4), 234–244.
31. Graesser, A. C., VanLehn, K., Rose, C., Jordan, P., & Harter, D. (2001). Intelligent tutoring systems with conversational dialogue. AI Magazine, 22, 39–51.
32. Hagemans, M. G., van der Meij, H., & de Jong, T. (2013). The effects of a concept map-based support tool on simulation-based inquiry learning. Journal of Educational Psychology, 105(1), 1–24.
33. Hmelo-Silver, C., Duncan, R., & Chinn, C. (2007). Scaffolding and achievement in problem-based and inquiry learning: A response to Kirschner, Sweller, and Clark. Educational Psychologist, 42(2), 99–107.
34. Huang, Y.-M., Lin, Y.-T., & Cheng, S.-C. (2009). An adaptive testing system for supporting versatile educational assessment. Computers & Education, 52(1), 53–67.
35. Jin, H., & Anderson, C. W. (2012). A learning progression for energy in socio-ecological systems. Journal of Research in Science Teaching, 49(9), 1149–1180.
36. Jochems, W., Van Merriënboer, J. J. G., & Koper, R. (Eds.). (2004). Integrated E-learning: Implications for pedagogy, technology, and organization. London: RoutledgeFalmer.
37. Kegel, C. A. T., & Bus, A. G. (2012). Links between DRD4, executive attention, and alphabetic skills in a nonclinical sample. Journal of Child Psychology and Psychiatry, 54(3), 305–312.
38. Kesidou, S., & Roseman, J. E. (2002). How well do middle school science programs measure up? Findings from Project 2061's curriculum review. Journal of Research in Science Teaching, 39(6), 522–549.
39. Kluger, A. N., & DeNisi, A. (1996). The effects of feedback interventions on performance: A historical review, a meta-analysis and a preliminary feedback interventional theory. Psychological Bulletin, 119(2), 254–284.
40. Koedinger, K. R., & Aleven, V. (2007). Exploring the assistance dilemma in experiments with Cognitive Tutors. Educational Psychology Review, 19(3), 239–264.
41. Koedinger, K. R., McLaughlin, E. A., & Heffernan, N. T. (2010). A quasi-experimental evaluation of an on-line formative assessment and tutoring system. Journal of Educational Computing Research, 43(4), 489–510.
42. Krause, U.-M., Stark, R., & Mandl, H. (2009). The effects of cooperative learning and feedback on e-learning in statistics. Learning and Instruction, 19(2), 158–170.
43. Lee, H.-S., & Liu, O. L. (2009). Assessing learning progression of energy concepts across middle school grades: The knowledge integration perspective. Science Education, 94(4), 665–688.
44. Lin, S. S. J., Liu, E. Z., & Yuan, S.-M. (2001). Learning effects of web-based peer assessment for students with various thinking styles. Journal of Computer Assisted Learning, 17(4), 420–432.
45. Linn, M. C., & Eylon, B.-S. (2011). Science learning and instruction: Taking advantage of technology to promote knowledge integration. New York: Routledge.
46. Linn, M. C., Gerard, L., Ryoo, K., McElhaney, K., Liu, O. L., & Rafferty, A. N. (2014). Computer-guided inquiry to improve science learning. Science, 344(6180), 155–156.
47. Litman, D., Rose, C., Forbes-Riley, K., VanLehn, K., Bhembe, D., & Silliman, S. (2004). Spoken Versus Typed Human and Computer Dialogue Tutoring. In Proceedings of Intelligent Tutoring Systems.
48. Liu, O. L., Lee, H., Hofstetter, C., & Linn, M. C. (2008). Assessing knowledge integration in science: Construct, measures, and evidence. Educational Assessment, 13, 33–55.
49. Liu, O. L., Lee, H. S., & Linn, M. C. (2011). Measuring knowledge integration: Validation of four-year assessments. Journal of Research in Science Teaching, 48, 1079–1107.
50. Liu, O. L., Rios, J. A., Heilman, M., Gerard, L., & Linn, M. C. (2016). Validation of automated scoring of science assessments. Journal of Research in Science Teaching, 53(2), 215–233.
51. Liu, O. L., Ryoo, K., Sato, E., Svihla, V., & Linn, M. C. (2015). Measuring knowledge integration learning of energy topics: A two-year longitudinal study. International Journal of Science Education, 37(7), 1–23.
52. Luckie, D. B., Harrison, S. H., & Ebert-May, D. (2004, September). Introduction to C-Tools: Concept mapping tools for online learning. In Proceedings of the 1st International Conference on Concept Mapping (pp. 261–264).
53. Lukasenko, R., Anohina-Naumeca, A., Vilkelis, M., & Grundspenkis, J. (2010). Feedback in the concept map based intelligent knowledge assessment system. Scientific Journal of Riga Technical University, 43, 17–26.
54. Lynch, C., Ashley, K. D., Pinkwart, N., & Aleven, V. (2009). Concepts, structures, and goals: Redefining ill-definedness. International Journal of Artificial Intelligence in Education, 19(3), 253–266.
55. Mason, B. J., & Bruning, R. (2001). Providing feedback in computer-based instruction: What the research tells us. Center for Instructional Innovation, University of Nebraska-Lincoln: 14. Retrieved June 1, 2006, from http://dwb.unl.edu/Edit/MB/MasonBruning.html
56. McLaren, B. M., DeLeeuw, K. E., & Mayer, R. E. (2011). A politeness effect in learning with web-based intelligent tutors. International Journal of Human-Computer Studies, 69(1), 70–79.
57. McNeil, K., & Krajcik, J. (2009). Synergy between teacher practices and curricular scaffolds to support students in using domain-specific and domain-general knowledge in writing arguments to explain phenomena. Journal of the Learning Sciences, 18(3), 416–460.
58. Minner, D. D., Levy, A. J., & Century, J. (2010). Inquiry-based science instruction—What is it and does it matter? Results from a research synthesis years 1984–2002. Journal of Research in Science Teaching, 47(4), 474–496.
59. Mohan, L., Chen, H., & Anderson, W. C. (2009). Developing a multi-year learning progression for carbon cycling in socio-ecological systems. Journal of Research in Science Teaching, 46(6), 675–698.
60. Moreno, R. (2004). Decreasing cognitive load for novice students: Effects of explanatory versus corrective feedback in discovery-based Mmultimedia. Instructional Science, 32(1–2), 99–113.
61. Nathan, M. J. (1998). Knowledge and situational feedback in a learning environment for algebra story problem Solving. Interactive Learning Environments, 5(1), 135–159.
62. Okita, S. Y., & Schwartz, D. L. (2013). Learning by teaching human pupils and teachable agents: The importance of recursive feedback. Journal of the Learning Sciences, 22(3), 375–412.
63. Osborne, J. (2010). Arguing to learn in science: The role of collaborative, critical discourse. Science, 328(5977), 463–466.
64. Person, N. K., Olney, A., D'Mello, S. K., & Lehman, B. (2012, May). Interactive Concept Maps and Learning Outcomes in Guru. In FLAIRS Conference.
65. Quintana, C., Reiser, B., Davis, E. A., Krajcik, J., Fretz, E., Duncan, R. G., … Soloway, E. (2004). A scaffolding design framework for software to support science inquiry. Journal of the Learning Sciences, 13(3), 337–386.
66. Raes, A, Schellens, T, & De Wever, B. (2014). Web-based collaborative inquiry to bridge gaps in secondary science education. Journal of the Learning Sciences, 23(3), 316–347.
67. Rafferty, A. N., Gerard, L., McElhaney, K., Linn, M. C. (2013). Automating guidance for students’ chemistry drawings. Proceedings of Formative Feedback in Interactive Learning Environments (AIEDWorkshop).
68. Rebolledo-Mendez, G., Luckin, R., & du Boulay, B (2011). Designing adaptive motivational scaffolding, scaffolding for a tutoring system. In R. A. Calvo, & S. K. D'Mello (Eds.), New perspectives on affect and learning technologies. (pp. 155–168). New York: Springer.
69. Rivet, A. E., & Krajcik, J. S. (2004). Contextualizing instruction in project-based science: Activating students’ prior knowledge and experiences to support learning. In Y. B. Kafai, W. A. Sandoval, N. Enyedy, A. S. Nixon, & F. Herrera (Eds.), Embracing diversity in the learning sciences: The proceedings of the Sixth International Conference of the Learning Sciences (ICLS 2004). Santa Monica, CA: Lawrence Erlbaum.
70. Roll, I., Aleven, V., McLaren, B. M., & Koedinger, K. R. (2011). Improving students’ help-seeking skills using metacognitive feedback in an intelligent tutoring system. Learning and Instruction, 21(2), 267–280.
71. Roscoe, R. D., & McNamara, D. S. (2013). Writing pal: Feasibility of an intelligent writing strategy tutor in the high school classroom. Journal of Educational Psychology, 105(4), 1010–1025.
72. Rosé, C., & VanLehn, K. (2005). An evaluation of a hybrid language understanding approach for robust selection of tutoring goals. International Journal of Artificial Intelligence in Education, 15(4), 325–355.
73. Ruiz-Primo, M. A., & Furtak, E. M. (2007). Exploring teachers’ informal formative assessment practices and students’ understanding in the context of scientific inquiry. Journal of Research in Science Teaching, 44(1), 57–84.
74. Ruiz-Primo, M. A., & Shavelson, R. J. (1996). Problem and issues in the use of concept maps in science assessment. Journal of Research in Science Teaching, 33(6), 569–600.
75. Ryoo, K., & Linn, M. C. (2012). Can dynamic visualizations improve middle school students’ understanding of energy in photosynthesis? Journal of Research in Science Teaching, 49(2), 218–243.
76. Ryoo, K., & Linn, M. C. (2014). Designing guidance for interpreting dynamic visualizations: Generating versus reading explanations. Journal of Research in Science Teaching, 51(2), 147–174.
77. Ryoo, K., & Linn, M. C. (2015). Designing and validating assessments of complex thinking in science. Theory into Practice, 54(3), 238–254.
78. Schoppek, W., & Tulis, M. (2010). Enhancing arithmetic and word problem solving skills efficiently by individualized computer-assisted practice. The Journal of Educational Research, 103(4), 239–252.
79. Segedy, J. R., Kinnebrew, J. S., & Biswas, G. (2013). The effect of contextualized conversational feedback in a complex open-ended learning environment. Educational Technology Research and Development, 61(1), 71–89.
80. Shute, V. J. (2008). Focus on formative feedback. Review of Educational Research, 78(1), 153–189.
81. Smits, M., Boon, J., Sluijsmans, D. M. A., & Van Gog, T. (2008). Content and timing of feedback in a web-based learning environment: Effects on learning as a function of prior knowledge. Interactive Learning Environments, 16(2), 183–193.
82. Songer, N. B. (1996). Exploring learning opportunities in coordinated network-enhanced classrooms: A case of kids as global scientists. The Journal of the Learning Sciences, 5(4), 297–327.
83. Stern, L, & Roseman, J. E. (2004). Can middle-school science textbooks help students learn important ideas? Journal of Research in Science Teaching, 41(6), 538–568.
84. Tan, J., Biswas, G., & Schwartz, D. (2006). Feedback for metacognitive support in learning by teaching environments. In Proceedings of the 28th Annual Meeting of the Cognitive Science Society, Vancouver, Canada (pp. 828–833).
85. Walker, E., Rummel, N., & Koedinger, K. R. (2011). In S. B. G. Biswas & A. M. J. Kay (Eds.), Using automated dialog analysis to assess peer tutoring and trigger effective support. Proceedings of the 10th International Conference on Artificial Intelligence in Education. Berlin: Springer. 385–393.
86. Wang, H. C., Chang, C. Y., & Li, T. Y. (2008). Assessing creative problem-solving with automated text grading. Computers & Education, 51(4), 1450–1466.
87. White, B. Y., & Frederiksen, J. R. (1998). Inquiry, modeling, and metacognition: Making science accessible to all students. Cognition and Instruction, 16(1), 3–118.
88. Whyte, M. M., Karolick, D. M., Neilsen, M. C., Elder, G. D., & Hawley, W. T. (1995). Cognitive styles and feedback in computer-assisted instruction. Journal of Educational Computing Research, 12(2), 195–203.
89. Wichmann, A., & Leutner, D. (2009). Inquiry learning: Multi-level support with respect to inquiry, explanations and regulation during an inquiry cycle. Zeitschrift für Pädagogische Psychologie, 23(2), 117–127.
90. Wijekumar, K. K. (2006). Implementing web-based intelligent tutoring systems in K-12 settings: A case study on approach and challenges. Journal of Educational Technology Systems, 35(2), 193–208.
91. Williams, M., DeBarger, A. H., Montgomery, B. L., Zhou, X., & Tate, E. (2012). Exploring middle school students’ conceptions of the relationship between genetic inheritance and cell division. Science Education, 96(1), 78–103.
92. Wu, P. H., Hwang, G. J., Milrad, M., Ke, H. R., & Huang, Y. M. (2012). An innovative concept map approach for improving students’ learning performance with an instant feedback mechanism. British Journal of Educational Technology, 43(2), 217–232.
93. Yin, Y., Vanides, J., Ruiz-Primo, M. A., Ayala, C. C., & Shavelson, R. J. (2005). Comparison of two concept-mapping techniques: Implications for scoring, interpretation, and use. Journal of Research in Science Teaching, 42(2), 166–184.