Guided retrieval practice of educational materials using automated scoring

Journal of Educational Psychology

Volumn 106, Issue 1, 2014, Pages 58-68

  Grimaldi, Phillip J ^a   Karpicke, Jeffrey D ^a  

^a PURDUE UNIVERSITY   (United States)

Author keywords

Computer based learning; Retrieval based learning; Testing effect

Indexed keywords

EID: 84892936800     PISSN: 00220663     EISSN: None     Source Type: Journal    
DOI: 10.1037/a0033208     Document Type: Article

References (46)

1. Agarwal, P. K., Karpicke, J. D., Kang, S. H. K., Roediger, H. L., & McDermott, K. B. (2008). Examining the testing effect with open-and closed-book tests. Applied Cognitive Psychology, 22, 861-876. doi: 10.1002/acp.1391.
2. Butler, A. C. (2010). Repeated testing produces superior transfer of learning relative to repeated studying. Journal of Experimental Psychology: Learning, Memory, and Cognition, 36, 1118-1133. doi:10.1037/a0019902.
3. Dunlosky, J., Hartwig, M., Rawson, K. A., & Lipko, A. R. (2011). Improving college students' evaluation of text learning using idea-unit standards. Quarterly Journal of Experimental Psychology, 64, 467-484. doi:10.1080/17470218.2010.502239.
4. Dunlosky, J., & Rawson, K. A. (2012). Overconfidence produces underachievement: Inaccurate self evaluations undermine students' learning and retention. Learning and Instruction, 22, 271-280. doi:10.1016/j .learninstruc.2011.08.003.
5. Fellbaum, C. (1998). WordNet: An electronic lexical database. Cambridge, MA: MIT Press.
6. Fleiss, J. L., & Paik, L. B. (2003). Statistical methods for rates and proportions. New York, NY: Wiley. doi:10.1002/0471445428.
7. Foltz, P., Laham, D., & Landauer, T. (1999). The intelligent essay assessor: Applications to educational technology. Interactive Multimedia Electronic Journal of Computer-Enhanced Learning, 1(2). Retrieved from http://imej.wfu.edu/index.asp.
8. Graesser, A. C., Penumatsa, P., Ventura, M., Cai, Z., & Hu, X. (2007). Using LSA in AutoTutor: Learning through mixed-initiative dialogue in natural language. In T. K. Landauer, D. S. McNamara, S. Dennis, & W. Kintsch (Eds.), Handbook of latent semantic analysis (pp. 243-262). Mahwah, NJ: Erlbaum.
9. Johnson, C. I., & Mayer, R. E. (2009). A testing effect with multimedia learning. Journal of Educational Psychology, 101, 621-629. doi: 10.1037/a0015183.
10. Karpicke, J. D. (2009). Metacognitive control and strategy selection: Deciding to practice retrieval during learning. Journal of Experimental Psychology: General, 138, 469-486. doi:10.1037/a0017341.
11. Karpicke, J. D. (2012). Retrieval-based learning: Active retrieval promotes meaningful learning. Current Directions in Psychological Science, 21, 157-163. doi:10.1177/0963721412443552.
12. Karpicke, J. D., & Bauernschmidt, A. (2011). Spaced retrieval: Absolute spacing enhances learning regardless of relative spacing. Journal of Experimental Psychology: Learning, Memory, and Cognition, 37, 1250-1257. doi:10.1037/a0023436.
13. Karpicke, J. D., & Blunt, J. R. (2011). Retrieval practice produces more learning than elaborative studying with concept mapping. Science, 331, 772-775. doi:10.1126/science.1199327.
14. Karpicke, J. D., Butler, A. C., & Roediger, H. L. (2009). Metacognitive strategies in student learning: Do students practice retrieval when they study on their own? Memory, 17, 471-479. doi:10.1080/09658210802647009.
15. Karpicke, J. D., & Grimaldi, P. J. (2012). Retrieval-based learning: A perspective for enhancing meaningful learning. Educational Psychology Review. Advance online publication. doi:10.1007/s10648-012-9202-2.
16. Karpicke, J. D., & Roediger, H. L. (2007). Repeated retrieval during learning is the key to long-term retention. Journal of Memory and Language, 57, 151-162. doi:10.1016/j.jml.2006.09.004.
17. Karpicke, J. D., & Roediger, H. L. (2008). The critical importance of retrieval for learning. Science, 319, 966-968. doi:10.1126/science.1152408.
18. Karpicke, J. D., & Roediger, H. L. (2010). Is expanding retrieval a superior method for learning text materials? Memory & Cognition, 38, 116-124. doi:10.3758/MC.38.1.116.
19. Karpicke, J. D., & Smith, M. A. (2012). Separate mnemonic effects of retrieval practice and elaborative encoding. Journal of Memory and Language, 67, 17-29. doi:10.1016/j.jml.2012.02.004.
20. Kintsch, E., Steinhard, D., Stahl, G., & the LSA Research Group. (2000). Developing summarization skills through the use of LSA-based feedback. Interactive Learning Environments, 8, 87-109. doi:10.1076/1049-4820(200008)8:2;1-B;FT087.
21. Kornell, N., & Bjork, R. A. (2007). The promise and perils of selfregulated study. Psychonomic Bulletin & Review, 14, 219-224. doi: 10.3758/BF03194055.
22. Kornell, N., & Son, L. K. (2009). Learners' choices and beliefs about self-testing. Memory, 17, 493-501. doi:10.1080/09658210902832915.
23. Leacock, C., & Chodorow, M. (2003). C-rater: Automated scoring of short-answer questions. Computers and the Humanities, 37, 389-405. doi:10.1023/A:1025779619903.
24. Mayer, R. E., Stull, A., DeLeeuw, K., Almeroth, K., Bimber, B., Chun, D., Zhang, H. (2009). Clickers in the classroom: Fostering learning with questioning methods in large lecture classes. Contemporary Educational Psychology, 34, 51-57. doi:10.1016/j.cedpsych.2008.04.002.
25. McDaniel, M. A., Agarwal, P. K., Huelser, B. J., McDermott, K. B., & Roediger, H. L. (2011). Test-enhanced learning in a middle school science classroom: The effects of quiz frequency and placement. Journal of Educational Psychology, 103, 399-414. doi:10.1037/a0021782.
26. McDaniel, M. A., Howard, D., & Einstein, G. O. (2009). The read-recite-review study strategy: Effective and portable. Psychological Science, 20, 516-522. doi:10.1111/j.1467-9280.2009.02325.x.
27. McNamara, D. S., Boonthum, C., Levinstein, I., & Millis, K. (2007). Evaluating self-explanations in iSTART: Comparing word-based and LSA algorithms. In T. K. Landauer, D. S. McNamara, S. Dennis, & W. Kintsch (Eds.), Handbook of latent semantic analysis (pp. 227-241). Mahwah, NJ: Erlbaum.
28. Miller, G. A. (1995). WordNet: A lexical database for English. Communications of the ACM, 38, 39-41. doi:10.1145/219717.219748.
29. Mohler, M., Bunescu, R., & Mihalcea, R. (2011). Learning to grade short answer questions using semantic similarity measures and dependency graph alignments. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies (pp. 752-762). Portland, OR: Association for Computational Linguistics.
30. Mohler, M., & Mihalcea, R. (2009). Text-to-text semantic similarity for automatic short-answer grading. In Proceedings of the 12th Conference of the European Chapter of the ACL (pp. 567-575). Athens, Greece: Association for Computational Linguistics.
31. Nehm, R. H., Ha, M., & Mayfield, E. (2012). Transforming biology assessment with machine learning: Automated scoring of written evolutionary explanations. Journal of Science Education Technology, 21, 183-196. doi:10.1007/s10956-011-9300-9.
32. Noorbehbahani, F., & Kardan, A. A. (2011). The automatic assessment of free text answers using a modified BLEU algorithm. Computers & Education, 56, 337-345. doi:10.1016/j.compedu.2010.07.013.
33. Pérez-Marín, D., Pascual-Nieto, I., & Rodríguez, P. (2009). Computerassisted assessment of free-text answers. The Knowledge Engineering Review, 24, 353-374. doi:10.1017/S026988890999018X.
34. Porter, M. F. (1980). An algorithm for suffix stripping. Program, 14, 130-137. doi:10.1108/eb046814.
35. Pyc, M. A., & Rawson, K. A. (2009). Testing the retrieval effort hypothesis: Does greater difficulty correctly recalling information lead to higher levels of memory? Journal of Memory and Language, 60, 437-447. doi:10.1016/j.jml.2009.01.004.
36. Pyc, M. A., & Rawson, K. A. (2010). Why testing improves memory: Mediator effectiveness hypothesis. Science, 330, 335. doi:10.1126/science.1191465.
37. Rao, J. N. K., & Scott, A. J. (1992). A simple method for the analysis of clustered binary data. Biometrics, 48, 577-585. doi:10.2307/2532311.
38. Rawson, K. A., & Dunlosky, J. (2007). Improving students' self-evaluation of learning for key concepts in textbook materials. European Journal of Cognitive Psychology, 19, 559-579. doi:10.1080/09541440701326022.
39. Rawson, K. A., & Dunlosky, J. (2011). Optimizing schedules of retrieval practice for durable and efficient learning: How much is enough? Journal of Experimental Psychology: General, 140, 283-302. doi:10.1037/a0023956.
40. Roediger, H. L., Agarwal, P. K., McDaniel, M. A., & McDermott, K. B. (2011). Test-enhanced learning in the classroom: Long-term improvements from quizzing. Journal of Experimental Psychology: Applied, 17, 382-395. doi:10.1037/a0026252.
41. Roediger, H. L., & Butler, A. C. (2011). The critical role of retrieval practice in long-term retention. Trends in Cognitive Sciences, 15, 20-27. doi:10.1016/j.tics.2010.09.003.
42. Roediger, H. L., & Karpicke, J. D. (2006). Test-enhanced learning: Taking memory tests improves long-term retention. Psychological Science, 17, 249-255. doi:10.1111/j.1467-9280.2006.01693.x.
43. Sukkarieh, J. Z., Pulman, S. G., & Raikes, N. (2004). Auto-Marking 2: An update on the UCLES-Oxford University research into using computational linguistics to score short, free text responses. Philadelphia, PA: International Association of Educational Assessment.
44. Wiemer-Hastings, P., & Graesser, A. (2000). Select-a-kibitzer: A computer tool that gives meaningful feedback on student compositions. Interactive Learning Environments, 8, 149-169. doi:10.1076/1049-4820(200008)8:2;1-B;FT149.
45. Wissman, K. T., Rawson, K. A., & Pyc, M. A. (2011). The interim test effect: Testing prior material can facilitate the learning of new material. Psychonomic Bulletin & Review, 18, 1140-1147. doi:10.3758/s13423-011-0140-7.
46. Wissman, K. T., Rawson, K. A., & Pyc, M. A. (2012). How and when do students use flashcards? Memory, 20, 568-579. doi:10.1080/09658211 .2012.687052.