The peril and promise of pretests in informal massive open online courses

Early Warning Systems and Targeted Interventions for Student Success in Online Courses

Volumn , Issue , 2020, Pages 22-36

  Janelli, Maria ^a   Lipnevich, Anastasiya ^b  

^a DIVISION OF INVERTEBRATE ZOOLOGY   (United States)

^b CITY UNIVERSITY OF NEW YORK   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 85121477888     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.4018/978-1-7998-5074-8.ch002     Document Type: Chapter

References (49)

1. Bjork, E. L., Storm, B. C., & deWinstanley, P. A. (2010). Learning from the consequences of retrieval: Another test effect. In Successful remembering and successful forgetting: A festschrift in honor of Robert A. Bjork (First Edition). Psychology Press.
2. Black, P., & Wiliam, D. (1998). Assessment and classroom learning. Assessment in Education: Principles, Policy & Practice, 5(1), 7-74. doi:10.1080/0969595980050102
3. Brookhart, S. M. (2018). Summative and formative feedback. In A. A. Lipnevich & J. K. Smith (Eds.), The Cambridge Handbook of Instructional Feedback. Cambridge University Press. doi:10.1017/9781316832134.005
4. Bull, J., & Stephens, D. (1999). The use of question mark software for formative and summative assessment in two universities. Innovations in Education & Training International, 36(2), 128-136. doi:10.1080/1355800990360205
5. Butler, A. C., & Roediger, H. L. III. (2008). Feedback enhances the positive effects and reduces the negative effects of multiple-choice testing. Memory & Cognition, 36(3), 604-616. doi:10.3758/MC.36.3.604 PMID:18491500
6. Carpenter, S. K. (2009). Cue strength as a moderator of the testing effect: The benefits of elaborative retrieval. Journal of Experimental Psychology. Learning, Memory, and Cognition, 35(6), 1563-1569. doi:10.1037/a0017021 PMID:19857026
7. Clariana, R. B., Ross, S. M., & Morrison, G. R. (1991). The effects of different feedback strategies using computer-administered multiple-choice questions as instruction. Educational Technology Research and Development, 39(2), 5-17. doi:10.1007/BF02298149
8. Dunlosky, J., Rawson, K. A., Marsh, E. J., Nathan, M. J., & Willingham, D. T. (2013). Improving 135 participants' learning with effective learning techniques: Promising directions from cognitive and educational psychology. Psychological Science in the Public Interest, 14(1), 4-58. doi:10.1177/1529100612453266 PMID:26173288
9. Evans, J. B., Baker, R. B., & Dee, T. (2015). Persistence Patterns in Massive Open Online Courses (MOOCs) (CEPA Working Paper No.15-09). Retrieved from Stanford Center for Education Policy Analysis: https://cepa.stanford.edu/wp15-09
10. Floratos, N., Guasch, T., & Espasa, A. (2015). Recommendations on Formative Assessment and Feedback Practices for stronger engagement in MOOCs. Open Praxis, 7(2), 141-152. doi:10.5944/openpraxis.7.2.194
11. Gené, O. B., Núñez, M. M., & Blanco, Á. F. (2014). Gamification in MOOC: Challenges, Opportunities and Proposals for Advancing MOOC Model. Proceedings of the 2nd International Conference on Technological Ecosystems for Enhancing Multiculturality (TEEM). 10.1145/2669711.2669902
12. Gikandi, J. W., Morrow, D., & Davis, N. E. (2011). Online formative assessment in higher education: A review of the literature. Computers & Education, 57(4), 2333-2351. doi:10.1016/j.compedu.2011.06.004
13. Glover, J. A. (1989). The "testing" phenomenon: Not gone but nearly forgotten. Journal of Educational Psychology, 81(3), 392-399. doi:10.1037/0022-0663.81.3.392
14. Gregori, E. B., Zhang, J., Galván-Fernández, C., & Fernández-Navarro, F. de A. (2018). Learner support in MOOCs: Identifying variables linked to completion. Computers & Education, 122, 153-168. doi:10.1016/j.compedu.2018.03.014
15. Guskey, T. R. (2018). Feedback, correctives, and the use of pre-assessments. In A. A. Lipnevich & J. K. Smith (Eds.), The Cambridge Handbook of Instructional Feedback. Cambridge University Press. doi:10.1017/9781316832134.021
16. Hartley, J., & Davies, I. K. (1976). Preinstructional strategies: The role of pretests, behavioral objectives, overviews and advance organizers. Review of Educational Research, 12(2), 239-265. doi:10.3102/00346543046002239
17. Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77(1), 16-17. doi:10.3102/003465430298487
18. Hew, K. F., & Cheung, W. S. (2014). Students' and instructors' use of massive open online courses (MOOCs): Motivations and challenges. Educational Research Review, 12, 4558. doi:10.1016/j.edurev.2014.05.001
19. Hone, K. S., & El Said, G. R. (2016). Exploring the factors affecting MOOC retention: A survey study. Computers & Education, 98, 157-168. doi:10.1016/j.compedu.2016.03.016
20. Janelli, M. (2019). Effects of Pre-Tests and Feedback on Performance Outcomes in Massive Open Online Courses: What Works and What Doesn't? Retrieved from PQDT - Global.
21. Janelli, M., & Lipnevich, A. (in press). Effects of pre-tests and feedback on performance outcomes and persistence in Massive Open Online Courses. Academic Press.
22. Jung, Y., & Lee, J. (2018). Learning engagement and persistence in Massive Open Online Courses (MOOCS). Computers & Education, 122, 9-22. doi:10.1016/j.compedu.2018.02.013
23. Kalkanis, I. (2019). Learner pathways: Exploring clickstream data to understand how learners personalise their experiences. Coursera Conference, London, UK.
24. Karpicke, J. D., & Roediger, H. L. III. (2008). The critical importance of retrieval for learning. Science, 319(5865), 966-968. doi:10.1126cience.1152408 PMID:18276894
25. Kingston, N., & Nash, B. (2011). Formative assessment: A meta-analysis and a call for research. Educational Measurement: Issues and Practice, 30(4), 28-37. doi:10.1111/j.1745-3992.2011.00220.x
26. Koedinger, K. R., McLaughlin, E. A., & Heffernan, N. T. (2010). A quasi-experimental evaluation of an on-line formative assessment and tutoring system. Journal of Educational Computing Research, 43(4), 489-510. doi:10.2190/EC.43.4.d
27. Koedinger, K. R., Mclaughlin, E. A., Kim, J., Jia, J. Z., & Bier, N. L. (2015). Learning is not a spectator sport: Doing is better than watching for learning from a MOOC. L@S 2015, 111-120.
28. Lipnevich, A. A., Berg, D. A. G., & Smith, J. K. (2017). Toward a model of student response to feedback. In G. T. L. Brown & L. R. Harris (Eds.), The Handbook of Human and Social Conditions in Assessment (pp. 169-185). Routledge.
29. Maggioncalda, J. (2019). Keynote address. Coursera Conference, London, UK.
30. Marsh, E. J., Roediger, H. L., Bjork, R. A., & Bjork, E. L. (2007). The memorial consequences of multiple-choice testing. Psychonomic Bulletin & Review, 14(2), 194-199. doi:10.3758/BF03194051 PMID:17694900
31. Narciss, S., & Huth, K. (2002). How to design informative tutoring feedback for multi-media learning. In H. Niegemann, R. Brünken, & D. Leutner (Eds.), Instructional design for multimedia learning (pp. 2-16). Waxmann.
32. Pappano, L. (2012, November 4). The Year of the MOOC. The New York Times, p. ED26. Retrieved from https://www.nytimes.com/2012/11/04/education/edlife/massive-open-online-courses-aremultiplying-ata-rapid-pace.html
33. Parshall, C. G. (1995). Practical issues in computer-based testing. Journal of Instruction Delivery Systems, 13-17.
34. Pressley, M., Tanenbaum, R., McDaniel, M. A., & Wood, E. (1990). What happens when university participants try to answer prequestions that accompany textbook material? Contemporary Educational Psychology, 15(1), 27-35. doi:10.1016/0361-476X(90)90003-J
35. Richland, L. E., Kornell, N., & Kao, L. S. (2009). The pretesting effect: Do unsuccessful retrieval attempts enhance learning? Journal of Experimental Psychology. Applied, 15(3), 243257. doi:10.1037/a0016496 PMID:19751074
36. Roediger, H. L. III, & Karpicke, J. D. (2006). Test-enhanced learning: Taking memory tests improves long-term retention. Psychological Science, 17(3), 249-255. doi:10.1111/j.1467-9280.2006.01693.x PMID:16507066
37. Shah, D. (2018). By The Numbers: MOOCs in 2018. Class Central.
38. Smith, J. K., & Lipnevich, A. A. (2018). Instructional feedback: Analysis, synthesis, and extrapolation. In A. A. Lipnevich & J. K. Smith (Eds.), The Cambridge Handbook of Instructional Feedback. Cambridge University Press. doi:10.1017/9781316832134.029
39. Undergraduate Retention and Graduation Rates. (2019). Retrieved February 5, 2019, from https://nces.ed.gov/programs/coe/indicator_ctr.asp
40. Whitelock, D. (2011). Activating assessment for learning: Are we on the way with web 2.0? In M. J. W. Lee & C. McLoughlin (Eds.), Web 2.0-Based E-Learning: Applying Social Informatics for Tertiary Teaching (pp. 319-341). Information Science Reference. doi:10.4018/978-1-60566-294-7.ch017
41. Williams, K. M., Stafford, R. E., Corliss, S. B., & Reilly, E. D. (2018). Examining student characteristics, goals, and engagement in Massive Open Online Courses. Computers & Education, 126, 433-442. doi:10.1016/j.compedu.2018.08.014
42. Beckman, W. S. (2008). Pre-testing as a method of conveying learning objectives. Journal of Aviation/Aerospace Education Research, 17(172). Advance online publication. doi:10.15394/jaaer.2008.1447
43. DeBoer, J., Ho, A. D., Stump, G. S., & Breslow, L. (2014). Changing "course": Reconceptualizing educational variables for massive open online courses. Educational Researcher, 43(2), 74-84. doi:10.3102/0013189X14523038
44. Guskey, T. R. (2018). Feedback, correctives, and the use of pre-assessments. In A. A. Lipnevich & J. K. Smith (Eds.), The Cambridge Handbook of Instructional Feedback. Cambridge University Press. doi:10.1017/9781316832134.021
45. Janelli, M., & Lipnevich, A. (n.d.). The Peril and Promise of Pre-tests in Informal Massive Open Online Courses. In D. Glick, A. Cohen, & C. Chang (Eds.), Early Warning Systems and Targeted Interventions for Student Success in Online Courses. IGI Global.
46. Lipnevich, A. A., Berg, D. A. G., & Smith, J. K. (2017). Toward a model of student response to feedback. In G. T. L. Brown & L. R. Harris (Eds.), The Handbook of Human and Social Conditions in Assessment (pp. 169-185). Routledge.
47. Seaton, D. T., Bergner, Y., Chuang, I., Mitros, P., & Pritchard, D. E. (2014). Who does what in a massive open online course? Communications of the ACM, 57(4), 58-65. Advance online publication. doi:10.1145/2500876
48. Smith, J. K., & Lipnevich, A. A. (2018). Instructional feedback: Analysis, synthesis, and extrapolation. In A. A. Lipnevich & J. K. Smith (Eds.), The Cambridge Handbook of Instructional Feedback. Cambridge University Press. doi:10.1017/9781316832134.029
49. Wisniewski, B., Zierer, K., & Hattie, J. (2020). The power of feedback revisited: A meta-analysis of educational feedback research. Frontiers in Psychology, 10, 1-14. doi:10.3389/fpsyg.2019.03087 PMID:32038429