Peer assessment for massive open online courses (MOOCs)

International Review of Research in Open and Distance Learning

Volumn 15, Issue 3, 2014, Pages 312-327

  Suen, Hoi K ^a  

^a PENNSYLVANIA STATE UNIVERSITY   (United States)

Author keywords

Calibrated peer review; Credibility index; Formative evaluation; Massive open online courses (MOOCs); Peer assessment

Indexed keywords

EID: 84905123144     PISSN: None     EISSN: 14923831     Source Type: Journal    
DOI: 10.19173/irrodl.v15i3.1680     Document Type: Article

References (41)

1. Anderson, T. (2008). Towards a theory of online learning. In T. Anderson (Ed.), The theory and practice of online learning (pp. 91-119). Athabasca, Canada: Athabasca University Press. Retrieved from: http://cde.athabascau.ca/online_book/second_edition.html
2. Balfour, S. P. (2013). Assessing writing in MOOCs: Automated essay scoring and calibrated peer review. Research&Practice in Assessment, 8. Retrieved from http://www.rpajournal.com/assessing-writing-in-moocs-automated-essay-scoring-and-calibrated-peer-review/
3. Barber, M., Donnelly, K.,&Rizvi, K. (2013). An avalanche is coming: Higher education and the revolution ahead. London, England: Institute for Public Policy Research.
4. Bloom, B. (1971). Mastery learning. New York: Holt, Rinehart,&Winston.
5. Bouzidi, L.,&Jaillet, A. (2009). Can online peer assessment be trusted? Educational Technology&Society, 12(4), 257-268.
6. Chaudhary, S.V.S.,&Dey, N. (2013). Assessment in open and distance learning system (ODL): A challenge. Open Praxis, 5(3), 207-216.
7. Costello, J.,&Crane, D. (2013). Technologies for learner-centered feedback. Open Praxis, 5(3), 217-225.
8. Falchikov, N.,&Goldfinch, J. (2000). Student peer assessment in higher education: A meta-analysis comparing peer and teacher marks. Review of Educational Research, 70(3), 287-322
9. Feuerstein, R., Feuerstein, S., Falik, L.,&Rand, Y. (2002). Dynamic assessments of cognitive modifiability. Jerusalem, Israel: ICELP Press.
10. Frederiksen, J.R.,&Collins, A. (1989). A systems approach to educational testing. Educational Researcher, 18(9), 27-32.
11. Furman, B.,&Robinson, W. (2003). Improving engineering report writing with Calibrated Peer Review™. In D. Budny (Ed.), Proceedings of the 33rd Annual Frontiers in Education Conference. Piscataway, NJ: IEEE Digital Library.
12. Gielen, S., Dochy, F., Onghena, P., Struyven, K.,&Smeets, S. (2011). Goals of peer assessment and their associated quality concepts. Studies in Higher Education, 36(6), 719-735.
13. Goldin, I. M. (2012). Accounting for peer reviewer bias with Bayesian models. Proceedings of the Workshop on Intelligent Support for Learning Groups at the 11th International Conference on Intelligent Tutoring Systems. Chania, Greece.
14. Goldin, I. M.,&Ashley, K. D. (2011). Peering inside peer review with Bayesian models. In G. Biswas, S. Bull, J. Kay,&A. Mitrović (Eds.), Proceedings of 15th International Conference Artificial Intelligence in Education (Vol. 6738, pp. 90-97), Springer.
15. Hartberg, Y., Guernsel, A. B., Simpson, N. J.,&Balaster, V. (2008). Development of student writing in biochemistry using calibrated peer review. Journal for the Scholarship of Teaching and Learning, 8(1), 29-44.
16. Haywood, C. H.&Lidz, C. S. (2007). Dynamic assessment in practice: Clinical and educational applications. New York: Cambridge University Press
17. Jordan, K. (2013). MOOC completion rates: The data. Retrieved from http://www.katyjordan.com/MOOCproject
18. Kauza, J. (2014). MOOC assigned. In S.D. Krause&C. Lowe (Eds.), More questions than answers: Scratching at the surface of MOOCs in higher education (pp. 105-113). Anderson, S.C.: Parlor Press.
19. Kirschner, P.A.,&van Merrienboer, J.J.G. (2013). Do learners really know best? Urban legends in education. Educational Psychologist, 48(3), 169-183.
20. Krause, S. D. (2014). MOOC assigned. In S.D. Krause&C. Lowe (Eds.), Invasion of the MOOCs: The promises and perils of massive open online courses (pp. 122-129). Anderson, S.C.: Parlor Press.
21. nd ed). The Pennsylvania State University, Center for Advanced Language Proficiency Education and Research.
22. Letseka, M.,&Pitsoe, V. (2013). Reflections on assessment in open distance learning (ODL): The case of the University of South Africa (UNISA). Open Praxis, 5(3), 197-206.
23. Li, H.L., Xiong, Y., Zang, X.J., Kornbaher, M., Lyu, Y.S., Chung, K.S.,&Suen, H.K. (2014, April). Peer assessment in a digital age: A meta-analysis comparing peer and teacher ratings. Presentation at the annual meeting of the American Educational Research Association, Philadelphia, PA.
24. Likkel, L. (2012). Calibrated Peer Review™ essays increase student confidence in assessing their own writing. Journal of College Science Teaching, 41(3), 42-47.
25. McCarty, T., Parkes, M.V., Anderson, T. T., Mines, J., Skipper, B. J.,&Grebosky, J. (2005). Improved patient notes from medical students during web-based teaching using faculty-Calibrated Peer Review and self-assessment. Academic Medicine, 80(10 suppl.), S67-70.
26. Norton, S.M. (1992). Peer assessments of performance and ability: An exploratory meta-analysis of statistical artifacts and contextual moderators. Journal of Business and Psychology, 6(3), 38-399.
27. Owasso Public Schools v. Falvo, No. 00-1073. Retrieved from http://laws.findlaw.com/us/000/00-1073.html
28. Pelaez, N. (2002). Problem-based writing with peer review improves academic performance in physiology. Advances in Physiology Education, 26, 174-184.
29. Piech, C., Huang, J., Chen, Z., Do, C., Ng, A.,&Koller, D. (2013) Tuned models of peer assessment in MOOCs. Palo Alto, CA: Stanford University. Retrieved from http://www.stanford.edu/~cpiech/bio/papers/tuningPeerGrading.pdf
30. Reynolds, J. A.,&Moskovitz, C. (2008). Calibrated Peer Review™ assignments in science courses: Are they designed to promote critical thinking and writing skills? Journal of College Science Teaching, 38(2), 60-66.
31. Siemens, G. (2005). Connectivism: A learning theory for the digital age. International Journal of Instructional Technology&Distance Learning, 2(1). Retrieved from http://www.itdl.org/Journal/Jan_05/index.htm.
32. Skinner, B. F. (1968). The technology of teaching. New York: Appleton-Century-Crofts.
33. Suen, H. K. (2013a, October). Role and current methods of peer assessment in massive open online courses (MOOCs). Presentation at the First International Workshop on Advanced Learning Sciences (IWALS), University Park, Pennsylvania, U.S.A.
34. Suen, H. K. (2013b, November). Peer assessment in MOOCs. Presentation at the Educause Learning Initiative, 2013 Online Fall Focus Group. http://www.educause.edu/events/eli-online-fall-focus-session-2013
35. Suen, H. K.,&Parkes, J.T. (1996). Challenges and opportunities for student assessment in distance education. DEOSNEWS, 6(7), file no. 96-00007.
36. Suen, H. K.,&Pursel, B.K. (2014, March). Scalable formative assessment in Massive Open Online Courses (MOOCs). Presentation at the Teaching and Learning with Technology Symposium, University Park, Pennsylvania, U.S.A.
37. Topping, K. J. (2005): Trends in peer learning. Educational Psychology: An International Journal of Experimental Educational Psychology, 25(6), 631-645.
38. Waldrop, M. M. (2013, March). Online learning: Campus 2.0: Massive open online courses are transforming higher education - and providing fodder for scientific research. Nature, 495, 160-163. Retreived from http://www.nature.com/news/online-learning-campus-2-0-1.12590.
39. Williamson, M., Xi, X.,&Breyer, F.J. (2012). A framework for evaluation and use of automated scoring. Educational Measurement: Issues and Practice, 31(1), 2-13.
40. Xiong, Y., Goins, D., Suen, H.K., Pun, W.H.,&Zang, X. (2014, April). A proposed credibility index (CI) in peer assessment. Presentation at the annual meeting of the National Council on Measurement in Education, Philadelphia, PA.
41. Zhang, B. (2013). Contrasting automated and human scoring of essays. R&D Connections, No. 21(March). Princeton, NJ: Educational Testing Services.