Peer grading in a MOOC: Reliability, validity, and perceived effects

Journal of Asynchronous Learning Network

Volumn 18, Issue 2, 2014, Pages

  Luo, Heng ^a   Robinson, Anthony C ^a   Park, Jae Young ^a  

^a PENNSYLVANIA STATE UNIVERSITY   (United States)

Author keywords

MOOC; Peer grading; Reliability; Validity

Indexed keywords

EID: 84905124754     PISSN: 19395256     EISSN: 10928235     Source Type: Journal    
DOI: 10.24059/olj.v18i2.429     Document Type: Article

References (51)

1. Billington, H. L. (1997). Poster presentations and peer assessment: novel forms of evaluation and assessment. Journal of Biological Education, 31(3), 218-220.
2. Bloom, B. S. (1956). Taxonomy of educational objectives: Vol. 1. Cognitive domain. New York: McKay.
3. Bostock, S. (2000). Student peer assessment. Keele, Staffordshire: Centre for Learning Technology, Keele University. Retrieved from http://www.reading.ac.uk/web/FILES/engageinassessment/Student_peer_assessment_-_Stephen_Bostock.pdf
4. Bouzidi, L., & Jaillet, A. (2009). Can Online Peer Assessment be Trusted? Educational Technology & Society, 12 (4), 257-268.
5. Brown, S., Race, P., & Rust, C. (1995). Using and experiencing assessment, in P. Knight (Ed.) Assessment for Learning in Higher Education (pp.75-85). London: Kogan Page/SEDA.
6. Brown, S., Rust, C., & Gibbs, G. (1994). Strategies for diversifying assessment in higher education. Oxford: Oxford Centre for Staff Development.
7. Butcher, A. C., Stefani, L. A. J., & Tariq, V. N. (1995). Analysis of peer-, self-and staff-assessment in group project work. Assessment in Education, 2(2), 165-185.
8. Cheng, W., & Warren, M. (1999). Peer and teacher assessment of the oral and written tasks of a group project. Assessment & Evaluation in Higher Education, 24, 301-314.
9. Cho, K., Schunn, C., & Wilson, R. (2006). Validity and Reliability of Scaffolded Peer Assessment of Writing from Instructor and Student Perspectives. Journal of Educational Psychology, 98(4), 891-901.
10. Coursera. (n.d.). Pedagogical Foundations. Retrieved from https://www.coursera.org/about/pedagogy
11. Coursera. (2014, March 12) How will my grade be determined? Retrieved from. http://help.coursera.org/customer/portal/articles/1163304-how-will-my-grade-be-determined
12. Dancey, C. P., & Reidy, J. (2002). Statistics without maths for psychology (2nd ed). London: Prentice Hall.
13. Falchikov, N. (1994). Learning from peer feedback marking: student and teacher perspectives. In H. C. Foot, C. J. Howe, A. Anderson, A. K. Tolmie, & D. A. Warden (Eds.), Group and interactive learning (pp. 411-416). Southampton and Boston: Computational Mechanics Publications.
14. Falchikov, N., & Goldfinch, J. (2000). Student Peer Assessment in Higher Education: A Meta-Analysis Comparing Peer and Teacher Marks. Review of Educational Research, 70(3), 287-322.
15. Freeman, M. (1995). Peer assessment by groups of group work, Assessment and Evaluation in Higher Education, 20(3), 289-300.
16. Fry, S. A. (1990). Implementation and evaluation of peer marking in higher education. Assessment and Evaluation in Higher Education, 15(3), 177-189.
17. Gay, L. R., & Airasian, P. (2003). Educational research: Competencies for analysis and application (7th ed.). Columbus, OH: Merrill, Prentice Hall.
18. Haaga, D. A. F. (1993). Peer review of term papers in graduate psychology courses. Teaching of Psychology, 20(1), 28-32.
19. Hammond, K. R., & Kern, F. (1959). Teaching comprehensive medical care: a psychological study of a change in medical education. Cambridge, MA: Harvard University Press.
20. Kaimann, R. A. (1974). The coincidence of student evaluation by professor and peer group using rank correlation. The Journal of Educational Research, 68(4), 152-153.
21. Korman, M., & Stubblefield, R. L. (1971). Medical school evaluation and internship performance. Journal of Medical Education, 46, 670-673.
22. Lu, R., & Bol, L. (2007). A comparison of anonymous versus identifiable e-peer review on college student writing performance and the extent of critical feedback. Journal of Interactive Online Learning, 6(2), 100-115.
23. Lu, J., & Law, N. (2012). Online peer assessment: effects of cognitive and affective feedback. Instructional Science, 40(2), 257-275.
24. Magin, D. (1993). Should student peer ratings be used as part of summative assessment? Higher Education Research and Development, 16, 537-542.
25. Magin, D. (2001). Reciprocity as a source of bias in multiple peer assessment of group work. Studies in Higher Education, 26(1), 53-63.
26. Marcoulides, G. A., & Simkin, M. G. (1995). The consistency of peer review in student writing projects. Journal of Education for Business, 70, 220-223.
27. McEwen, K. (2013, January 7). Getting to Know Coursera: Peer Assessments. Retrieved from http://cft.vanderbilt.edu/2013/01/getting-to-know-coursera-peer-assessments/
28. McGarr, O., & Clifford, A. M. (2013). 'Just enough to make you take it seriously': exploring students' attitudes towards peer assessment. Higher education, 65(6), 677-693.
29. Miller, P. J. (2003). The effect of scoring criteria specificity on peer and self-assessment. Assessment & Evaluation in Higher Education, 28(4), 383-394.
30. Mok, J. (2011). A case study of students' perceptions of peer assessment in Hong Kong. ELT journal, 65(3), 230-239.
31. Morrison, D. (2013, March 9). Why and When Peer Grading is Effective for Open and Online Learning. Retrieved from http://onlinelearninginsights.wordpress.com/2013/03/09/why-and-when-peer-grading-is-effective-for-open-and-online-learning/
32. Mowl, G., & Pain, R. (1995). Using self and peer assessment to improve students' essay writing-A case study from geography. Innovations in Education and Training International, 32, 324-335.
33. Neidlinger, J. (2013, May 13). Does peer grading of essays really work in a Coursera online class? Retrieved from http://loneprairie.net/peer-grading-coursera/
34. Oldfield, K. A., & Macalpine, M. K. (1995). Peer and self-assessment at tertiary level-an experimental report. Assessment and Evaluation in Higher Education, 20(1), 125-131.
35. Orpen, C. (1982). Student versus lecturer assessment of learning: a research note. Higher Education, 11, 567-572.
36. Pappano, L. (2012, November 2). The Year of the MOOC. The New York Times. Retrieved from http://www.nytimes.com/2012/11/04/education/edlife/massive-open-oline-courses-are-multiplying-at-a-rapid-pace.html?pagewanted=all&_r=0
37. Piech, C., Huang, J., Chen, Z., Do, C., Ng, A., & Koller, D. (2013). Tuned Models of Peer Assessment in MOOCs. Retrieved from http://www.stanford.edu/~jhuang11/research/pubs/edm13/edm13.pdf
38. Race, P. (1998). Practical pointers on peer-assessment. In S. Brown (Ed.) Peer Assessment in Practice (SEDA Paper 102) (pp.113-122). Birmingham: SEDA.
39. Rees, J. (2013, March 5). Peer Grading Can't Work. Retrieved from http://www.insidehighered.com/views/2013/03/05/essays-flaws-peer-grading-moocs#ixzz2MiKxNP7b
40. Sadler, P., & Good, E. (2006). The impact of self-and peer-grading on student learning. Educational Assessment, 11 (1), 1-31.
41. Shrout, P. E., & Fleiss, J. L. (1979). Intraclass correlations: uses in assessing rater reliability. Psychological Bulletin, 86(2), 420-428.
42. Stefani, L. A. J. (1994). Peer, self and tutor assessment: Relative reliabilities. Studies in Higher Education, 19(1), 69-75.
43. Strijbos, J. W., Narciss, S., & Dünnebier, K. (2010). Peer feedback content and sender's competence level in academic writing revision tasks: are they critical for feedback perceptions and efficiency? Learning and instruction, 20(4), 291-303.
44. Strijbos, J. W., & Sluijsmans, D. (2010). Unravelling peer assessment: Methodological, functional, and conceptual developments. Learning and Instruction, 20(4), 265-269.
45. Topping, K. J. (2009). Peer assessment. Theory into Practice, 48(1), 20-27.
46. Topping, K. J., Smith, E. F., Swanson, I., & Elliot, A. (2000). Formative peer assessment of academic writing between postgraduate students. Assessment & Evaluation in Higher Education, 25(2), 149-169.
47. Topping, K. (1998). Peer assessment between students in colleges and universities. Review of Educational Research, 68(3), 249-276.
48. Vu, T. T., & Dall'Alba, G. (2007). Students' experience of peer assessment in a professional course. Assessment & Evaluation in Higher Education, 32(5), 541-556.
49. Watters, A. (2012, August 27). The Problems with Peer Grading in Coursera. Retrieved from http://www.insidehighered.com/blogs/hack-higher-education/problems-peer-grading-coursera
50. Wen, M. L., Tsai, C. C., & Chang, C. Y. (2006). Attitudes towards peer assessment: A comparison of the perspectives of pre-service and in-service teachers. Innovations in Education and Teaching International, 43(1), 83-92.
51. Zhang, B., Johnston, L., & Kilic, G. B. (2008). Assessing the reliability of self-and peer rating in student group work. Assessment & Evaluation in Higher Education, 33(3), 329-340.