A systematic review of tools that support peer assessment

Computer Science Education

Volumn 19, Issue 4, 2009, Pages 209-232

  Luxton Reilly, Andrew ^a  

^a UNIVERSITY OF AUCKLAND   (New Zealand)

Author keywords

Peer assessment; Peer review; Peer assisted learning; Social learning

Indexed keywords

EID: 77955810285     PISSN: 08993408     EISSN: 17445175     Source Type: Journal    
DOI: 10.1080/08993400903384844     Document Type: Article

References (58)

1. Anderson, N., and Shneiderman, B., Use of peer ratings in evaluating computer program quality. Proceedings of the 15th Annual SIGCPR Conference. Arlington, Virginia, USA. pp. 218–226. New York, NY: ACM.
2. Ballantyne, R., Hughes, K., and Mylonas, A., 2002. Developing procedures for implementing peer assessment in large classes using an action research process. Assessment and Evaluation in Higher Education, 27 (5): 427–441.
3. Bhalerao, A., and Ward, A., 2001. Towards electronically assisted peer assessment: A case study. Association for Learning Technology Journal, 9 (1): 26–37.
4. Brereton, P., Kitchenham, B., Budgen, D., Turner, M., and Khalil, M., 2007. Lessons from applying the systematic literature review process within the software engineering domain. The Journal of Systems and Software, 80: 571–583.
5. Chalk, B., and Adeboye, K., Peer assessment of program code: A comparison of two feedback instruments. 6th HEA-ICS annual conference. pp. 106–110. York: University of York.
6. Chapman, O., and Fiore, M., 2000. Calibrated peer review™. Journal of Interactive Instruction Development, 12 (3): 11–15.
7. Cho, K., and Schunn, C. D., 2007. Scaffolded writing and rewriting in the discipline: A web-based reciprocal peer review system. Computers and Education, 48 (3): 409–426.
8. Davies, P., 2000. Computerized peer assessment. Innovations in Education and Training International, 37 (4): 346–355.
9. Davies, P., 2003. Closing the communications loop on the computerized peer-assessment of essays. ALT-J, 11 (1): 41–54.
10. Davies, P., 2004. Don't write, just mark: The validity of assessing student ability via their computerized peer-marking of an essay rather than their creation of an essay. ALT-J, 12 (3): 261–277.
11. Davies, P., 2006. Peer assessment: Judging the quality of the students' work by comments rather than marks. Innovations in Education and Training International, 43 (1): 69–82.
12. Davies, P., 2009. Review and reward within the computerised peer-assessment of essays. Assessment and Evaluation in Higher Education, 34: 321–333.
13. de Raadt, M., Lai, D., and Watson, R., An evaluation of electronic individual peer assessment in an introductory programming course. Seventh Baltic sea conference on computing education research (Koli Calling 2007). Edited by: Lister, R., and Simon. Vol. 88, pp. 53–64. Koli National Park, Finland: ACS.
14. de Raadt, M., Toleman, M., and Watson, R., Electronic peer review: A large cohort teaching themselves? In. Proceedings of the 22nd Annual Conference of the Australasian Society for Computers in Learning in Tertiary Education (ASCILITE'05). Brisbane, Australia. pp. 159–168. http://www.ascilite.org.au/conferences/brisbane05/blogs/proceedings/17_de%20Raadt.pdf
15. Denny, P., Hamer, J., Luxton-Reilly, A., and Purchase, H., PeerWise: Students sharing their multiple choice questions. ICER '08: Proceeding of the fourth international workshop on Computing education research. Sydney, Australia. pp. 51–58. New York, NY: ACM.
16. Denny, P., Luxton-Reilly, A., and Hamer, J., The PeerWise system of student contributed assessment questions. Tenth Australasian computing education conference (ACE 2008). Edited by: Simon and Hamilton, M., pp. 69–74. Wollongong, NSW, Australia: ACS.
17. Denny, P., Luxton-Reilly, A., and Hamer, J., Student use of the PeerWise system. ITICSE '08: Proceedings of the 13th annual SIGCSE conference on Innovation and technology in computer science education. pp. 73–77. Madrid, Spain: ACM.
18. Denny, P., Luxton-Reilly, A., and Simon, B., Quality of student contributed questions using PeerWise. Eleventh Australasian computing education conference (ACE 2009) (Vol. 95 of CRPIT. Wellington, New Zealand. Edited by: Hamilton, M., and Clear, T., pp. 55–64. Wellington, New Zealand: Australian Computer Society.
19. Dochy, F., Segers, M., and Sluijsmans, D., 1999. The use of self-, peer and co-assessment in higher education: A review. Studies in Higher Education, 24 (3): 331–350.
20. Downing, T., and Brown, I., 1997. Learning by cooperative publishing on the World-Wide Web. Active Learning, 7: 14–16.
21. Falchikov, N., 1995. Peer feedback marking: Developing Peer Assessment. Innovations in Education and Teaching International, 32 (2): 175–187.
22. Falchikov, N., and Goldfinch, J., 2000. Student peer assessment in higher education: A meta-analysis comparing peer and teacher marks. Review of Educational Research, 70 (3): 287–322.
23. Figl, K., Bauer, C., and Mangler, J., Online versus face-to-face peer team reviews. 36th ASEE/IEEE Frontiers in Education Conference. pp. 7–12. San Diego: IEEE.
24. Freeman, M., and McKenzie, J., 2002. SPARK, a confidential web-based template for self and peer assessment of student teamwork: Benefits of evaluating across different subjects. British Journal of Educational Technology, 33 (5): 551–569.
25. Gehringer, E., Strategies and mechanisms for electronic peer review. Frontiers in Education Conference, 2000. FIE 2000. 30th Annual, 1, F1B/2–F1B/7 (Vol. 1). Kansas City, MO: IEEE.
26. Hamer, J., Cutts, Q., Jackova, J., Luxton-Reilly, A., McCartney, R., Purchase, H., 2008. Contributing student pedagogy. SIGCSE Bulletin, 40 (4): 194–212.
27. Hamer, J., Kell, C., and Spence, F., Peer assessment using aropä. ACE '07: Proceedings of the 9th Australasian conference on computing education. Ballarat, Victoria, Australia. pp. 43–54. Darlinghurst, Australia: Australian Computer Society.
28. Hamer, J., Ma, K. T.K., and Kwong, H. H.F., A method of automatic grade calibration in peer assessment. ACE '05: Proceedings of the 7th Australasian conference on computing education. Newcastle, New South Wales, Australia. pp. 67–72. Darlinghurst, Australia: Australian Computer Society.
29. Kali, Y., and Ronen, M., Design principles for online peer-evaluation: Fostering objectivity. Proceedings of CSCL 2005: Computer support for collaborative learning: The next 10 years!. Edited by: Koschmann, T., Suthers, D. D., and Chan, T. W., pp. 247–251. Mahwah, NJ: Lawrence Erlbaum Associates.
30. Kitchenham, B., Procedures for perfoming systematic reviews. Technical report TR/SE-0401, Keele University. http://www.scientificcommons.org/42461878
31. Lin, S., Liu, E., and Yuan, S., 2001. Web-based peer assessment: Feedback for students with various thinking-styles. Journal of Computer Assisted Learning, 17 (4): 420–432.
32. Liu, E. Z.F., Lin, S., Chiu, C. H., and Yuan, S. M., 2001. Web-based peer review: The learner as both adapter and reviewer. Education, IEEE Transactions on, 44 (3): 246–251.
33. Lutteroth, C., and Luxton-Reilly, A., Flexible learning in CS2: A case study. Proceedings of the 21st annual conference of the national advisory committee on computing qualifications. Edited by: Mann, S., and Lopez, M., Auckland, New Zealand: NACCQ.
34. Mann, B., The post and vote model of web-based peer assessment. Proceedings of World conference on educational multimedia, hypermedia and telecommunications 2005. Edited by: Kommers, P., and Richards, G., pp. 2067–2074. Chesapeake, VA: AACE.
35. McLuckie, J., and Topping, K. J., 2004. Transferable skills for online peer learning. Assessment and Evaluation in Higher Education, 29 (5): 563–584.
36. Miao, Y., and Koper, R., An efficient and flexible technical approach to develop and deliver online peer assessment. Proceedings of the 7th computer supported collaborative learning (CSCL 2007) conference ‘Mice, Minds, and Society’. July, New Jersey, USA. Edited by: Chinn, C. A., Erkens, G., and Puntambekar, S., pp. 502–511. International Society of the Learning Sciences
37. Millard, D., Sinclair, P., and Newman, D., PeerPigeon: A web application to support generalised peer review. E-Learn 2008–World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education. November17–212008.
38. Miller, P. J., 2003. The effect of scoring criteria specificity on peer and self-assessment. Assessment and Evaluation in Higher Education, 28 (4): 383–394.
39. Murphy, L., and Wolff, D., 2005. Take a minute to complete the loop: Using electronic classroom assessment techniques in computer science labs. Journal of Computing Sciences in Colleges, 21 (1): 150–159.
40. Ngu, A. H.H., and Shepherd, J., 1995. Engineering the ‘Peers’ system: The development of a computer-assisted approach to peer assessment. Research and Development in Higher Education, 18: 582–587.
41. Paré, D., and Joordens, S., 2008. Peering into large lectures: Examining peer and expert mark agreement using peerScholar, an online peer assessment tool. Journal of Computer Assisted Learning, 24 (6): 526–540.
42. Plimmer, B., and Apperley, M., Making paperless work. CHINZ '07: Proceedings of the 7th ACM SIGCHI New Zealand chapter's international conference on computer–human interaction. Hamilton, New Zealand. pp. 1–8. New York, NY: ACM.
43. Price, B., and Petre, M., Teaching programming through paperless assignments: An empirical evaluation of instructor feedback. ITiCSE '97: Proceedings of the 2nd conference on Integrating technology into computer science education. Uppsala, Sweden. pp. 94–99. New York, NY: ACM.
44. Raban, R., and Litchfield, A., 2007. Supporting peer assessment of individual contributions in groupwork. Australasian Journal of Educational Technology, 23 (1): 34–47.
45. Ronen, M., Kohen-Vacs, D., and Raz-Fogel, N., Adopt & adapt: Structuring, sharing and reusing asynchronous collaborative pedagogy. ICLS '06: Proceedings of the 7th international conference on Learning sciences. pp. 599–605. Bloomington, IN: International Society of the Learning Sciences.
46. Sackett, D. L., Richardson, W. S., Rosenberg, W., and Haynes, R. B., 1997. Evidence-based medicine: How to practice and teach EBM, London, UK: Churchill Livingstone.
47. Sitthiworachart, J., and Joy, M., Effective peer assessment for learning computer programming. ITiCSE '04: Proceedings of the 9th annual SIGCSE conference on Innovation and technology in computer science education. Leeds, UK. pp. 122–126. New York, NY: ACM.
48. Sitthiworachart, J., and Joy, M., 2008. Computer support of effective peer assessment in an undergraduate programming class. Journal of Computer Assisted Learning, 24 June: 217–231.
49. Sluijsmans, D. M.A., Brand-Gruwel, S., and van Merrinboer, J. J.G., 2002. Peer assessment training in teacher education: Effects on performance and perceptions. Assessment and Evaluation in Higher Education, 27 (5): 443–454.
50. Sung, Y. T., Chang, K. E., Chiou, S. K., and Hou, H. T., 2005. The design and application of a web-based self- and peer-assessment system. Computers and Education, 45 (2): 187–202.
51. Topping, K., 1998. Peer assessment between students in colleges and universities. Review of Educational Research, 68 (3): 249–276.
52. Trahasch, S., 2004. “ From peer assessment towards collaborative learning ”. In Frontiers in Education, 2004. FIE 2004. 34th Annual, Vol. 2, F3F–16–20. Savannah, GA: IEEE.
53. Trivedi, A., Kar, D. C., and Patterson-McNeill, H., 2003. Automatic assignment management and peer evaluation. Journal of Computing Sciences in Colleges, 18 (4): 30–37.
54. Walvoord, M. E., Hoefnagels, M. H., Gaffin, D. D., Chumchal, M. M., and Long, D. A., 2008. An analysis of calibrated peer review (CPR) in a science lecture classroom. Journal of College Science Teaching, 37 (4): 66–73.
55. Webster, J., and Watson, R. T., 2002. Analyzing the past to prepare for the future: Writing a literature review. MIS Quarterly, 26 (2): xii–xxiii.
56. Wolfe, W. J., Online student peer reviews. CITC5 '04: Proceedings of the 5th conference on Information technology education. Salt Lake City, UT. pp. 33–37. New York, NY: ACM.
57. Xiao, Y., and Lucking, R., 2008. The impact of two types of peer assessment on students’ performance and satisfaction within a Wiki environment. The Internet and Higher Education, 11 (3–4): 186–193. Special Section of the AERA Education and World Wide Web Special Interest Group (EdWeb/SIG)
58. Zeller, A., Making students read and review code. ITiCSE '00: Proceedings of the 5th annual SIGCSE/SIGCUE ITiCSE conference on Innovation and technology in computer science education. Helsinki, Finland. pp. 89–92. New York, NY: ACM.