A survey of automated assessment approaches for programming assignments

Computer Science Education

Volumn 15, Issue 2, 2005, Pages 83-102

  Ala Mutka, Kirsti M ^a  

^a TAMPERE UNIVERSITY OF TECHNOLOGY   (Finland)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 33749363486     PISSN: 08993408     EISSN: 17445175     Source Type: Journal    
DOI: 10.1080/08993400500150747     Document Type: Article

References (65)

1. Ala-Mutka K, Järvinen H-M, 2004 Assessment Process for Programming Assignments Proceedings of the 4th IEEE International Conference on Advanced Learning Technologies, Finland 181 185
2. Ala-Mutka, K, Uimonen, T, and Järvinen, H-M. 2004. Supporting students In C + + Programming Courses with Automatic Program Style Assessment. Journal of Information Technology Education, 3: 245–262.
3. Antoniol, G, Casazza, G, Di Penta, M, and Fiuten, R. 2001. Object-oriented design patterns recovery. Journal of Systems and Software, 59: 181–196.
4. Arnow D, Barshay O, 1999 WebToTeach: An interactive focused programming exercise system Proceedings of the 29th ASEE/IEEE Frontiers in Education Conference 12A9/39-12A9/44
5. Becker K, 2003 Grading programming assignments using rubrics In Proceedings of the 8th annual conference on Innovation and technology in computer science education Greece, 253
6. Bettini L, Crescenzi P, Innocenti G, Loreti M, Cecchi L, 2004 An Environment for Self-Assessing Java Programming Skills in Undergraduate First Programming Courses In Proceedings of the 4th IEEE International Conference on Advanced Learning Technologies Finland, 161-165
7. Bloom BS, (Ed.) 1956 Taxonomy of Educational Objectives: The Classification of Educational Goals. Handbook I. Cognitive Domain New York: David McKay Company, Inc
8. Carter, J, English, J, Ala-Mutka, K, Dick, M, Fone, W, Fuller, U, and Sheard, J. 2003. How shall we assess this?. ACM SIGCSE Bulletin, 35 (4): 107–123.
9. Cheang, B, Kurnia, A, Lim, A, and Oon, W-C. 2003. On automated grading of Programming Assignments in an academic institution. Computers & Education, 41: 121–131.
10. Checkstyle. [Computer Software] Retrieved November 15, 2004, from http://checkstyle.sourceforge.net/
11. Chen, P. 2004. An Automated Feedback System for Computer Organization Projects. IEEE Transactions on Education, 47: 232–240.
12. Cox, K, and Clark, D. 1998. The Use of Formative Quizzes for Deep Learning. Computers & Education, 30: 157–167.
13. Darwin I, 1990 Checking C Programs with Lint New York: OReilly & Associates
14. Dromey, RG. 1995. A model for software product quality. IEEE Transactions on Software Engineering, 21: 146–162.
15. Edwards, S. 2004. Improving Student Performance by Evaluating How Well Students Test Their Own Programs. ACM Journal of Educational Resources in Computing, 3 (3): 1–24.
16. Ellsworth C, Fenwick J, Kurtz B, 2004 The Quiver System In Proceedings of the 35th SIGCSE technical symposium on Computer Science Education US, 205-209
17. English J, 2004 Automatic Assessment of GUI Programs using JEWL In Proceedings of 9th annual conference on Innovation and technology in computer science education UK, 137-141
18. English J, 2002 Experience with a computer-assisted formal programming examination In Proceedings of 7th annual conference on Innovation and technology in computer science education Denmark, 51-54
19. Foxley E, 1999 Ceilidh Documentation on the World Wide Web Retrieved November 15, 2004, from http://www.cs.nott.ac.uk/∼ceilidh/papers.html
20. GCC compiler. [Computer software] Retrieved November 15, 2004, from http://gcc.gnu.org/
21. Halstead MH, 1977 Elements of Software Science New York: Elsevier North-Holland
22. Hansen H, Ruuska M, 2003 Assessing time-efficiency in a course on data structures and algorithms In Proceedings of the 3rd Annual Finnish/Baltic Sea Conference on Computer Science Education, Finland
23. Higgins, C, Hergazy, T, Symeonidis, P, and Tsinsifas, A. 2003. The CourseMarker CBA system: Improvements over Ceilidh. Education and Information Technologies, 8: 287–304.
24. Higgins C, Symeonidis P, Tsinsifas A, 2002 Diagram-based CBA using DATsys and CourseMaster In Proceedings of the International Conference on Computers in Education 367 372
25. Howles, T. 2003. Fostering the growth of a software quality culture. ACM SIGCSE Bulletin, 35 (2): 45–47.
26. Hong, J. 2004. Guided Programming and Automated Error Analysis in an Intelligent Prolog Tutor. International Journal of Human-Computer Studies, 61: 505–534.
27. Hung, S-L, Kwok, L-F, and Chan, R. 1993. Automatic programming assessment. Computers & Education, 20: 183–190.
28. IMS Global Learning Consortium 2003 IMS Question & Test interoperability specification (Version 1.2.1). Retrieved November 15, 2004, from http:// www.imsproject.org/question/
29. Jackson D, 2000 A semi-automated approach to online assessment Proceedings of 5th annual conference on Innovation and technology in computer science education, Finland 164 167
30. Jackson D, Usher M, 1997 Grading Student Programs using ASSYST Proceedings of the 28th SIGCSE technical symposium on Computer science education, USA 335 339
31. Joy M, Griffiths N, 2004 Online Submission of Coursework–a Technological Perspective In Proceedings of the 4th IEEE International Conference on Advanced Learning Technologies Finland, 430-434
32. JPLAG. [Computer software] Retrieved November 15, 2004, from http://www.jplag.de/
33. Kay D, Scott T, Isaacson P, Reek K, 1994 Automated grading assistance for student programs In Proceedings of the 25th SIGCSE technical symposium on Computer science education USA, 381-382
34. Korhonen A, Malmi L, 2000 Algorithm simulation with automatic assessment Proceedings of 5th annual conference on Innovation and technology in computer science education, Finland 160 163
35. Lister R, Leaney J, 2003 First Year Programming: Let All the Flowers Bloom In T. Greening & R. Lister (Eds.), Computing Education 2003 Fifth Australasian Computing Education Conference (pp. 221-230)
36. Luck, M, and Joy, M. 1999. A Secure On-line Submission System. Software–Practice and Experience, 29: 721–740.
37. McCracken, M, Almstrum, V, Diaz, D, Guzdial, M, Hagan, D, Kolikant, Y, Laxer, G, Thomas, L, Utting, I, and Wilusz, T. 2001. A Multi-national, Multi-institutional Study of Assessment of Programming Skills of First-year CS Students. ACM SIGCSE Bulletin, 33 (4): 125–180.
38. MacNish, C. 2000. Java facilities for automating analysis, feedback and assessment of laboratory work. Computer Science Education, 10: 147–163.
39. McCabe, TJ. 1976. A Complexity Measure. IEEE Transactions on Software Engineering, SE-2: 308–320.
40. Mengel SA, Ulans JV, 1999 A case study of the analysis of the quality of novice students programs In Proceedings of the 12th Conference on Software Engineering Education and Training 40 49
41. Michaelson G, 1996 Automatic Analysis of Functional Program Style In Proceedings of Australian Software Engineering Conference 38 46
42. Morris D, 2003 Automatic Grading of Student's Programming Assignments: An Interactive Process and Suite of Programs In Proceedings of the 33rd ASEE/IEEE Frontiers in Education Conference S3F-1-S3F-5
43. MOSS. [Computer software] Retrieved November 15, 2004, from http://www.cs.berkeley.edu/∼aiken/moss.html
44. Olson DM, 1988 The reliability of analytic and holistic methods in rating students' computer programs In Proceedings of the 19th SIGCSE technical symposium on Computer science education USA, 293-298
45. Oman PW, Cook CR, 1990 A taxonomy for programming style In Proceedings of the 1990 ACM Annual Conference on Cooperation 244 250
46. PASS Program Analysis and Style System Retrieved November 15, 2004, from http://www3.sqi.gu.edu.au/pass/
47. Pillay, N. 2003. Developing intelligent programming tutors for novice programmers. ACM SIGCSE Bulletin, 35 (2): 78–82.
48. Prechelt, L, Malpohl, G, and Philippsen, M. 2002. Finding plagiarisms among a set of programs with JPlag. Journal of Universal Computer Science, 8: 1016–1038.
49. Purao, S, and Vaishnavi, V. 2003. Product metrics for object-oriented systems. ACM Computing Surveys, 35: 191–221.
50. Reek KA, 1989 The TRY system -or- how to avoid testing student programs In Proceedings of the 20th SIGCSE technical symposium on Computer science education USA, 112-116
51. Rees, MJ. 1982. Automatic assessment aid for pascal programs. SIGPLAN Notices, 17: 33–42.
52. Rintala M, 2002 Tutnew memory management library Retrieved November 15, 2004, from http://www.cs.tut.fi/∼bitti/tutnew/english/
53. Robins, A, Rountree, J, and Rountree, N. 2003. Learning and teaching programming: A review and discussion. Computer Science Education, 13: 137–172.
54. Ruehr, F, and Orr, G. 2002. Interactive program demonstration as a form of student program assessment. Journal of Computing in Small Colleges, 18: 65–78.
55. Saikkonen R, Malmi L, Korhonen A, 2001 Fully automatic assessment of programming exercises Proceedings of 6th annual conference on Innovation and technology in computer science education, UK 133 136
56. Schleimer S, Wilkerson D, Aiken A, 2003 Winnowing: Local algorithms for document fingerprinting Proceedings of the 2003 ACM SIGMOD international conference on Management of data, USA 76 85
57. Schorsch T, 1995 CAP: an automated self-assessment tool to check Pascal programs for syntax, logic and style errors In Proceedings of the 26th SIGCSE technical symposium on Computer science education 168 172
58. Sitthiworachart J, Joy M, 2003 Effective Peer Assessment for Learning Computer Programming Proceedings of the 8th annual conference on Innovation and technology in computer science education, UK 122 126
59. Thorburn, G, and Rowe, G. 1997. PASS: An automated system for program assessment. Computers & Education, 29: 195–206.
60. Truong N, Roe P, Bancroft P, 2003 A Web Based Environment for Learning to Program In Proceedings of the 25th Australasian Computer Science Conference Australia, 255-264
61. Truong N, Roe P, Bancroft P, 2004 Static Analysis of Students' Java Programs In Proceedings of the sixth Australian Computing Education Conference New Zealand, 317-325
62. Verco, K, and Wise, M. 1996. A comparison of automated systems for detecting suspected plagiarism. The Computer Journal, 39: 741–750.
63. Woit D, Mason D, 2003 Effectiveness of online assessment In Proceedings of the 34th SIGCSE technical symposium on Computer science education USA, 137-141
64. Xie Y, Engler D, 2002 Using Redundancies to Find Errors In Proceedings of SIGSOFT 2002/FSE-10 USA, 51-60
65. Zeller A, 2000 Making students read and review code In Proceedings of 5th annual conference on Innovation and technology in computer science education Finland, 89-92