Automated feedback generation for introductory programming assignments

Proceedings of the ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI)

Volumn , Issue , 2013, Pages 15-26

  Singh, Rishabh ^a   Gulwani, Sumit ^b   Solar Lezama, Armando ^a  

^a MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

^b MICROSOFT RESEARCH   (United States)

Author keywords

Automated Grading; Computer Aided Education; Program Synthesis

Indexed keywords

AUTOMATED FEEDBACK; AUTOMATED GRADING; COMPUTER-AIDED EDUCATION; INTRODUCTION TO PROGRAMMING; INTRODUCTORY PROGRAMMING; PROGRAM SYNTHESIS; REFERENCE IMPLEMENTATION; TRANSLATION STRATEGIES;

COMPUTER AIDED INSTRUCTION; COMPUTER PROGRAMMING LANGUAGES; EDUCATION COMPUTING; GRADING; STUDENTS;

PROGRAM TRANSLATORS;

EID: 84883074046     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2462156.2462195     Document Type: Conference Paper

References (44)

1. A. Adam and J.-P. H. Laurent. LAURA, A System to Debug Student Programs. Artif. Intell., 15(1-2):75-122, 1980.
2. U. Ahmed, S. Gulwani, and A. Karkare. Automatically generating problems and solutions for natural deduction. In IJCAI, 2013.
3. R. Alur, L. D'Antoni, S. Gulwani, D. Kini, and M. Viswanathan. Automated grading of dfa constructions. In IJCAI, 2013.
4. E. Andersen, S. Gulwani, and Z. Popovic. A trace-based framework for analyzing and synthesizing educational progressions. In CHI, 2013.
5. A. Arcuri. On the automation of fixing software bugs. In ICSE Companion, 2008.
6. T. Ball, M. Naik, and S. K. Rajamani. From symptom to cause: localizing errors in counterexample traces. In POPL, 2003.
7. M. S. Bernstein, G. Little, R. C. Miller, B. Hartmann, M. S. Ackerman, D. R. Karger, D. Crowell, and K. Panovich. Soylent: a word processor with a crowd inside. In UIST, 2010.
8. R. Bodík, S. Chandra, J. Galenson, D. Kimelman, N. Tung, S. Barman, and C. Rodarmor. Programming with angelic nondeterminism. In POPL, 2010.
9. C. Daly. Roboprof and an introductory computer programming course. ITiCSE, 1999.
10. V. Debroy and W. Wong. Using mutation to automatically suggest fixes for faulty programs. In ICST, 2010.
11. C. Douce, D. Livingstone, and J. Orwell. Automatic test-based assessment of programming: A review. J. Educ. Resour. Comput., 5(3), Sept. 2005.
12. P. Ertmer, J. Richardson, B. Belland, D. Camin, P. Connolly, G. Coulthard, K. Lei, and C. Mong. Using peer feedback to enhance the quality of student online postings: An exploratory study. Journal of Computer-Mediated Communication, 12(2):412-433, 2007.
13. R. G. Farrell, J. R. Anderson, and B. J. Reiser. An interactive computer-based tutor for lisp. In AAAI, 1984.
14. S. Forrest, T. Nguyen, W. Weimer, and C. L. Goues. A genetic programming approach to automated software repair. In GECCO, 2009.
15. A. Groce, S. Chaki, D. Kroening, and O. Strichman. Error explanation with distance metrics. STTT, 8(3):229-247, 2006.
16. S. Gulwani, S. Srivastava, and R. Venkatesan. Program analysis as constraint solving. In PLDI, 2008.
17. S. Gulwani, V. A. Korthikanti, and A. Tiwari. Synthesizing geometry constructions. In PLDI, 2011.
18. S. Gulwani, W. R. Harris, and R. Singh. Spreadsheet data manipulation using examples. In CACM, 2012.
19. P. Hawkins, A. Aiken, K. Fisher, M. C. Rinard, and M. Sagiv. Data representation synthesis. In PLDI, 2011.
20. P. Hawkins, A. Aiken, K. Fisher, M. C. Rinard, and M. Sagiv. Concurrent data representation synthesis. In PLDI, 2012.
21. J. B. Hext and J. W. Winings. An automatic grading scheme for simple programming exercises. Commun. ACM, 12(5), May 1969.
22. D. Jackson and M. Usher. Grading student programs using assyst. SIGCSE, 1997.
23. B. Jobstmann, A. Griesmayer, and R. Bloem. Program repair as a game. In CAV, pages 226-238, 2005.
24. W. L. Johnson and E. Soloway. Proust: Knowledge-based program understanding. IEEE Trans. Software Eng., 11(3):267-275, 1985.
25. M. Jose and R. Majumdar. Cause clue clauses: error localization using maximum satisfiability. In PLDI, 2011.
26. U. Junker. QUICKXPLAIN: preferred explanations and relaxations for over-constrained problems. In AAAI, 2004.
27. R. Könighofer and R. P. Bloem. Automated error localization and correction for imperative programs. In FMCAD, 2011.
28. C. Kulkarni and S. R. Klemmer. Learning design wisdom by augmenting physical studio critique with online self-assessment. Technical report, Stanford University, 2012.
29. V. Kuncak, M. Mayer, R. Piskac, and P. Suter. Complete functional synthesis. PLDI, 2010.
30. G. Little, L. B. Chilton, M. Goldman, and R. C. Miller. Turkit: human computation algorithms on mechanical turk. In UIST, 2010.
31. W. R. Murray. Automatic program debugging for intelligent tutoring systems. Computational Intelligence, 3:1-16, 1987.
32. W. Sack, E. Soloway, and P. Weingrad. From PROUST to CHIRON: Its design as iterative engineering: Intermediate results are important! In In J.H. Larkin and R.W. Chabay (Eds.), Computer-Assisted Instruction and Intelligent Tutoring Systems: Shared Goals and Complementary Approaches., pages 239-274, 1992.
33. R. Singh and S. Gulwani. Learning semantic string transformations from examples. PVLDB, 5, 2012.
34. R. Singh and A. Solar-Lezama. Synthesizing data structure manipulations from storyboards. In SIGSOFT FSE, 2011.
35. R. Singh, S. Gulwani, and S. K. Rajamani. Automatically generating algebra problems. In AAAI, 2012.
36. R. Singh, S. Gulwani, and A. Solar-Lezama. Automated semantic grading of programs. CoRR, abs/1204.1751, 2012.
37. A. Solar-Lezama. Program Synthesis By Sketching. PhD thesis, EECS Dept., UC Berkeley, 2008.
38. A. Solar-Lezama, R. Rabbah, R. Bodik, and K. Ebcioglu. Programming by sketching for bit-streaming programs. In PLDI, 2005.
39. E. Soloway, B. P. Woolf, E. Rubin, and P. Barth. Meno-II: An Intelligent Tutoring System for Novice Programmers. In IJCAI, 1981.
40. S. Srivastava, S. Gulwani, and J. Foster. From program verification to program synthesis. POPL, 2010.
41. S. S. Staber, B. Jobstmann, and R. P. Bloem. Finding and fixing faults. In Correct Hardware Design and Verification Methods, Lecture notes in computer science, pages 35 - 49, 2005.
42. M. Vechev, E. Yahav, and G. Yorsh. Abstraction-guided synthesis of synchronization. In POPL, 2010.
43. D. S. Weld, E. Adar, L. Chilton, R. Hoffmann, and E. Horvitz. Personalized online education - a crowdsourcing challenge. In Workshops at the Twenty-Sixth AAAI Conference on Artificial Intelligence, 2012.
44. A. Zeller and R. Hildebrandt. Simplifying and isolating failure-inducing input. IEEE Transactions on Software Engineering, 28:183-200, 2002.