Assistance and Feedback Mechanism in an Intelligent Tutoring System for Teaching Conversion of Natural Language into Logic

International Journal of Artificial Intelligence in Education

Volumn 27, Issue 3, 2017, Pages 475-514

  Perikos, Isidoros ^a   Grivokostopoulou, Foteini ^a   Hatzilygeroudis, Ioannis ^a  

^a UNIVERSITY OF PATRAS   (Greece)

Author keywords

Answer categorization; Error categorization; Feedback effectiveness evaluation; Feedback framework; Feedback sequencing; Learning analytics; Student assistance

Indexed keywords

COMPUTER AIDED INSTRUCTION; COMPUTER CIRCUITS; EDUCATION COMPUTING; FEEDBACK CONTROL; KNOWLEDGE REPRESENTATION; NATURAL LANGUAGE PROCESSING SYSTEMS; STUDENTS; TEACHING;

ANSWER CATEGORIZATION; AUTOMATICALLY GENERATED; EFFECTIVENESS EVALUATION; ERROR CATEGORIZATIONS; INTELLIGENT TUTORING SYSTEM; KNOWLEDGE REPRESENTATION AND REASONING; LEARNING ANALYTICS; NATURAL LANGUAGE GENERATION;

EDUCATION;

EID: 85025070040     PISSN: 15604292     EISSN: 15604306     Source Type: Journal    
DOI: 10.1007/s40593-017-0139-y     Document Type: Article

References (70)

1. Aleven, V., & Koedinger, K. R. (2000). Limitations of student control: Do students know when they need help?. In Proceedings of the International Conference on Intelligent Tutoring Systems (pp. 292–303). Springer Berlin Heidelberg.
2. Aleven, V., Stahl, E., Schworm, S., Fischer, F., & Wallace, R. (2003). Help seeking and help design in interactive learning environments. Review of Educational Research, 73(3), 277–320.
3. Aleven, V., Mclaren, B., Roll, I., & Koedinger, K. (2006a). Toward meta-cognitive tutoring: a model of help seeking with a cognitive tutor. International Journal of Artificial Intelligence in Education, 16(2), 101–128.
4. Aleven, V., McLaren, B. M., Sewall, J., Koedinger, K. R. (2006). The cognitive tutor authoring tools (CTAT): preliminary evaluation of efficiency gains. In Proceedings of the International Conference on Intelligent Tutoring Systems (pp. 61–70). Springer Berlin Heidelberg.
5. Aleven, V., Mclaren, B. M., Sewall, J., & Koedinger, K. R. (2009). A new paradigm for intelligent tutoring systems: example-tracing tutors. International Journal of Artificial Intelligence in Education, 19(2), 105–154.
6. Aleven, V., Roll, I., McLaren, B. M., & Koedinger, K. R. (2016). Help helps, but only so much: research on help seeking with intelligent tutoring systems. International Journal of Artificial Intelligence in Education, 26(1), 205–223.
7. Alonso, J. A., Aranda, G. A., Martn–Mateos, F. J. (2007). KRRT: Knowledge representation and reasoning tutor system. In Computer Aided Systems Theory–EUROCAST 2007 (pp. 400–407). Springer Berlin Heidelberg.
8. Anderson, J. R., Corbett, A. T., Koedinger, K. R., & Pelletier, R. (1995). Cognitive tutors: lessons learned. The Journal of the Learning Sciences, 4(2), 167–207.
9. Barker-Plummer, D., Cox, R., Dale, R., Etchemendy, J. (2008). An empirical study of errors in translating natural language into logic. In Proceedings of the 30th Annual Meeting of the Cognitive Science Society/CogSci (pp. 505–510).
10. Barker-Plummer, D., Cox, R., Dale, R. (2009). Dimensions of difficulty in translating natural language into first order logic. International Working Group on Educational Data Mining.
11. Barker-Plummer, D., Cox, R., Dale, R. (2011). Student translations of natural language into logic: The Grade Grinder corpus release 1.0. In Proceedings of the 4th international conference on educational data mining (pp. 51–60).
12. Barker-Plummer, D., Dale, R., Cox, R. (2012). Using edit distance to analyse errors in a natural language to logic translation corpus. In Proceedings of the International Conference on Educational Data Mining, 134.
13. Berlanga, A. J., Van Rosmalen, P., Boshuizen, H. P., & Sloep, P. B. (2012). Exploring formative feedback on textual assignments with the help of automatically created visual representations. Journal of Computer Assisted Learning, 28(2), 146–160.
14. Bitchener, J. (2008). Evidence in support of written corrective feedback. Journal of Second Language Writing, 17(2), 102–118.
15. Black, P., & Wiliam, D. (1998). Assessment and classroom learning. Assessment in Education, 5(1), 7–74.
16. Brachman, R. J., Levesque, H. J. (2004) Knowledge representation and reasoning. Morgan Kaufmann Publishers, Massachusetts, US, 9, 9.
17. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). New Jersey: Hillsdale.
18. Corbett, A. T., & Anderson, J. R. (2001). Locus of feedback control in computer-based tutoring: Impact on learning rate, achievement and attitudes. In Proceedings of the SIGCHI conference on Human factors in computing systems (pp. 245–252). ACM.
19. Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16(3), 297–334.
20. De Marneffe, M. C., MacCartney, B., & Manning, C. D. (2006). Generating typed dependency parses from phrase structure parses. Proceedings of LREC, 6, 449–454.
21. Dempsey, J. V., & Wager, S. U. (1988). A taxonomy for the timing of feedback in computer-based instruction. Educational Technology, 28(10), 20–25.
22. Dempsey, J. V., Driscoll, M. P., Swindell, L. K. (1993). Text-based feedback. Interactive instruction and feedback, 21–54.
23. Fiedler, A., & Tsovaltzi, D. (2003). Automating hinting in an intelligent tutorial dialog system for mathematics. Knowledge Representation and Automated Reasoning for E-Learning Systems, 23.
24. Finkel, J. R., Grenager, T., Manning, C. (2005). Incorporating non-local information into information extraction systems by gibbs sampling. In Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics (pp. 363–370). Association for Computational Linguistics.
25. Genesereth, M. R., & Nilsson, N. J. (1987). Logical foundations of artificial Intelligence. Morgan Kaufmann.
26. Gerdes, A., Heeren, B., Jeuring, J., van Binsbergen, L. T. (2016). Ask-Elle: an adaptable programming tutor for Haskell giving automated feedback. International Journal of Artificial Intelligence in Education, 1–36.
27. Gheorghiu, R., & VanLehn, K. (2008). XTutor: an intelligent tutor system for science and math based on excel. In Proceedings of the International Conference on Intelligent Tutoring Systems (pp. 749–751). Springer Berlin Heidelberg.
28. Goldin, I. M., Koedinger, K. R., Aleven, V. (2013). Hints: You can’t have just one. In S. K. D’Mello, R. A. Calvo, A. Olney (Eds.), In Proceedings of the 6th International Conference on Educational Data Mining (EDM 2013) (pp. 232–235).
29. Gouli, E., Gogoulou, A., Papanikolaou, K. A., Grigoriadou, M. (2006). An adaptive feedback framework to support reflection, guiding and tutoring. Advances in web-based education: Personalized learning environments, 178–202.
30. Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77(1), 81–112.
31. Hatzilygeroudis, I. (2007). Teaching NL to FOL and FOL to CF conversions. In FLAIRS Conference (pp. 309–314).
32. Hatzilygeroudis, I., & Perikos, I. (2009). A web-based interactive system for learning NL to FOL conversion. In New Directions in Intelligent Interactive Multimedia Systems and Services-2 (pp. 297–307). Springer Berlin Heidelberg.
33. Havnes, A., Smith, K., Dysthe, O., & Ludvigsen, K. (2012). Formative assessment and feedback: making learning visible. Studies in Educational Evaluation, 38(1), 21–27.
34. Hirashima, T., Horiguchi, T., Kashihara, A., & Toyoda, J. (1998). Error-based simulation for error-visualization and its management. International Journal of Artificial Intelligence in Education, 9(1–2), 17–31.
35. Koedinger, K. R., & Aleven, V. (2007). Exploring the assistance dilemma in experiments with cognitive tutors. Educational Psychology Review, 19(3), 239–264.
36. Kulhavy, R. W., & Stock, W. A. (1989). Feedback in written instruction: the place of response certitude. Educational Psychology Review, 1(4), 279–308.
37. Lopez, L. (2009). Effects of delayed and immediate feedback in the computer-based testing environment. ProQuest.
38. Maestro-Prieto, J. A., Simon-Hurtado, A. (2015). Learner-Adaptive Pedagogical Model in SIAL, an Open-Ended Intelligent Tutoring System for First Order Logic. In Proceedings of the International Conference on Artificial Intelligence in Education (pp. 702–705). Springer International Publishing.
39. Mandernach, B. J. (2005). Relative effectiveness of computer-based and human feedback for enhancing student learning. The Journal of Educators Online, 2(1).
40. Mathan, S., & Koedinger, K. R. (2003). Recasting the feedback debate: Benefits of tutoring error detection and correction skills. In Proceedings of the International Conference on Artificial Intelligence in Education (pp. 13–20).
41. Mathan, S. A., & Koedinger, K. R. (2005). Fostering the intelligent novice: learning from errors with metacognitive tutoring. Educational Psychologist, 40(4), 257–265.
42. McKendree, J. (1990). Effective feedback content for tutoring complex skills. Human-Computer Interaction, 5(4), 381–413.
43. Mitrovic, A., Martin, B., & Suraweera, P. (2007). Intelligent tutors for all: the constraint-based approach. IEEE Intelligent Systems, 4, 38–45.
44. Mitrovic, A., Martin, B., Suraweera, P., Zakharov, K., Milik, N., Holland, J., & McGuigan, N. (2009). ASPIRE: an authoring system and deployment environment for constraint-based tutors. International Journal of Artificial Intelligence in Education, 19(2), 155–188.
45. Mitrovic, A., Ohlsson, S., & Barrow, D. K. (2013). The effect of positive feedback in a constraint-based intelligent tutoring system. Computers & Education, 60(1), 264–272.
46. Moreno, A., & Budesca, N. (2000). Mathematical logic tutor-propositional calculus. In Proceedings of the First International Congress on Tools for Teaching Logic (pp. 99–106).
47. Moreno, R., & Mayer, R. (2007). Interactive multimodal learning environments. Educational Psychology Review, 19(3), 309–326.
48. Mory, E. H. (2004). Feedback research revisited. Handbook of research on educational communications and technology, 2, 745–783.
49. Muñoz-Merino, P. J., Kloos, C. D., & Muñoz-Organero, M. (2011). Enhancement of student learning through the use of a hinting computer e-learning system and comparison with human teachers. IEEE Transactions on Education, 54(1), 164–167.
50. Narciss, S. (2008). Feedback strategies for interactive learning tasks. Handbook of research on educational communications and technology, 3, 125–144.
51. Narciss, S., Sosnovsky, S., Andres, E. (2014a). Adapting tutoring feedback strategies to motivation. In Open Learning and Teaching in Educational Communities (pp. 288–301). Springer International Publishing.
52. Narciss, S., Sosnovsky, S., Schnaubert, L., Andrès, E., Eichelmann, A., Goguadze, G., & Melis, E. (2014b). Exploring feedback and student characteristics relevant for personalizing feedback strategies. Computers & Education, 71, 56–76.
53. Perikos, I., Grivokostopoulou, F., Hatzilygeroudis, I., Kovas, K. (2011). Difficulty estimator for converting natural language into first order logic. In Proceedings of the International Conference on Intelligent Decision Technologies (pp. 135–144). Springer Berlin Heidelberg.
54. Perikos, I., Grivokostopoulou, F., Hatzilygeroudis, I. (2012). Automatic marking of NL to FOL conversions. In Proceedings of 15th IASTED International Conference on Computers and Advanced Technology in Education (CATE), Napoli, Italy (pp. 227–233).
55. Perikos, I., Grivokostopoulou, F., Kovas, K., & Hatzilygeroudis, I. (2016). Automatic estimation of exercises' difficulty levels in a tutoring system for teaching the conversion of natural language into first‐order logic. Expert Systems, 33(6), 569–580.
56. Roll, I., Aleven, V., McLaren, B. M., & Koedinger, K. R. (2011). Improving students’ help-seeking skills using metacognitive feedback in an intelligent tutoring system. Learning and Instruction, 21(2), 267–280.
57. Roll, I., Baker, R. S. D., Aleven, V., & Koedinger, K. R. (2014). On the benefits of seeking (and avoiding) help in online problem-solving environments. Journal of the Learning Sciences, 23(4), 537–560.
58. Russell, S., & Norvig, P. (2010). Artificial intelligence: a modern approach (3rd ed.). Upper Saddle River: Prentice Hall.
59. Shute, V. J. (2008). Focus on formative feedback. Review of Educational Research, 78(1), 153–189.
60. Sieg, W. (2007). The AProS project: strategic thinking & computational logic. Logic Journal of IGPL, 15(4), 359–368.
61. van der Kleij, F. M., Eggen, T. J., Timmers, C. F., & Veldkamp, B. P. (2012). Effects of feedback in a computer-based assessment for learning. Computers & Education, 58(1), 263–272.
62. VanLehn, K. (2006). The behavior of tutoring systems. International Journal of Artificial Intelligence in Education, 16(3), 227–265.
63. VanLehn, K. (2011). The relative effectiveness of human tutoring, intelligent tutoring systems, and other tutoring systems. Educational Psychologist, 46(4), 197–221.
64. VanLehn, K., Lynch, C., Schulze, K., Shapiro, J. A., Shelby, R., Taylor, L., & Wintersgill, M. (2005). The Andes physics tutoring system: lessons learned. International Journal of Artificial Intelligence in Education, 15(3), 147–204.
65. Vasilyeva, E., Pechenizkiy, M., De Bra, P. (2008). Adaptation of elaborated feedback in e-learning. In Adaptive hypermedia and adaptive web-based systems (pp. 235–244). Springer Berlin Heidelberg.
66. Wager, W., & Wager, S. (1985). Presenting questions, processing responses, and providing feedback in CAI. Journal of Instructional Development, 8(4), 2–8.
67. Wang, S. L., & Wu, P. Y. (2008). The role of feedback and self-efficacy on web-based learning: the social cognitive perspective. Computers & Education, 51(4), 1589–1598.
68. Yacef, K. (2002). Intelligent teaching assistant systems. In Computers in Education, 2002. Proceedings International Conference on IEEE (pp. 136–140).
69. Yacef, K. (2005). The Logic-ITA in the classroom: a medium scale experiment. International Journal of Artificial Intelligence in Education, 15(1), 41–62.
70. Zhou, Y., Freedman, R., Glass, M., Michael, J. A., Rovick, A. A., Evens, M. W. (1999). Delivering Hints in a Dialogue-Based Intelligent Tutoring System. In AAAI/IAAI (pp. 128–134).