Stupid Tutoring Systems, Intelligent Humans

International Journal of Artificial Intelligence in Education

Volumn 26, Issue 2, 2016, Pages 600-614

  Baker, Ryan S ^a  

^a Och Spine at New York Presbyterian Hospitals   (United States)

Author keywords

Automated adaptation; Decision making; Intelligence amplification; Intelligent tutoring system

Indexed keywords

AMPLIFICATION; ARTIFICIAL INTELLIGENCE; COMPUTER AIDED INSTRUCTION; DATA MINING; DECISION MAKING; EDUCATION COMPUTING;

AUTOMATED ADAPTATION; COMPLEX LEARNING; EDUCATIONAL DATA MINING; HUMAN DECISION MAKING; INTELLIGENT TUTORING SYSTEM; ONLINE LEARNING; TUTORING SYSTEM;

TEACHING;

EID: 84963584391     PISSN: 15604292     EISSN: 15604306     Source Type: Journal    
DOI: 10.1007/s40593-016-0105-0     Document Type: Article

References (101)

1. Aleven, V.; Sewall, J.; Popescu, O.; Xhakaj, F.; Chand, D.; Baker, R.; Wang, Y.; Siemens, G.; Rosé, C.; & Gasevic, D. (2015). The Beginning of a beautiful friendship? Intelligent tutoring systems and MOOCs. Proceedings of the 17th International Conference on Artificial Intelligence in Education, 525-528.
2. Anderson, J. R.; Corbett, A. T.; Koedinger, K. R.; & Pelletier, R. (1995). Cognitive tutors: lessons learned. Journal of the Learning Sciences, 4, 167-207.
3. Arnold, K. E.; Pistilli, M. D. (2012) Course Signals at Purdue: Using learning analytics to increase student success. Proceedings of the 2nd International Conference on Learning Analytics and Knowledge. New York, NY: ACM, pp. 267-270.
4. Arroyo, I.; Ferguson, K.; Johns, J.; Dragon, T.; Meheranian, H.; Fisher, D.; Barto, A.; Mahadevan, S.; & Woolf, B. P. (2007). Repairing disengagement with non-invasive interventions. Proceedings of the International Conference on Artificial Intelligence in Education, 195-202.
5. Arroyo, I.; Woolf, B. P.; Burelson, W.; Muldner, K.; Rai, D.; & Tai, M. (2014). A multimedia adaptive tutoring system for mathematics that addresses cognition, metacognition and affect. International Journal of Artificial Intelligence in Education, 24(4), 387-426.
6. Baker, R. S. J. d. (2010). Data mining for education. In B. McGaw, P. Peterson, & E. Baker (Eds.) International Encyclopedia of Education 3rd edn, (pp. 112-118). Oxford, UK: Elsevier.
7. Baker, R. S. (2014). Big Data And Education. New York, NY: Teachers College, Columbia University.
8. Baker, R.S. (2016) Stupid Tutoring Systems, Intelligent humans. Manuscript under review.
9. Baker, R.; & Siemens, G. (2014). Educational data mining and learning analytics. In K. Sawyer (Ed.) Cambridge Handbook of the Learning Sciences 2nd edn, (pp. 253-274).
10. Baker, R. S. J. d.; & Yacef, K. (2009). The state of educational data mining in 2009: A review and future visions. Journal of Educational Data Mining, 1(1), 3-17.
11. Baker, R. S. J. d.; Corbett, A. T.; Koedinger, K. R.; Evenson, S. E.; Roll, I.; Wagner, A. Z.; Naim, M.; Raspat, J.; Baker, D. J.; & Beck, J. (2006). Adapting to when students game an intelligent tutoring system. Proceedings of the 8th International Conference on Intelligent Tutoring Systems, 392-401.
12. Barber, R.; Sharkey, M. (2012) Course correction: using analytics to predict course success. Proceedings of the 2nd International Conference on Learning Analytics and Knowledge, New York, NY: ACM, 259-262.
13. Beck, J.E. (1997) Modeling the Student with Reinforcement Learning. Proceedings of the Machine learning for User Modeling Workshop at the Sixth International Conference on User Modeling.
14. Beck, J.E.; Mostow, J. (2008) How who should practice: Using learning decomposition to evaluate the efficacy of different types of practice for different types of students. Proceedings of the 9th International Conference on Intelligent Tutoring Systems, 5091, 353-362.
15. Beck, J.E.; Woolf, B.P.; Beal, C.R. (2000) ADVISOR: A machine learning architecture for intelligent tutor construction. Proceedings of the 7th National Conference on Artificial Intelligence,New York, NY: ACM, 552-557.
16. Bergman, P. (under review). Parent-child information frictions and human capital investment: evidence from a field experiment. Manuscript under review. Working paper retrieved online 2/10/2016 from http://www.columbia.edu/~psb2101/BergmanSubmission.pdf.
17. Brabham, D. C. (2008). Crowdsourcing as a model for problem solving: An introduction and cases. Convergence: The International Journal of Research into New Media Technologies, 14(1), 75-90.
18. Bransford, J. D.; Brown, A. L.; & Cocking, R. R. (1999). How people learn: Brain, mind, experience, and school. Washington, DC: National Academy Press.
19. Broderick, Z.; O'Connor, C.; Mulcahy, C.; Heffernan, C.; & Heffernan, N. (2010). Increasing parent engagement in student learning using an intelligent tutoring system. Journal of Interactive Learning Research, 22(4), 523-550.
20. Brooks, F. P. (1975). The mythical man-month. Boston, MA: Addison-Wesley.
21. Bull, S.; Nghiem, T. (2002) Helping learners to understand themselves with a learner model open to students, peers and instructors. Proceedings of the Workshop on Individual and Group Modelling Methods that Help Learners Understand Themselves, 5-13. Biarritz, France.
22. Campbell, D. T. (1976). Assessing the impact of planned social change. Hanover, NH: Dartmouth College.
23. Carbonell, J. (1970). AI in CAI: An artificial-intelligence approach to computer-assisted instruction. IEEE Transactions on Man-Machine Systems, 11(4), 190-202.
24. Chi, M.; VanLehn, K.; Litman, D. (2010) Do micro-level tutorial decisions matter: Applying reinforcement learning to induce pedagogical tutoring tactics. Proceedings of the 10th International Conference on Intelligent Tutoring Systems, 184-193.
25. Clark, R. C.; & Mayer, R. E. (2003). E-learning and the science of instruction. San Francisco: Jossey-Bass.
26. Corbett, A. T.; & Anderson, J. R. (1995). Knowledge tracing: Modeling the acquisition of procedural knowledge. User modeling and user-adapted interaction, 4(4), 253-278.
27. Corbett, A. T.; & Bhatnagar, A. (1997). Student modeling in the ACT programming tutor: adjusting a procedural learning model with declarative knowledge. User Modeling: Proceedings of the Sixth International Conference, UM97, 243-254.
28. Corbett, A. T.; Koedinger, K. R.; & Hadley, W. (2001). Cognitive tutors: from the research classroom to all classrooms. In P. S. Goodman (Ed.), Technology enhanced learning: Opportunities for change (pp. 235-263). Mahwah, NJ: Lawrence Erlbaum Associates, Inc.
29. Craig, S. D.; Hu, X.; Graesser, A. C.; Bargagliotti, A. E.; Sterbinsky, A.; Cheney, K. R.; & Okwumabua, T. (2013). The impact of a technology-based mathematics after-school program using ALEKS on student's knowledge and behaviors. Computers & Education, 68, 495-504.
30. D'Mello, S.; Lehman, B.; Sullins, J.; Daigle, R.; Combs, R.; Vogt, K. (2010) A time for emoting: when affect-sensitivity is and isn't effective at promoting deep learning. Proceedings of the 10th International Conference on Intelligent Tutoring Systems, 6094, 245-254.
31. Darling-Hammond, L. (2000) Teacher Quality and Student Achievement. Education Policy Analysis Archives. Available online at http://epaa.asu.edu/ojs/index.php/epaa/article/view/392
32. Dekker, G. W.; Pechenizkiy, M.; & Vleeshouwers, J. M. (2009). Predicting student drop out: A case study. Proceedings of the 2nd International Conference on Educational Data Mining (EDM'09), 41-50.
33. Del Soldato, T.; & du Boulay, B. (1995). Implementation of motivational tactics in tutoring systems. Journal of Artificial Intelligence in Education, 6(4), 337-378.
34. Diaz, G.; Garcia Loro, F.; Castro, M.; Tawfik, M.; Sancristobal, E.; Monteso, S. (2013) Remote electronics lab within a MOOC: Design and preliminary results. Proceedings of Experiment@ International Conference, Redwood, CA: Deepdyve, 89-93.
35. D'Mello, S. K.; & Graesser, A. C. (2012). AutoTutor and affective AutoTutor: learning by talking with cognitively and emotionally intelligent computers that talk back. ACM Transactions on Interactive Intelligent Systems, 2(4), 1-38.
36. Falmagne, J.; Albert, D.; Doble, C.; Eppstein, D.; & Hu, X. (2013). Knowledge spaces: Applications in education. Berlin-Heidelberg: Springer.
37. Farber, B.; & Miller, J. (1981). Teacher burnout: A psycho-educational perspective. Teachers College Record, 83(2), 235-243.
38. Feng, M.; & Heffernan, N. T. (2005). Informing teachers live about student learning: reporting in the assistment system. Technology, Instruction, Cognition and Learning, 3, 1-14.
39. Freund, Y. (2013) Artificial intelligence vs intelligence amplification. California Institute for Telecommunications and Information Technology. https://www.youtube.com/watch?v=sEGdszE86bY
40. Goldstein, I. J. (1979). The genetic graph: a representation for the evolution of procedural knowledge. International Journal of Man-Machine Studies, 11(1), 51-77.
41. Greeno, J. G. (1997). On claims that answer the wrong question. Educational Researcher, 26(1), 5-17.
42. Hawn, A. (2015). The bridge report: bringing learning analytics to low-income, urban schools. Proceedings of the Fifth International Conference on Learning Analytics And Knowledge. Washington, DC: Association for Computing Machinery, 410-411.
43. Heffernan, N. T.; & Heffernan, C. L. (2014). The ASSISTments ecosystem: building a platform that brings scientists and teachers together for minimally invasive research on human learning and teaching. International Journal of Artificial Intelligence in Education, 24(4), 470-497.
44. Heffernan, N.T.; Koedinger, K.R. (2002) An intelligent tutoring system incorporating a model of an experienced human tutor. Proceedings of the 6th International Conference on Intelligent Tutoring Systems, 2363, 596-608.
45. Jayaprakash, S. M.; Moody, E. W.; Lauria, E. J. M.; Regan, J. R.; & Baron, J. D. (2014). Early alert of academically At-risk students: An open source analytics initiative. Journal of Learning Analytics, 1(1), 6-47.
46. Jeong, H.; Biswas, G. (2008) Mining Student Behavior Models in Learning-by-Teaching Environments. Proceedings of the 1st International Conference on Educational Data Mining, Worcester, MA: International Educational Data Mining Society, 127-136.
47. Joksimović, S.; Kovanović, V.; Jovanović, J.; Zouaq, A.; Gašević, D.; & Hatala, M. (2015). What do cMOOC participants talk about in Social Media? A Topic Analysis of Discourse in a cMOOC. Proceedings of the 5th International Learning Analytics and Knowledge (LAK) Conference, New York, NY: ACM, 156-165
48. Khachatryan, G.; Romashov, A.; Khachatryan, A.; Gaudino, S.; Khachatryan, J.; Guarian, K.; & Yufa, N. (2014). Reasoning mind Genie 2: An Intelligent learning system as a vehicle for international transfer of instructional methods in mathematics. International Journal of Artificial Intelligence in Education, 24(3), 333-382.
49. Koedinger, K. R.; McLaughlin, E. A.; & Heffernan, N. T. (2010). A quasi-experimental evaluation of an on-line formative assessment and tutoring system. Journal of Educational Computing Research, 43(4), 489-510.
50. Koedinger, K. R.; Corbett, A. T.; & Perfetti, C. (2012a). The knowledge-learning-instruction framework: bridging the science-practice chasm to enhance robust student learning. Cognitive Science, 36, 757-798.
51. Koedinger, K. R.; McLaughlin, E. A.; & Stamper, J. C. (2012b). Automated student model improvement. Proceedings of the 5th International Conference on Educational Data Mining, 17-24.
52. Koedinger, K.R.; Stamper, J.; McLaughlin, E.; Nixon, T. (2013) Using Data-Driven Discovery of Better Student Models to Improve Student Learning. Proceedings of the International Conference on Artificial Intelligence and Education, 7926, 421-430.
53. Kovačić, Z. (2010). Early Prediction of Student Success: Mining Students Enrolment Data. Proceedings of Informing Science & IT Education Conference (InSITE), 647-665.
54. Kulik, C. C.; & Kulik, J. A. (1991). Effectiveness of computer-based instruction: an updated analysis. Computers in Human Behavior, 7, 75-95.
55. Leelawong, K.; & Biswas, G. (2008). Designing learning by teaching agents: the Betty's brain system. International Journal of Artificial Intelligence in Education, 18(3), 181-208.
56. Lepper, M. R.; Woolverton, M.; Mumme, D. L.; & Gurtner, J. (1993). Motivational techniques of expert human tutors: lessons for the design of computer-based tutors. Computers as Cognitive Tools, 1993, 75-105.
57. Matsuda, N.; Keiser, V.; Raizada, R.; Tu, A.; Stylianides, G.; Cohen, W. (2010) Learning by Teaching SimStudent: Technical Accomplishments and an Initial Use with Students. Proceedings of the 10th International Conference on Intelligent Tutoring Systems, 6094, 317-326.
58. Maull, K. E.; Saldivar, M. G.; & Sumner, T. (2011). Online curriculum planning behavior of teachers. Proceedings of the 3rd International Conference on Educational Data Mining, 121-130.
59. McArthur, D.; Stasz, C.; & Zmuidzinas, M. (1990). Tutoring techniques in algebra. Cognition and Instruction, 7(3), 197-244.
60. McAuley, A.; Stewart, B.; Siemens, G.; Cormier, D. (2010) The MOOC model for digital practice. Available online at http://www.davecormier.com/edblog/wp-content/uploads/MOOC-Final.pdf
61. McCaffrey, D. F.; Lockwood, J. R.; Koretz, D. M.; & Hamilton, L. S. (2003). Evaluating value-added models for teacher accountability. Santa Monica, CA: RAND Corporation.
62. Merrill, D. C.; Reiser, B. J.; Ranney, M.; & Trafton, J. G. (1992). Effective tutoring techniques: A comparison of human tutors and intelligent tutoring systems. The Journal of the Learning Sciences, 2(3), 277-305.
63. Miller, W. L.; Baker, R.; Labrum, M.; Petsche, K.; Liu, Y-H.; Wagner, A. (2015). Automated detection of proactive remediation by teachers in reasoning mind classrooms. Proceedings of the 5th International Learning Analytics and Knowledge Conference, 290-294.
64. Ming, N.C.; Ming, V.L. (2012) Automated Predictive Assessment from Unstructured Student Writing. Proceedings of the 1st international Conference on Data Analytics.
65. Mitrovic, A.; & Ohlsson, S. (1999). Evaluation of a constraint-based tutor for a database language. International Journal of Artificial Intelligence in Education, 10, 238-256.
66. Mostow, J. (2008). Experience from a Reading tutor that listens: evaluation purposes, excuses, and methods. In C. K. Kinzer, & L. Verhoeven (Eds.), Interactive Literacy Education: Facilitating Literacy Environments Through Technology (pp. 117-148). New York: Lawrence Erlbaum Associates, Taylor & Francis Group.
67. Mulqueeny, K.; Kostyuk, V.; Baker, R. S.; & Ocumpaugh, J. (2015). Incorporating effective e-learning principles to improve student engagement in middle-school mathematics. International Journal of STEM Education, 2(15).
68. Murray, R.C.; VanLehn, K. (2005) Effects of Dissuading Unnecessary Help Requests While Providing Proactive Help. Proceedings of the International Conference on Artificial Intelligence in Education, Amsterdam, Netherlands: IOS Press, 887-889
69. Norvig, P. (2012) Google's Peter Norvig on Online Education. Public lecture given at Stanford University. http://www.youtube.com/watch?v=1ylzPGD3jRY
70. Nye, B. D.; Graesser, A. C.; & Hu, X. (2014). AutoTutor and family: A review of 17 years of natural language tutoring. International Journal of Artificial Intelligence in Education, 24(4), 427-469.
71. O'Shea, T. (1982). A self-improving quadratic tutor. In D. Sleeman, & J. S. Brown (Eds.), Intelligent Tutoring Systems (pp. 309-336). New York: Academic Press.
72. Ostashewski, N. (2013) Building for massive scalability: the production case of an astronomy MOOC. Proceedings of the World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education, Chesapeake, VA: AACE
73. Ostrow, K. S.; & Heffernan, N. T. (2014). Testing the multimedia principled in the real world: A comparison of video vs. In Text feedback in Authentic Middle School math assignments. Proceedings of the 7th International Conference on Educational Data Mining (pp. 296-299).
74. Ostrow, K. S.; Schultz, S. E.; & Arroyo, I. (2014). Promoting Growth Mindset Within Intelligent Tutoring Systems. In CEUR-WS (1183), Gutierrez-Santos, S.; & Santos, O.C. (eds.) EDM 2014 Extended Proceedings, 88-93.
75. Pane, J. F.; Griffin, B. A.; McCaffrey, D. F.; & Karam, R. (2014). Effectiveness of cognitive tutor algebra I at scale. Educational Evaluation and Policy Analysis, 36(2), 127-144.
76. Papousek, J.; Pelanek, R.; & Stanislav, V. (2014). Adaptive practice of facts in domains with varied prior knowledge. Proceedings of the International Conference on Educational Data Mining (EDM), 6-13.
77. Pardos, Z. A.; Baker, R. S. J. D.; Gowda, S. M.; & Heffernan, N. T. (2011). The sum is greater than the parts: ensembling models of student knowledge in Educational software. SIGKDD Explorations, 13(2), 37-44.
78. Pashler, H.; Bain, P. M.; Bottqe, B. A.; Graesser, A.; Koedinger, K.; McDaniel, M.; Metcalfe, J. (2007). Organizing instruction and study to improve student learning. IES practice guide NCER 2007-2004. Washington, DC: US Department of Education, Institute of Education Sciences.
79. Pavlik, P.I.; Cen, H.; Koedinger, K.R. (2009) Performance Factors Analysis - A New Alternative to Knowledge Tracing. Proceedings of the International Conference on Artificial Intelligence in Education, Amsterdam, The Netherlands: IOS Press, 531-538
80. Quinn, A.J.; Bederson, B.B. (2011) Human computation: a survey and taxonomy of a growing field. Proceedings of ACM SIGCHI, 1403-1412, New York, NY: ACM
81. Rafferty, A. N.; Brunskill, E.; Griffiths, T.L.; Shafto, P. (2011) Faster teaching by POMDP planning. Proceedings of the 15th International Conference on Artificial Intelligence in Education, 6738, 280-287.
82. Raghuveer, V.;Tripathy, B.; Singh, T.; Khanna, S. (2014) Reinforcement learning approach towards effective content recommendation in mooc environments. Proceedings of IEEE International Conference on MOOC, Innovation and Technology in Education (MITE), 285-289, Patiala: IEEE
83. Razzaq, L.; & Heffernan, N. T. (2006). Scaffolding vs. In Hints in the assistment system. Proceedings of the 8th International Conference on Intelligent Tutoring Systems (ITS 2006) (pp. 635-644).
84. Rodrigo, M. M. T.; Baker, R. S. J. d.; Rossi, L. (2013). Student off-task behavior in computer-based learning in the Philippines: comparison to prior research in the USA. Teachers College Record, 115(10), 1-27.
85. Rodriguez, C. O. (2012). MOOCs and the AI-Stanford like courses: two successful and distinct course formats for massive open online courses. European Journal of Open, Distance and E-Learning, 2012(2).
86. Schofield, J. (1995). Computers and classroom culture. London: Cambridge University Press.
87. Self, J. A. (1990). Theoretical foundations of intelligent tutoring systems. Journal of Artificial Intelligence in Education, 1(4), 3-14.
88. Self, J. A. (1998). The defining characteristics of intelligent tutoring systems research: ITSs care, precisely. International Journal of Artificial Intelligence in Education (IJAIED), 10, 350-364.
89. Serrano-Laguna, A.; & Fernández-Manjón, B. (2014). Applying learning analytics to simplify serious games deployment in the classroom. In IEEE International Global Engineering Education Conference (EDUCON), Istanbul, 872-877.
90. Shute, V. (1990). Rose garden promises of intelligent tutoring systems: blossom or thorn? paper presented at the space. Applications and Research (SOAR) Symposium: Operations.
91. Siemens, G.; Baker, R.S.J.d. (2012) Learning Analytics and Educational Data Mining: Towards Communication and Collaboration. Proceedings of the 2nd International Conference on Learning Analytics and Knowledge, New York, NY: ACM
92. Slotta, J. D.; Tissenbaum, M.; & Lui, M. (2013). Orchestrating of complex inquiry: three roles for learning analytics in a smart classroom infrastructure. In Proceedings of the Third International Conference on Learning Analytics and Knowledge, 270-274. New York, NY: ACM.
93. Stevens, A.; & Collins, A. (1977). The goal structure of a Socratic tutor. Technical Report #3518. Cambridge, MA: Bolt, Beranek, and Newman, Inc.
94. Van Leeuwen, A.; Janssen, J.; Erkens, G.; & Brekelmans, M. (2014). Supporting teachers in guiding collaborating students: effects of learning analytics in CSCL. Computers & Education, 79, 28-39.
95. van Leeuwen, A.; Janssen, J.; Erkens, G.; & Brekelmans, M. (2015). Teacher regulation of cognitive activities during student collaboration: effects of learning analytics. Computers & Education, 90, 80-94.
96. VanLehn, K.; Lynch, C.; Schulze, K.; Shapiro, J. A.; Shelby, R.; Taylor, L.; Treacy, D.; Weinstein, A.; & Wintersgill, M. (2005). The. Andes physics tutoring system: Lessons learned, International Journal of Artificial Intelligence in Education, 15(3), 1-47.
97. Walonoski, J. A.; & Heffernan, N. T. (2006). Prevention of off-task gaming behavior in intelligent tutoring systems. Proceedings of the International Conference on Intelligent Tutoring Systems. Berlin-Heidelberg: Springer, 722-724.
98. Wenger, E. (1987). Artificial Intelligence And Tutoring Systems. Los Altos, CA: Morgan Kauffman.
99. Winograd, T.; & Flores, F. (1986). Understanding computers and cognition. Reading, MA: Addison-Wesley.
100. Wood, D.S.; Williams, G. (2013) Data-Driven Factors that Increase Student Course Completion: A Two-Year Study. Proceedings of the 9th Annual Symposium on Student Retention.
101. Zapata-Rivera, D.; & Katz, I. (2014). Keeping your audience in mind: applying audience analysis to the design of interactive score reports. Assessments in Education: Principles, Policy, and Practice, 21(4), 442-463.