Addressing the assessment challenge with an online system that tutors as it assesses

User Modeling and User-Adapted Interaction

Volumn 19, Issue 3, 2009, Pages 243-266

  Feng, Mingyu ^a,b   Heffernan, Neil ^a   Koedinger, Kenneth ^c  

^a Worcester Polytechnic Institute   (United States)

^b 3101 Wells Branch Pkwy   (United States)

^c Carnegie Mellon University   (United States)

Author keywords

Assistance metrics; ASSISTments; Dynamic assessment; Intelligent tutoring system; Interactive tutoring

Indexed keywords

ASSISTANCE METRICS; ASSISTMENTS; DYNAMIC ASSESSMENT; INTELLIGENT TUTORING SYSTEM; INTERACTIVE TUTORING;

COMPUTER AIDED INSTRUCTION; E-LEARNING; EDUCATION COMPUTING; LAWS AND LEGISLATION; SCHOOL BUILDINGS; TEACHING; TESTING;

STUDENTS;

EID: 68149170049     PISSN: 09241868     EISSN: 15731391     Source Type: Journal    
DOI: 10.1007/s11257-009-9063-7     Document Type: Article

References (41)

1. Anozie, N., Junker, B.W.: Predicting end-of-year accountability assessment scores from monthly student records in an online tutoring system. In: Beck, J., Aimeur, E., Barnes, T. (eds.) Educational Data Mining: Papers from the AAAI Workshop, pp. 1-6. AAAI Press, Menlo Park, CA Technical Report WS-06-05 (2006) (Pubitemid 46053302)
2. Ayers E., Junker B.W.: Do skills combine additively to predict task difficulty in eighth grade mathematics? In: Beck, J., Aimeur, E., Barnes, T. (eds.) Educational Data Mining: Papers from the AAAI Workshop, pp. 14-20. AAAI Press, Menlo Park, CA, Technical Report WS-06-05 (2006) (Pubitemid 46053304)
3. Baker, R.S., Corbett, A.T., Koedinger, K.R.: Detecting student misuse of intelligent tutoring systems. In: James, C.L., Vicari, R.M., Paraguacu, F. (eds.) Intelligent Tutoring Systems: 7th International Conference ITS 2004, Maceió, Alagoas, Brazil Proceedings, pp. 531-540. Springer-Verlag Berlin Heidelberg, Berlin, Germany (2004)
4. Baker, R.S., Roll, I., Corbett, A.T., Koedinger, K.R.: Do performance goals lead students to game the system? In: Proceedings of the 12th International Conference on Artificial Intelligence in Education, pp. 57-64. Netherlands, Amsterdam (2005)
5. J.E. Beck J. Sison 2006 Using knowledge tracing in a noisy environment to measure student reading proficiencies Int. J. Artif. Intell. Educ. 16 129 143
6. J.E. Beck P. Jia J. Mostow 2004 Automatically assessing oral reading fluency in a computer tutor that listens Technol. Instr. Cogn. Learn. 2 61 81
7. P. Black D. Wiliam 1998 Assessment and classroom learning Assess. Educ.: Princ., Policy Pract. 5 7 74
8. P. Black D. Wiliam 1998 Inside the black box: raising standards through classroom assessment Phi Delta Kappan 80 2 139 149
9. Boston, C.: The concept of formative assessment. Pract. Assess. Res. Eval. 8(9) (2002)
10. Campione J.C., Brown A.L., Bryant R.J.: Individual differences in learning and memory. In: Sternberg, R.J. (eds) Human Abilities: An Information-processing Approach, pp. 103-126. W. H. Freeman, New York (1985)
11. Corbett, A.T., Bhatnagar, A.: Student modeling in the ACT Programming Tutor: Adjusting a procedural learning model with declarative knowledge. User Modeling: Proceedings of the Sixth International Conference on User Modeling UM97 Chia Laguna, Sardinia, Italy, pp. 243-254. Springer-Verlag Wein, New York (1997)
12. Corbett A.T., Anderson J.R., O'Brien A.T.: Student modeling in the ACT programming tutor. In: Nichols, P., Chipman, S., Brennan, R. (eds) Cognitively Diagnostic Assessment., Erlbaum, Hillsdale, NJ (1995)
13. Computer Research Association.: Cyberinfrastructure for Education and Learning for the Future: a Vision and Research Agenda. Final report of Cyberlearning Workshop Series workshops held Fall 2004-Spring 2005 by the Computing Research Association and the International Society of the Learning Sciences. Retrieved from http://www.cra.org/reports/cyberinfrastructure.pdf on 10 November 2006 (2005)
14. S.E. Embretson 1992 Structured Rasch models for measuring individual-difference in learning and change Int. J. Psychol. 27 3-4 372 372
15. M. Feng N.T. Heffernan 2007 Towards live informing and automatic analyzing of student learning: Reporting in the assistment system J. Interact. Learn. Res. 18 2 207 230 AACE, Chesapeake, VA
16. Feng, M., Heffernan, N.T., Koedinger, K.R.: Addressing the testing challenge with a web-based e-assessment system that tutors as it assesses. In: Carr, L.A., De Roure, D.C., Iyengar, A., Goble, C.A., Dahlin, M. (eds.) Proceedings of the Fifteenth International World Wide Web Conference, pp. 307-316. Edinburgh UK, 2006. ACM Press, New York, NY (2006)
17. Feng M., Heffernan N., Beck J., Koedinger K.: Can we predict which groups of questions students will learn from?. In: Baker, Beck (eds) Proceedings of the First International Conference on Educational Data Mining, pp. 218-225. Montreal, Canada (2008)
18. Feng M., Beck J., Heffernan N., Koedinger K.: Can an intelligent tutoring system predict math proficiency as well as a standardized test? In: Baker, Beck (eds) Proceedings of the First International Conference on Educational Data Mining, pp. 107-116. Montreal, Canada (2008)
19. Fischer G., Seliger E.: Multidimensional linear logistic models for change. Chap. 19. In: Linden, W.J., Hambleton, R.K. (eds) Handbook of Modern Item Response Theory, Springer-Verlag, New York (1997)
20. E.L. Grigorenko R.J. Sternberg 1998 Dynamic testing Psychol. Bull. 124 75 111
21. C.L. Hulin R.I. Lissak F. Drasgow 1982 Recovery of two- and three-parameter logistic item characteristic curves: A Monte Carlo study Appl. Psychol. Meas. 6 3 249 260
22. R.J. Jannarone 1986 Conjunctive item response theory kernels Psychometrika 55 3 357 373
23. Koedinger, K.R., Aleven, V., Heffernan, N.T., McLaren, B., Hockenberry, M.: Opening the door to non-programmers: authoring intelligent tutor behavior by demonstration. In: Proceedings of the 7th International Conference on Intelligent Tutoring Systems, pp. 162-173. Maceio, Brazil (2004)
24. Massachusetts Department of Education.: Massachusetts Mathematics Curriculum Framework. Retrieved from http://www.doe.mass.edu/frameworks/math/ 2000/final.pdf, 6 November 2005 (2000)
25. MCAS technical report.: Retrieved from http://www.cs.wpi.edu/mfeng/pub/ mcas-techrpt01.pdf, 5 August 2005 (2001)
26. Mitchell T.: Machine Learning. McGraw-Hill, Columbus, OH (1997)
27. Mostow J., Aist G.: Evalutating tutors that listen: an overview of Project LISTEN. In: Feltovich, P. (eds) Smart Machines in Education, pp. 169-234. MIT/AAAI Press, Menlo Park, CA (2001)
28. Olson, L.: State test programs mushroom as NCLB Mandate Kicks. In: Education Week, 20 November, pp. 10-14 (2004)
29. Olson, L.: Special report: testing takes off. Education Week, 30 November 2005, pp. 10-14 (2005)
30. A.E. Raftery 1995 Bayesian model selection in social research Sociol Methodol 25 111 163
31. Razzaq, L., Heffernan, N.T.: Scaffolding vs. hints in the Assistment System. In: Ikeda, Ashley, Chan (eds.) Proceedings of the 8th International Conference on Intelligent Tutoring Systems, pp. 635-644. Springer-Verlag, Jhongli, Taiwan, Berlin, Germany (2006) (Pubitemid 44117635)
32. Razzaq, L., Feng, M., Nuzzo-Jones, G., Heffernan, N.T., Koedinger, K.R., Junker, B., Ritter, S., Knight, A., Aniszczyk, C., Choksey, S., Livak, T., Mercado, E., Turner, T.E., Upalekar, R., Walonoski, J.A., Macasek, M.A., Rasmussen, K.P.: The ASSISTment project: blending assessment and assisting. In: Proceedings of the 12th Annual Conference on Artificial Intelligence in Education. Amsterdam, The Netherlands, pp. 555-562. ISO Press, Amsterdam (2005)
33. Razzaq, L., Heffernan, N.T., Lindeman, R.W.: What level of tutor interaction is best? In: Luckin, Koedinger (eds.) Proceedings of the 13th Conference on Artificial Intelligence in Education, pp. 222-229. IOS Press, Los Angeles, CA, Amsterdam, The Netherlands (2007)
34. H.L. Roediger III J.D. Karpicke 2006 The power of testing memory Perspect. Psychol. Sci. 1 3 181 210
35. R.J. Sternburg E.L. Grigorenko 2001 All testing is dynamic testing Issues Educ. 7 137 170
36. Sternburg R.J., Grigorenko E.L.: Dynamic Testing: The Nature and Measurement of Learning Potential. Cambridge University Press, Cambridge (2002)
37. E.S. Tan T. Imbos R.J.M. Does 1994 A distribution-free approach to comparing growth of knowledge J. Educ. Measure. 31 1 51 65
38. K.K. Tatsuoka 1983 Rule space: an approach for dealing with misconceptions based on item response theory J. Educ. Measure. 20 345 354
39. van der Linden, W.J., Hambleton, R.K. (eds.): Handbook of Modern Item Response Theory. Springer Verlag, New York, NY (1997)
40. Walonoski, J., Heffernan, N.T.: Detection and analysis of off-task gaming behavior in intelligent tutoring systems. In: Ikeda, Ashley, Chan (eds.) In: Proceedings of the 8th International Conference on Intelligent Tutoring Systems. Berlin, pp. 382-391. Springer-Verlag, Jhongli, Taiwan (2006)
41. Zimowski, M., Muraki, E., Mislevy, R., Bock, D.: BILOG-MG 3-Multiple-Group IRT Analysis and Test maintenance for Binary Items. Scientific Software International, Inc., Lincolnwood, IL. URL http://www.ssicentral.com/. (2005)