Using mixed-effects modeling to analyze different grain-sized skill models in an intelligent tutoring system

IEEE Transactions on Learning Technologies

Volumn 2, Issue 2, 2009, Pages 79-92

  Feng, Mingyu ^a   Heffernan, Neil T ^a   Heffernan, Cristina ^a   Mani, Murali ^a  

^a Worcester Polytechnic Institute   (United States)

Author keywords

Cognitive diagnostic assessment; Fine grained skill model; Intelligent tutoring systems; Mixed effects model; Statistical analysis of skill models

Indexed keywords

COGNITIVE DIAGNOSTIC ASSESSMENT; FINE-GRAINED SKILL MODEL; INTELLIGENT TUTORING SYSTEMS; MIXED-EFFECTS MODEL; STATISTICAL ANALYSIS OF SKILL MODELS;

COGNITIVE SYSTEMS; E-LEARNING; EDUCATION COMPUTING; INTELLIGENT VEHICLE HIGHWAY SYSTEMS; STUDENTS;

COMPUTER AIDED INSTRUCTION;

EID: 84857285263     PISSN: 19391382     EISSN: None     Source Type: Journal    
DOI: 10.1109/TLT.2009.17     Document Type: Conference Paper

References (46)

1. R.G. Almond, L.V. DiBello, B. Moulder, and D. Zapata-Rivera, "Modeling Diagnostic Assessment with Bayesian Networks," J. Educational Measurement, vol. 44, no. 4, pp. 341-359, 2007.
2. J.R. Anderson, Rules of Mind. Lawrence Erlbaum Assoc., 1993.
3. J.R. Anderson and C. Lebiere, The Atomic Components of Thought. Lawrence Erlbaum Assoc., 1998.
4. J.R. Anderson, A.T. Corbett, K.R. Koedinger, and R. Pelletier, "Cognitive Tutors: Lessons Learned," J. Learning Sciences, vol. 4, no. 2, pp. 167-207, 1995.
5. E. Ayers and B.W. Junker, "Do Skills Combine Additively to Predict Task Difficulty in Eighth Grade Mathematics?" Proc. Educational Data Mining: Papers from the Am. Assoc. for Artificial Intelligence (AAAI) Workshop, J. Beck, E. Aimeur, and T. Barnes, eds., pp. 14-20, 2006.
6. T. Barnes, "Q-Matrix Method: Mining Student Response Data for Knowledge," Educational Data Mining: Papers from the 2005 Am. Assoc. for Artificial Intelligence (AAAI) Workshop, J. Beck, ed., 2005.
7. D. Bates, "Linear Mixed Model Implementation in lme4," Manuscript, Univ. of Wisconsin, May 2007.
8. R.D. Bock, R. Gibbons, and E.J. Muraki, "Full Information Item Factor Analysis," Applied Psychological Measurement, vol. 12, pp. 261-280, 1988.
9. H. Cen, K. Koedinger, and B. Junker, "Automating Cognitive Model Improvement by A*Search and Logistic Regression," Proc. Educational Data Mining: Papers from the 2005 Am. Assoc. for Artificial Intelligence (AAAI) Workshop, J. Beck, ed., 2005.
10. J. Collins, J. Greer, and S. Huang, "Adaptive Assessment of Using Granularity Hierarchies and Bayesien Nets," Proc. Int'l Conf. Intelligent Tutoring Systems, pp. 569-577, 1996.
11. J. Confrey, A. Valenzuela, and A. Ortiz, "Recommendation to the Texas State Board of Education on the Setting of TAKS Standards: A Call to Responsible Action," http://www.syrce.org/State_Board.htm, 2002.
12. A.T. Corbett, J.R. Anderson, and A.T. O'Brien, "Student Modeling in the ACT Programming Tutor," Cognitively Diagnostic Assessment, P. Nichols, S. Chipman, and R. Brennan, eds., Lawrence Erlbaum Assoc., 1995.
13. E. Croteau, N.T. Heffernan, and K.R. Koedinger, "Why Are Algebra Word Problems Difficult? Using Tutorial Log Files and the Power Law of Learning to Select the Best Fitting Cognitive Model," Proc. Seventh Int'l Conf. Intelligent Tutoring Systems, pp. 240-250, 2004.
14. B. Daniel, D. Zapata-Rivera, and G. McCalla, "A Bayesian Belief Network Approach for Modeling Complex Domains," Bayesian Network Technologies: Applications and Graphical Models, A. Mittal and A. Kassim, eds., pp. 13-41, IRM Press, 2007.
15. K.L. Draney, P. Pirolli, and M. Wilson, "A Measurement Model for a Complex Cognitive Skill," Cognitively Diagnostic Assessment, P. Nichols, S. Chipman, and R. Brennan, eds., Lawrence Erlbaum Assoc., 1995.
16. S.E. Embretson, "Structured Rasch Models for Measuring Individual-Difference in Learning and Change," Int'l J. Psychology vol. 27, nos. 3/4, pp. 372-372, 1992.
17. M. Feng, N.T. Heffernan, and K.R. Koedinger, "Predicting State Test Scores Better with Intelligent Tutoring Systems: Developing Metrics to Measure Assistance Required," Proc. Eighth Int'l Conf. Intelligent Tutoring Systems, M. Ikeda, K.D. Ashley, and T.-W. Chan, eds., pp. 31-40, 2006.
18. M. Feng, N.T. Heffernan, M. Murali, and C. Heffernan, "Using Mixed-Effects Modeling to Compare Different Grain-Sized Skill Models," Proc. Educational Data Mining: Papers from the Am. Assoc. for Artificial Intelligence (AAAI) Workshop, J. Beck, E. Aimeur, and T. Barnes, eds., pp. 57-66, 2006.
19. M. Feng and N. Heffernan, "Towards Live Informing and Automatic Analyzing of Student Learning: Reporting in ASSISTment System," J. Interactive Learning Research, vol. 18, no. 2, pp. 207-230, 2007.
20. M. Feng, J. Beck, N. Heffernan, and K. Koedinger, "Can an Intelligent Tutoring System Predict Math Proficiency as well as a Standardized Test?" Proc. First Int'l Conf. Education Data Mining, R.S.J.d. Baker and J.E. Beck, eds., pp. 107-116, 2008.
21. M. Feng, N. Heffernan, J. Beck, and K. Koedinger, "Can We Predict Which Groups of Questions Students Will Learn From?" Proc. First Int'l Conf. Education Data Mining, R.S.J.d. Baker and J.E. Beck, eds., pp. 218-225, 2008.
22. M. Feng, N.T. Heffernan, and K.R. Koedinger, "Addressing the Assessment Challenge in an Online System That Tutors as It Assesses," to be published in User Modeling and User-Adapted Interaction: The J. Personalization Research.
23. K. Ferguson, I. Arroyo, S. Mahadevan, B. Woolf, and A. Barto, "Improving Intelligent Tutoring Systems: Using Expectation Maximization to Learn Student Skill Levels," Proc. Eighth Int'l Conf. Intelligent Tutoring Systems, M. Ikeda, K.D. Ashley, and T.-W. Chan, eds., pp. 453-462, 2006.
24. M.J. Gierl, C. Wang, and J. Zhou, "Using the Attribute Hierarchy Method to Make Diagnostic Inferences about Examinees' Cognitive Skills in Algebra on the SAT," J. Technology, Learning, and Assessment, vol. 6, no. 6, http://www.jtla.org, 2008.
25. D. Hedeker and R.D. Gibbons, Longitudinal Data Analysis. John Wiley & Sons, 2006.
26. N.T. Heffernan, T.E. Turner, A.L.N. Lourenco, M.A. Macasek, G. Nuzzo-Jones, and K.R. Koedinger, "The ASSISTment Builder: Towards an Analysis of Cost Effectiveness of ITS Creation," Proc. 19th Int'l Florida Artificial Intelligence Research Soc. Conf. (FLAIRS '06), G. Sutcliffe and R. Goebel, eds., pp. 515-520, 2006.
27. B. Junker, "Using On-Line Tutoring Records to Predict End-of-Year Exam Scores: Experience with the ASSISTments Project and MCAS Eighth Grade Mathematics," Assessing and Modeling Cognitive Development in School R.W. Lissitz, ed., JAM Press, 2007.
28. S. Katz, A. Lesgold, G. Eggan, and M. Gordin, "Modeling the Student in Sherlock II," Int'l J. Artificial Intelligence in Education, vol. 3, no. 4, pp. 495-518, 1992.
29. M. Militello, S. Sireci, and J. Schweid, "Intent, Purpose, and Fit: An Examination of Formative Assessment Systems in School Districts," Proc. Am. Educational Research Assoc. Ann. Meeting, 2008.
30. R. Mislevy, "Cognitive Psychology and Educational Assessment," Educational Measurement, fourth ed., R.L. Brennan, ed., Am. Council on Education, 2006.
31. M.J. Nathan and K.R. Koedinger, "An Investigation of Teachers' Beliefs of Students' Algebra Development," Cognition and Instruction, vol. 18, no. 2, pp. 209-237, 2000.
32. Cognitively Diagnostic Assessment, P.D. Nichols, S.F. Chipman, and R.L. Brennan, eds. Lawrence Erlbaum Assoc., 1995.
33. L. Olson, "State Test Programs Mushroom as NCLB Mandate Kicks," Education Week, pp. 10-14, Nov. 2004.
34. Z. Pardos, M. Feng, N.T. Heffernan, and C. Heffernan-Lindquist, "Analyzing Fine-Grained Skill Models Using Bayesian and Mixed Effect Methods," Proc. 13th Conf. Artificial Intelligence in Education, R. Luckin and K. Koedinger, eds., pp. 626-628, 2007.
35. Z.A. Pardos, N.T. Heffernan, C. Ruiz, and J. Beck, "The Composition Effect: Conjunctive or Compensatory? An Analysis of Multi-Skill Math Questions in ITS," Proc. First Int'l Conf. Educational Data Mining R.S.J.d. Baker and J.E. Beck, eds., 2008.
36. R Development Core Team "R: A Language and Environment for Statistical Computing," R Foundation for Statistical Computing, http://www.r-project.org, 2007.
37. A. Raftery, "Bayesian Model Selection in Social Research," Sociological Methodology, vol. 25, pp. 111-163, 1995.
38. E. Ramírez and K. Clark, "What Arne Duncan Thinks of No Child Left Behind: The New Education Secretary Talks about the Controversial Law and Financial Aid Forms," http://www.usnews.com/articles/education/ 2009/02/05/what-arne-duncanthinks-of-no-child-left-behind.html, 2009.
39. L. Razzaq and N.T. Heffernan, "Scaffolding vs. Hints in the Assistment System," Proc. Int'l Conf. Intelligent Tutoring Systems, M. Ikeda, K.D. Ashley, and T.-W. Chan, eds., pp. 635-644, 2006.
40. L. Razzaq, N.T. Heffernan, and R.W. Lindeman, "What Level of Tutor Interaction Is Best?," Proc. 13th Conf. Artificial Intelligence in Education, R. Luckin and K. Koedinger, eds., pp. 222-229, 2007.
41. J.D. Singer and J.B. Willett, Applied Longitudinal Data Analysis: Modeling Change and Occurrence. Oxford Univ. Press, 2003.
42. E.S. Tan, T. Imbos, and R.J.M. Does, "A Distribution-Free Approach to Comparing Growth of Knowledge," J. Education Measurement, vol. 31, no. 1, pp. 51-65, 1994.
43. K.K. Tatsuoka, "Toward an Integration of Item Response Theory and Cognitive Error Diagnosis," Diagnostic Monitoring of Skill and Knowledge Acquisition, N. Frederiksen, R. Glaser, A. Lesgold, and M.G. Shafto, eds., pp. 453-488, Lawrence Erlbaum Assoc., 1990.
44. Handbook of Modern Item Response Theory, W.J. Van Der Linden, and R.K. Hambleton eds. Springer Verlag, 1997.
45. E.C. Wylie and J. Ciofalo, "Supporting Teachers' Use of Individual Diagnostic Itemsm," http://www.tcrecord.org/ PrintContent.asp?ContentID=15363, 2008.
46. J.T. Yun, J. Willet, and R. Murnane, "Accountability-Based Reforms and Instruction: Testing Curricular Alignment for Instruction Using the Massachusetts Comprehensive Assessment System," Proc. Ann. Am. Educational Research Assoc. Meeting, 2004.