A multifactor approach to student model evaluation

User Modeling and User-Adapted Interaction

Volumn 18, Issue 4, 2008, Pages 349-382

  Yudelson, Michael V ^a,b   Medvedeva, Olga P ^a   Crowley, Rebecca S ^a,b,c  

^a UNIVERSITY OF PITTSBURGH SCHOOL OF MEDICINE   (United States)

^b UNIVERSITY OF PITTSBURGH   (United States)

^c UNIVERSITY OF PITTSBURGH CANCER INSTITUTE   (United States)

Author keywords

Bayesian models; Decision theory; Hidden Markov models; Intelligent medical training systems; Intelligent tutoring systems; Knowledge Tracing; Machine learning; Methodology; Model evaluation; Model selection; Probabilistic models; Student modeling

Indexed keywords

BAYESIAN; BUSINESS MEDIA; EVALUATING MODELS; INTELLIGENT TUTORING SYSTEMS (ITS); MODEL SELECTIONS; MULTI FACTORS; NOVEL DOMAIN; RESIDUAL ANALYSIS; STUDENT MODEL (SM); STUDENT MODELS;

COMPUTER AIDED INSTRUCTION; INTELLIGENT SYSTEMS; INTELLIGENT VEHICLE HIGHWAY SYSTEMS; STUDENTS;

LEARNING SYSTEMS;

EID: 48449092856     PISSN: 09241868     EISSN: 15731391     Source Type: Journal    
DOI: 10.1007/s11257-007-9046-5     Document Type: Article

References (35)

1. Anderson, J.: Rules of the Mind. Lawrence Erlbaum Associates, Hillsdale, NJ (1993)
2. Anderson, J., Schunn, C.: Implications of the ACT-R learning theory: no magic bullets. In: Glaser, R. (eds) Advances in Instructional Psychology: Educational Design and Cognitive Science, vol. 5, pp. 1-34. Erlbaum, Mahwah, NJ (2000)
3. Anderson, J.R., Corbett, A.T., Koedinger, K.R., Pelletier, R.: Cognitive tutors: lessons learned. J. Learn. Sci. 4(2), 167-207 (1995)
4. Atkinson, R., Shiffrin, R.: Human memory: a proposed system and its control processes. In: Spence, K.W., Spence, J.T. (eds) The Psychology of Learning and Motivation: Advances in Research and Theory, vol. 2, pp. 742-775. Academic Press, New York (1968)
5. Beck, J., Sison, J.: Using knowledge tracing to measure student reading proficiencies. In: Proceedings of the 7th International Conference on Intelligent Tutoring Systems, pp. 624-634. Springer-Verlag, Maceio, Brazil (2004)
6. Brand, M., Oliver, N., Pentland, A.: Coupled hidden Markov models for complex action recognition. In: Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition, pp. 994-999. San Juan, Puerto Rico (1997)
7. Byrne, M.D.: Perception and action. In: Anderson, J.R., Lebiére, C. (eds) Atomic Components of Thought, pp. 167-200. Erlbaum, Hillsdale (1998)
8. Chang, K., Beck, J., Mostow, J., Corbett, A.: Does help help? A Bayes net approach to modeling tutor interventions. In: Proceedings of the 21st Annual Meeting of the American Association for Artificial Intelligence, pp. 41-46. Boston, MA (2006)
9. Clancey, W.: Methodology for building an intelligent tutoring system. In: Kintsch, W., Miller, H., Poison, P. (eds) Methods and Tactics in Cognitive Science, pp. 51-84. Erlbaum, Hillsdale (1984)
10. Clancey, W., Letsinger, R.: NEOMYCIN: reconfiguring a rule based expert system for application to teaching. In: Proceedings of the Seventh International Joint Conference on AI, pp. 829-835. Vancouver, BC, Canada (1981)
11. Conati, C., Zhao, X.: Building and evaluating an intelligent pedagogical agent to improve the effectiveness of an educational game. In: Proceedings of the 9th International Conference on Intelligent User Interface, pp. 6-13. Funchal, Madeira, Portugal (2004)
12. Conati, C., Gertner, A., VanLehn, K.: Using Bayesian networks to manage uncertainty in student modeling. J. User Model. User-Adap. Interac. 12(4), 371-417 (2002)
13. Corbett, A., Anderson, J.: Knowledge tracing: modeling the acquisition of procedural knowledge. User Model. User-Adap. Interac. 4, 253-278 (1995)
14. Crowley, R., Medvedeva, O.: An intelligent tutoring system for visual classification problem solving. Artif. Intell. Med. 36(1), 85-117 (2006)
15. Crowley, R., Naus, G., Stewart, J., Friedman, C.: Development of visual diagnostic expertise in pathology - an information processing study. J. Am. Med. Inform. Assoc. 10(1), 39-51 (2003)
16. Crowley, R., Legowski, E., Medvedeva, O., Tseytlin, E., Roh, E., Jukic, D.: Evaluation of an Intelligent Tutoring system in pathology: effects of external representation on performance gains, metacognition, and acceptance. J. Am. Med. Inform. Assoc. 14(2), 182-190 (2007)
17. Davis, J., Goadrich, M.: The relationship between Precision-Recall and ROC curves. In: Proceedings of the 23rd International Conference on Machine Learning, vol. 148, pp. 233-240. Pittsburgh, PA (2006)
18. Ephraim, Y., Roberts, W.: Revisiting autoregressive hidden Markov modeling of speech signals. IEEE Sig. Proc. Lett. 12, 166-169 (2005)
19. Fawcett, T.: Graphs: notes and practical considerations for data mining researchers. Tech Reports HPL-2003-4. HP Laboratories, Palo Alto, CA (2003)
20. Ferguson, K., Arroyo, I., Mahadevan, S., Woolf, B., Barto, A.: Improving intelligent tutoring systems: Using EM to learn student skill levels, Intelligent Tutoring Systems, pp. 453-462. Springer-Verlag, Jhongli, Taiwan (2006)
21. Fogarty, J., Baker, R.S., Hudson, S.: Case studies in the use of ROC curve analysis for sensor-based estimates in Human Computer Interaction. In: Proceedings of Graphics Interface, pp. 129-136. Victoria, British Columbia, Canada (2005)
22. Hanley, J.A., McNeil, B.J.: The meaning and use of the area under a Receiver Operating Characteristic (ROC) Curve. Radiology 143, 29-36 (1982)
23. Jastrzembski, T., Gluck, K., Gunzelmann, G.: Knowledge tracing and prediction of future trainee performance. In: Proceedings of the 2006 Interservice/Industry Training, Simulation, and Education Conference, pp. 1498-1508. National Training Systems Association, Orlando, FL (2006)
24. Jonsson, A., Johns, J., Mehranian, H., Arroyo, I., Woolf, B., Barto, A., Fisher, D., Mahadevan, S.: Evaluating the feasibility of learning student models from data. In: AAAI05 Workshop on Educational Data Mining, pp. 1-6. Pittsburgh, PA (2005)
25. Kuenzer, A., Schlick, C., Ohmann, F., Schmidt, L., Luczak, H.: An empirical study of dynamic Bayesian networks for user modeling. In: UM'01 Workshop on Machine Learning for User Modeling, pp. 1-10. Sonthofen, Germany (2001)
26. Mayo, M., Mitrovic, A.: Optimizing ITS behavior with Bayesian networks and decision theory. Int. J. Artifi. Intell. Educ. 12, 124-153 (2001)
27. Medvedeva, O., Chavan, G., Crowley, R.: A data collection framework for capturing ITS data based on an agent communication standard. In: Proceedings of the 20th Annual Meeting of the American Association for Artificial Intelligence, pp. 23-30. Pittsburgh, PA (2005)
28. Moore, D., McCabe, G.: Introduction to the Practice of Statistics. W.H. Freeman and Company, New York (1993)
29. Murphy, K.: The Bayes Net Toolbox for Matlab. Computing Science and Statistics, vol. 33, pp. 1-20. URL: http://www.bnt.sourceforge.net (Accessed on December 6, 2008) (2001)
30. Provost, F., Fawcett, T., Kohavi, R.: The case against accuracy estimation for comparing induction algorithms. In: Proceeding of the 15th International Conference on Machine Learning, pp. 445-453. San Francisco, CA (1998)
31. Reye, J.: Student modeling based on belief networks. Int. J. Artif. Intell. Educ. 14, 1-33 (2004)
32. Seidemann, E., Meilijson, I., Abeles, M., Bergman, H., Vaadia, E.: Simultaneously recorded single units in the frontal cortex go through sequences of discrete and stable states in monkeys performing a delayed localization task. J. Neurosci. 16(2), 752-768 (1996)
33. Shang, Y., Shi, H., Chen, S.: An intelligent distributed environment for active learning. J. Educ. Resour. Comput. 1(2), 1-17 (2001)
34. VanLehn, K., Niu, Z.: Bayesian student modeling, user interfaces and feedback: a sensitivity analysis. Int. J. Artif. Intell. Educ. 12, 154-184 (2001)
35. Zukerman, I., Albrecht, D.W., Nicholson, A.E.: Predicting users' requests on the WWW. In: Proceedings of the Seventh International Conference on User Modeling, (UM-99), pp. 275-284. Banff, Canada (1999)