A methodology for eliciting, modelling, and evaluating expert knowledge for an adaptive work-integrated learning system

International Journal of Human Computer Studies

Volumn 68, Issue 4, 2010, Pages 185-208

  Ley, Tobias ^a,c   Kump, Barbara ^b   Albert, Dietrich ^c  

^a KNOW CENTER   (Austria)

^b GRAZ UNIVERSITY OF TECHNOLOGY   (Austria)

^c UNIVERSITY OF GRAZ   (Austria)

Author keywords

Competence based knowledge space theory; Domain modelling; Knowledge elicitation; Model evaluation; Work integrated learning

Indexed keywords

EDUCATIONAL SETTINGS; EVALUATING MODELS; EXPERT KNOWLEDGE; ITERATIVE REFINEMENT; KNOWLEDGE ELICITATION; KNOWLEDGE SPACE THEORY; LESSONS LEARNED; MODEL EVALUATION; MODELLING PROCESS; WORK DOMAINS; WORK-INTEGRATED LEARNING; WORKPLACE LEARNING;

CURRICULA;

KNOWLEDGE ENGINEERING;

EID: 76449086321     PISSN: 10715819     EISSN: 10959300     Source Type: Journal    
DOI: 10.1016/j.ijhcs.2009.12.001     Document Type: Article

References (73)

1. Albert D., Hockemeyer C., and Wesiak G. Current trends in eLearning based on knowledge space theory and cognitive psychology. Psychologische Beiträge 4 44 (2002) 478-494
2. Albert, D., Kaluscha, R., 1997. Adapting knowledge structures in dynamic domains. In: Herzog, C. (Ed.), Beiträge zum Achten Arbeitstreffen der GI-Fachgruppe 1.1.5/7.0.1 "Intelligente Lehr-/Lernsysteme", September 1997, Duisburg, Germany [Contributions of the 8th Workshop of the GI SIG "Intelligent Tutoring Systems"]. TU München, Munich, pp. 89-100.
3. In: Albert D., and Lukas J. (Eds). Knowledge Spaces: Theories, Empirical Research, and Applications (1999), Lawrence Erlbaum Associates, Mahwah, New Jersey
4. Albert, D., Pivec, M., Spörk-Fasching, T., Maurer, H., 2003. Adaptive intercultural competence testing: a computerized approach based on knowledge space theory. In: Proceedings of the UNESCO Conference on Intercultural Education, Jyväskylä, Finland. June 15-18, 2003.
5. ALEKS Corp., 2003. ALEKS-A Better State of Knowledge. Consulted March 4, 2008. 〈http://www.aleks.com〉.
6. Anderson L.W., and Krathwohl D.A. A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom's Taxonomy of Educational Objectives (2001), Longman, New York
7. Angele J., Fensel D., Landes D., and Studer R. Developing knowledge-based systems with MIKE. Automated Software Engineering 5 4 (1998) 389-418
8. APOSDLE Consortium, 2006. Use Cases and Application Requirements 1 (First Prototype), Project Deliverable D6.2, Consulted March 4, 2008 〈http://www.aposdle.org/media/multimedia/files/use_cases_application_requirements_1_first_prototype〉
9. Benyon D.R., and Murray D.M. Adaptive systems; from intelligent tutoring to autonomous agents. Knowledge-Based Systems 6 4 (1993) 197-219
10. Billett S. Constituting the Workplace Curriculum. Journal of Curriculum Studies 38 1 (2006) 31-48
11. Brewer M.B. Research design and issues of validity. In: Reis H.T., and Judd C.M. (Eds). Handbook of Research Methods in Social and Personality Psychology (2000), Cambridge University Press, Cambridge 3-16
12. Brusilovsky, P., Nijhavan, H., 2002. A framework for adaptive eLearning based on distributed re-usable learning activities. In: Driscoll, M., Reeves, T.C. (Eds.), Proceedings of the World Conference on eLearning, E-Learn 2002. AACE, Montreal, pp. 154-161.
13. Cardinali F. Innovating eLearning and Mobile Learning Technologies for Europe's Future Educational Challenges, Theory and Case Studies. In: Neidl W., and Tochtermann K. (Eds). Proceedings of the 1st European Conference on Technology-enhanced Learning (LNCS, vol. 4227) (2006), Springer, Heidelberg 2-7
14. Castro-Schez J.J., Jennings N.R., Xudong L., and Shadbolt N.R. Acquiring domain knowledge for negotiating agents: a case of study. International Journal of Human-Computer Studies 61 (2004) 3-31
15. Chin D.N. Empirical evaluation of user models and user-adapted systems. User Modeling and User-Adapted Interaction 11 1-2 (2001) 181-194
16. Conati C., Gertner A.S., and VanLehn K. Using bayesian networks to manage uncertainty in student modeling. User Modeling and User-Adapted Interaction 12 4 (2002) 371-417
17. Conlan O., Hockemeyer C., Wade V., and Albert D. Metadata driven approaches to facilitate adaptivity in personalized eLearning systems. The Journal of Information and Systems in Education 1 (2002) 38-44
18. Cooke N.J. Varieties of knowledge elicitation techniques. International Journal of Human-Computer Studies 41 (1994) 801-849
19. De Bra P., Aroyo L., and Chepegin V. The next big thing: adaptive web-based systems. Journal of Digital Information 5 (2004) 1
20. Desmarais M., Fu S., and Pu X. Tradeoff analysis between knowledge assessment approaches. In: Looi C., McCalla G.I., Bredeweg B., and Breuker J. (Eds). Artificial Intelligence in Education: Supporting Learning through Intelligent and Socially Informed Technology (2005), IOS Press, Amsterdam 209-216
21. Desmarais M., and Gagnon M. Bayesian Student Models Based on Item to Item Knowledge Structures. In: Neidl W., and Tochtermann K. (Eds). Proceedings of the 1st European Conference on Technology-enhanced Learning (LNCS, vol. 4227) (2006), Springer, Heidelberg 111-124
22. Doignon J. Knowledge Spaces and Skill Assignments. In: Fischer D.H., and Laming D. (Eds). Contributions to Mathematical Psychology, Psychometrics and Methodology (1994), Springer, New York 111-121
23. Doignon J., and Falmagne J. Spaces for the assessment of knowledge. International Journal of Man-Machine Studies 23 (1985) 175-196
24. Doignon J., and Falmagne J. Knowledge Spaces (1999), Springer, Heidelberg
25. Dowling C. Integrating different knowledge spaces. In: Fischer G.H., and Laming D. (Eds). Contributions to Mathematical Psychology (1994), Springer, New York 149-158
26. Düntsch I., and Gediga G. Skills and knowledge structures. British Journal of Mathematical and Statistical Psychology 48 (1995) 9-27
27. Eklund J., and Brusilovsky P. Interbook: an adaptive tutoring system. UniServe Science News 12 3 (1999) 8-13
28. Eraut M. Informal learning in the workplace. Studies in Continuing Education 26 2 (2004) 247-273
29. Falmagne J., Cosyn E., Doignon J., and Thiéry N. The assessment of knowledge, in theory and in practice. In: Missaoui R., and Schmid J. (Eds). Formal Concept Analysis (2006), Springer, Heidelberg 61-79
30. Falmagne J.-C., and Doignon J.-P. A class of stochastic procedures for the assessment of knowledge. British Journal of Mathematical and Statistical Psychology 41 (1988) 1-23
31. Falmagne J., Doignon J., Koppen M., Villano M., and Johannesen L. Introduction to knowledge spaces: how to build, test, and search them. Psychological Review 97 2 (1990) 201-224
32. Farmer, J., Lindstaedt, S.N., Droschl, G., Luttenberger, P., 2004. AD-HOC-work-integrated technology-supported teaching and learning, In: Proceedings of the 5th European Conference on Organisational Knowledge, Learning and Capabilities, Innsbruck, Austria, April 2-3, 2005, Innsbruck. Consulted October 12, 2009. 〈http://www2.warwick.ac.uk/fac/soc/wbs/conf/olkc/archive/oklc5/〉
33. Fischer G. User modeling in human-computer interaction. User Modeling and User-Adapted Interaction 11 1-2 (2001) 65-86
34. Gaines B.R., and Shaw M.L.G. Knowledge Acquisition Tools Based on Personal Construct Psychology. Knowledge Engineering Review 8 1 (1993) 49-85
35. Garavan T.N., Morley M., Gunnigle P., and McGuire D. Human resource development and workplace learning: emerging theoretical perspectives and organisational practices. Journal of European Industrial Training 26 2, 3, 4 (2002) 60-71
36. Hockemeyer, C., Albert, D., 1999. The Adaptive Tutoring System RATH. In: Auer, M., Ressler, U. (Eds.), ICL99 Workshop Interactive Computer Aided Learning: Tools and Applications, Technikum Kärnten, Villach/Austria.
37. Jameson A. Numerical uncertainty management in user and student modeling: an overview of systems and issues. User Modeling and User-Adapted Interaction 5 4 (1996) 193-251
38. Jameson A. Adaptive interfaces and agents. In: Jacko J.A., and Sears A. (Eds). Human-Computer Interaction Handbook (2003), Erlbaum, Mahwah, NJ 305-330
39. Kline T. Psychological Testing. A Practical Approach to Design and Evaluation (2005), Sage, Thousand Oaks
40. Korossy K. Extending the theory of knowledge spaces: a competence-performance approach. Zeitschrift für Psychologie 205 1 (1997) 53-82
41. Korossy K. Modeling knowledge as competence and performance. In: Albert D., and Lukas J. (Eds). Knowledge Spaces: Theories, Empirical Research, and Applications (1999), Lawrence Erlbaum Associates, Mahwah, New Jersey 103-132
42. Korossy K., and Held T. Theory-based knowledge modeling in a subdomain of elementary algebra. Zeitschrift für Psychologie 209 3 (2001) 277-315
43. Ley T. Organizational Competency Management-A Competence-Performance Approach (2006), Shaker, Aachen
44. Ley T., and Albert D. Kompetenzmanagement als formalisierbare Abbildung von Wissen und Handeln für das Personalwesen. Wirtschaftspsychologie 5 3 (2003) 86-93
45. Ley T., and Albert D. Identifying employee competencies in dynamic work domains: methodological considerations and a case study. Journal of Universal Computer Science 9 12 (2003) 1500-1518
46. Ley T., Albert D., and Lindstaedt S.N. Competency management using the competence-performance approach: modeling, assessment, validation and use. In: Sicilia M.A. (Ed). Competencies in organizational eLearning: concepts and Tools (2006), Idea Group, Hersey 83-119
47. Ley, T., Kump, B., Ulbrich, A., Scheir, P., Lindstaedt, S.N., 2008. A competence-based approach for formalizing learning goals in work-integrated learning. In: Proceedings of the Ed Media 2008 World Conference on Educational Multimedia, Hypermedia & Telecommunications, Vienna, Austria, June 30-July 4, AACE, Chesapeake, VA, pp. 2099-2108.
48. Lievens F., Sanchez J.I., and De Corte W. Easing the inferential leap in competency modeling: the effects of task-related information and subject matter expertise. Personnel Psychology 57 (2004) 881-904
49. Lindstaedt S.N., Ley T., Scheir P., and Ulbrich A. Applying 'Scruffy' methods to enable work-integrated learning. Upgrade: The European Journal of the Informatics Professional 9 3 (2008) 44-50
50. Lindstaedt S.N., and Mayer H. A Storyboard of the APOSDLE Vision. In: Neidl W., and Tochtermann K. (Eds). Proceedings of the 1st European Conference on Technology-enhanced Learning (LNCS, vol. 4227) (2006), Springer, Heidelberg 628-633
51. Maiden, N.A., Jones, S.V., 2004. The RESCUE Requirements Engineering Process-An Integrated User-centered Requirements Engineering Process, Version 4.1. Centre for HCI Design, The City University, London/UK. Consulted March 4, 2008 〈http://hcid.soi.city.ac.uk/research/Rescuedocs/RESCUE_Process_Doc_v4_1.pdf〉.
52. Mayring P. Qualitative Inhaltsanalyse. Grundlagen und Techniken (2003), Beltz, Weinheim
53. Millán E., and Pérez-De-La-Cruz J. A bayesian diagnostic algorithm for student modeling and its evaluation. User Modeling and User-Adapted Interaction 12 2-3 (2002) 281-330
54. *NET OnLine. Consulted March 4, 2008. 〈http://online.onetcenter.org〉.
55. Nussbaumer, A., Steiner, C., Albert, D., 2008. Visualisation Tools for Supporting Self-Regulated Learning through Exploiting Competence Structures. In: Tochtermann, K., Maurer, H. (Eds.), Proceedings of I-Know 2008-International Conference on Knowledge Management, 3-5 September 2008, Know-Center, Graz/Austria, pp. 288-296.
56. Pammer, V., Scheir, P., Lindstaedt, S., 2007. Two Protégé plug-ins for supporting document-based ontology engineering and ontological annotation at document-level. Paper presented at the 10th International Protégé Conference, Budapest, Hungary, July 15-18, 2007, Consulted April 24, 2009 〈http://protege.stanford.edu/conference/2007/presentations/05.01_Pammer.pdf〉
57. Pilgerstorfer M., Albert D., and Camhy D.G. Considerations on personalized training in philosophy for children. In: Camhy D.G., and Born R. (Eds). Encouraging Philosophical Thinking: Proceedings of the International Conference on Philosophy for Children in Graz (2006), Academia Verlag, Austria 85-89
58. Rich E. Users are individuals: individualizing user models. International Journal of Human-Computer Studies 51 2 (1999) 323-338
59. Rospocher, M., Ghidini, C., Serafini, L., Kump, B., Pammer, V., Lindstaedt, S., Faatz, A., Ley, T., 2008. Collaborative enterprise integrated modeling. In: Gangemi, A., Keizer, J., Presutti, V., Stoermer, H. (Eds.), 5th International Workshop on Semantic Web Applications and Perspectives (SWAP 2008), December 15th to 17th, 2008, Rome, Italy, Consulted April 24, 2009 〈http://sunsite.informatik.rwth-aachen.de/Publications/CEUR-WS/Vol-426/〉
60. Schmidt, A., 2004. Context-steered learning: the learning in process approach. In: Proceedings of the IEEE Conference on Advanced Learning Technologies (ICALT 04). IEEE Computer Society, Washington, DC, pp. 684-686.
61. Schmitt N., and Chan D. Personnel Selection (1998), Sage, Thousand Oaks
62. Schreiber G., Akkermans H., Anjewierden A., Hoog R., Shadbolt N., Velde W.V.d., and Wielinga B.J. Knowledge Engineering and Management: the CommonKADS Methodology (1999), MIT Press, Cambridge, Mass
63. Shadbolt N., and Burton M.A. The empirical study of knowledge elicitation techniques. SIGART Newsletter 108 (1989) 15-18
64. In: Sicilia M., (Ed). Competencies in Organizational E-Learning: Concepts and Tools. Information Science Publishing, Hershey (2007)
65. Sicilia M. Ontology-based competency management: infrastructures for the knowledge-intensive learning organization. In: Lytras M., and Naeve A. (Eds). Intelligent Learning Infrastructures in Knowledge Intensive Organizations: A Semantic Web perspective (2005), Idea Group, Hershey 302-324
66. Sohn Y.W., and Doane. Evaluating Comprehension-based User Models: Predicting Individual User Planning and Action. User Modeling and User-Adapted Interaction 12 2-3 (2002) 171-205
67. Stefanutti, L., Albert, D., 2002. Efficient assessment of organizational action based on knowledge space theory. In: Tochtermann, K., Maurer, H. (Eds.), Proceedings of I-KNOW '02, 2nd International Conference on Knowledge Management, Know-Center, Graz, pp. 183-190.
68. Studer R., Benjamins V.R., and Fensel D. Knowledge engineering: principles and methods. Data & Knowledge Engineering 25 (1998) 161-197
69. Sure Y., Gómez-Pérez A., Daelemans W., Reinberger M., Guarino N., and Noy N.F. Why evaluate ontology technologies? Because it works!. IEEE Intelligent Systems 19 4 (2004) 74-81
70. Uschold M., King M., House R., Moralee S., and Zorgios Y. The enterprise ontology. The Knowledge Engineering Review 13 (1998) 31-89
71. van den Berg P.T. Competencies for work domains in business computer science. European Journal of Work and Organisational Psychology 7 4 (1998) 517-531
72. Villano M. Probabilistic student models: bayesian belief networks and knowledge space theory. In: Frasson C., Gauthier G., and McCalla G.I. (Eds). Intelligent Tutoring Systems (LNCS vol.608) (1992), Springer, Heidelberg 491-498
73. Witten I.H., and Frank E. Data mining: Practical Machine Learning Tools and Techniques (2005), Elsevier, Morgan Kaufman, Amsterdam