Cross-validation study of methods and technologies to assess mental models in a complex problem solving situation

Computers in Human Behavior

Volumn 28, Issue 2, 2012, Pages 703-717

  Kim, Min Kyu ^a  

^a UNIVERSITY OF GEORGIA   (United States)

Author keywords

Assessment technology; Concept map; Mental models; Problem solving; Validation study

Indexed keywords

ASSESSMENT METHODS; BENCHMARK MODELS; COMPLEX PROBLEM SOLVING; CONCEPT MAPS; CROSS VALIDATION; MENTAL MODEL; MENTAL MODELS; NATURAL LANGUAGES; THREE MODELS; VALIDATION STUDY; WRITTEN TEXTS;

RATING; TECHNOLOGY;

PROBLEM SOLVING;

EID: 84855531758     PISSN: 07475632     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.chb.2011.11.018     Document Type: Article

References (63)

1. R. Axelrod Structure of decision: the cognitive maps of political elites 1976 Princeton University Press Princeton, NJ
2. A.L. Brown Design experiments: Theoretical and methodological challenges in creating complex interventions in classroom settings The Journal of the Learning Sciences 2 1992 141 178
3. K. Carley, and M. Palmquist Extracting, representing, and analyzing mental models Social Forces 70 1992 601 636
4. M.T.H. Chi, R. Glaser, and M.J. Farr The nature of expertise 1988 L. Erlbaum Associates Hillsdale, NJ
5. Clariana, R. B. (2010). Multi-decision approaches for eliciting knowledge structure. In Computer-based diagnostics and systematic analysis of knowledge (pp. 41-59).
6. R.B. Clariana, and P. Wallace A computer-based approach for deriving and measuring individual and team knowledge structure from essay questions Journal of Educational Computing Research 37 2007 211 227
7. R.B. Clariana, and P. Wallace A comparison of pair-wise, list-wise, and clustering approaches for eliciting structural knowledge in information systems courses International Journal of Instructional Media 36 2009 287 302
8. R. Clariana, P. Wallace, and V. Godshalk Deriving and measuring group knowledge structure from essays: The effects of anaphoric reference Educational Technology Research and Development 57 2009 725 737
9. A.M. Collins, and E.F. Loftus A spreading-activation theory of semantic processing Psychological Review 82 1975 407 428
10. Colman, A. M.; Shafir, E.; Tversky, Amos (2008). In Koertge, N. (Ed.), New dictionary of scientific biography (pp. 91-97). Farmington Hills, MI: Charles Scribner's Sons.
11. K.A. Coronges, A.W. Stacy, and T.W. Valente Structural comparison of cognitive associative networks in two populations Journal of Applied Social Psychology 37 2007 2097 2129
12. M.B. Curtis, and M.A. Davis Assessing knowledge structure in accounting education: an application of Pathfinder Associative Networks Journal of Accounting Education 21 2003 185 195 (Pubitemid 36946064)
13. F. Dochy, M. Segers, P. Van den Bossche, and D. Gijbels Effects of problem-based learning: a meta-analysis Learning and Instruction 13 2003 533 568 (Pubitemid 36831740)
14. A. Garnham Mental models as representations of discourse and text 1987 Halsted Press New York
15. A. Garnham Mental models and the interpretation of anaphora 2001 Taylor & Francis Philadelphia
16. D. Gijbels, F. Dochy, P. Van den Bossche, and M. Segers Effects of problem-based learning: A meta-analysis from the angle of assessment Review of Educational Research 75 2005 27 61 (Pubitemid 40747057)
17. T.E. Goldsmith, P.J. Johnson, and W.H. Acton Assessing structural knowledge Journal of Educational Psychology 83 1991 88 96
18. T.E. Goldsmith, and K. Kraiger Applications of structural knowledge assessment to training evaluation J.K. Ford, S.W.J. Kozlowski, Improving training effectiveness in work organizations 1997 Psychology Press Mahwah, NJ 73 95
19. C.M. Goodman The Delphi technique: a critique Journal of Advanced Nursing 12 1987 729 734
20. J.G. Greeno Situations, mental models and generative knowledge H.A. Simon, D. Klahr, K. Kotovsky, Complex information processing: The impact of Herbert a. simon 1989 Lawrence Erlbaum Associates, Publishers Hillsdale, NJ 285 318
21. P. Hage, and F. Harary Structural models in anthropology 1983 Cambridge University Press Cambridge, MA
22. C. Hsu, and B.A. Sandford The Delphi technique: making sense of consensus Practical Assessment Research & Evaluation 12 2007
23. K.A. Hutchison Is semantic priming due to association strength or feature overlap? A microanalytic review Psychonomic Bulletin & Review 10 2003 785 813 (Pubitemid 38483379)
24. D. Ifenthaler Relational structural and semantic analysis of graphical representations and concept maps Educational Technology Research and Development 2008
25. D. Ifenthaler, I. Masduki, and N.M. Seel The mystery of cognitive structure and how we can detect it: tracking the development of cognitive structures over time Instructional Science 2009
26. Jeong, A. C. (2008). jmap.
27. P.N. Johnson-Laird The history of mental models K.I. Manktelow, M.C. Chung, Psychology of reasoning: theoretical and historical perspectives 2005 Psychology Press New York, NY 179 212
28. P.N. Johnson-Laird Mental models and thoughts K.J. Holyoak, The Cambridge handbook of thinking and reasoning 2005 Cambridge University Press Cambridge, Mass 185 208
29. Jonassen, D. H.; Beissner, K.; & Yacci, M. (1993) Structural knowledge: Techniques for representing, conveying, and acquiring structural knowledge.
30. S. Kalyuga Assessment of learners' organised knowledge structures in adaptive learning environments Applied Cognitive Psychology 20 2006 333 342 (Pubitemid 43676200)
31. Kim, M. K. (2011). Development of an assessment technology for measuring knowledge structures using natural language response to a complex problem scenario.
32. D. Knoke, and J.H. Kuklinski Network analysis 1982 Sage Publications Newbury Park
33. J.R. McClure, B. Sonak, and H.K. Suen Concept map assessment of classroom learning: Reliability, validity, and logistical practicality Journal of Research in Science Teaching 36 1999 475 492 (Pubitemid 129529647)
34. P.R. Monge, and N.S. Contractor Theories of communication networks 2003 Oxford University Press New York, NY
35. V.K. Narayanan Causal mapping: An historical overview V.K. Narayanan, D.J. Armstrong, Causal mapping for research in information technology 2005 Idea Group Inc (IGI) Hershey, PA 1 19
36. A. Newell Unified theories of cognition 1994 Harvard University Press Cambridge, MA
37. A. Newell, and H.A. Simon Human problem solving 1972 Prentice-Hall Englewood Cliffs, NJ
38. Novak, J. D.; & Cañas, A. J. (2006). The origins of the concept mapping tool and the continuing evolution of the tool.
39. O'Connor, D. L. & Johnson, T. E. (2004). Measuring team cognition: Concept mapping elicitation as a means of constructing team shared mental models in an applied setting. In Pamplona, Spain.
40. C. Okoli, and S.D. Pawlowski The Delphi method as a research tool: an example, design considerations and applications Information & Management 42 2004 15 29
41. P. Pirnay-Dummer, and D. Ifenthaler Automated knowledge visualization and assessment D. Ifenthaler, N.M. Seel, P. Pirnay-Dummer, Computer-based diagnostics and systematic analysis of knowledge 2010 Springer New York, NY 77 116
42. P. Pirnay-Dummer, D. Ifenthaler, and J. Spector Highly integrated model assessment technology and tools Educational Technology Research and Development 2009
43. Quillian Semantic memory M.L. Minsky, Semantic information processing 1985 The MIT Press Cambridge, MA 216 270
44. Ruiz-Primo, M. A.; Schultz, S.; Li, M.; Shavelson, R. J. (1999). On the cognitive validity of interpretations of scores from alternative concept mapping techniques. University of California, National Center for Research on Evaluation, Standards, and Student Testing (CRESST).
45. Rupp, A. A.; Sweet, S.; & Choi, Y. (2010). Modeling learning trajectories with epistemic network analysis: A simulation-based investigation of a novel analytic method for epistemic games.
46. A.A. Rupp, M. Gushta, R.J. Mislevy, and D.W. Shaffer Evidence-centered design of epistemic games: Measurement principles for complex learning environments The Journal of Technology Learning and Assessment 8 2010
47. Schlomske, N.; & Pirnay-Dummer, P. (2008). Model based assessment of learning dependent change during a two semester class. In Kinshuk, Sampson, D.; Spector, J. M. (Eds.), IADIS international conference cognition and exploratory learning in digital age 2008. Freiburg, Germany (pp. 45-53).
48. Schvaneveldt, R. W. (Ed.) (1990). Pathfinder associative networks: studies in knowledge organization. Norwood, NJ: Ablex Publishing Corp.
49. R. Schvaneveldt, F.T. Durso, and D.W. Dearholt Network structures in proximity data G.H. Bower, The psychology of learning and motivation: Advances in research and theory 1989 Academic Press San Diego, CA 249 284
50. R.W. Schvaneveldt, F.T. Durso, T.E. Goldsmith, T.J. Breen, N.M. Cooke, and R.G. Tucker Measuring the structure of expertise International Journal of Man-Machine Studies 23 1985 699 728 (Pubitemid 16103298)
51. N.M. Seel Educational diagnosis of mental models: Assessment problems and technology-based solutions Journal of Structural Learning & Intelligent Systems 14 1999 153
52. N.M. Seel Epistemology, situated cognition, and mental models: "Like a bridge over troubled water Instructional Science 29 2001 403 427
53. M.S.R. Segers An alternative for assessing problem-solving skills: The overall test Studies in Educational Evaluation 23 1997 373 398 (Pubitemid 127169614)
54. D.W. Shaffer, D. Hatfield, G.N. Svarovsky, P. Nash, A. Nulty, and E. Bagley Epistemic network analysis: a prototype for 21st-century assessment of learning International Journal of Learning and Media 1 2009 33 53
55. Shute, V. J.; Jeong, A. C.; Spector, J. M.; Seel, N. M.; & Johnson, T. E. (2009). Model-based methods for assessment, learning, and instruction: Innovative educational technology at Florida State University. In Orey, M. (Ed.), Educational media and technology yearbook. New York, NY: The Greenwood Publishing Group.
56. V.J. Shute, and D. Zapata-Rivera Using an evidence-based approach to assess mental models D. Ifenthaler, P. Pirnay-Dummer, J.M. Spector, Understanding models for learning and instruction 2008 Springer US, Boston, MA 23 41
57. Spector, J.; & Koszalka, T. A. (2004). The DEEP methodology for assessing learning in complex domains. Syracuse, NY: Syracuse University.
58. J.M. Spector, V.P. Dennen, and T.A. Koszalka Causal maps, mental models and assessing acquisition of expertise Technology, Instruction, Cognition and Learning 3 2006 167 183
59. E.M. Taricani, and R.B. Clariana A technique for automatically scoring open-ended concept maps Educational Technology Research & Development 54 2006 65 82 (Pubitemid 43587832)
60. R.M. Thomas High-stakes testing: Coping with collateral damage 2005 Psychology Press Mahwah, NJ
61. A. Tversky Features of similarity Psychological Review 84 1977 327 352
62. A. Tversky, and E. Shafir Preference, belief, and similarity: Selected writings 1st ed. 2003 The MIT Press London, England
63. S. Wasserman, and K. Faust Social network analysis: methods and applications 1994 Cambridge University Press Cambridge, MA