Extending the assessment of complex problem solving to finite state automata: Embracing heterogeneity

European Journal of Psychological Assessment

Volumn 31, Issue 3, 2015, Pages 181-194

  Neubert, Jonas C ^a   Kretzschmar, Andre ^a   Wustenberg, Sascha ^a   Greiff, Samuel ^a  

^a UNIVERSITY OF LUXEMBOURG   (Luxembourg)

Author keywords

Complex problem solving; Finite state automata; MicroDYN; MicroFIN; Multitrait multimethod

Indexed keywords

EID: 84938328240     PISSN: 10155759     EISSN: 21512426     Source Type: Journal    
DOI: 10.1027/1015-5759/a000224     Document Type: Article

References (45)

1. Ackerman, P. L. (1992). Predicting individual differences in complex skill acquisition: Dynamics of ability determinants. Journal of Applied Psychology, 77, 598-614.
2. Anderson, J. A. (2006). Automata theory with modern applications. Cambridge, MA: Cambridge University Press.
3. Beckmann, J. F., & Goode, N. (2013). The benefit of being nave and knowing it: The unfavourable impact of perceived context familiarity on learning in complex problem solving tasks. Instructional Science, 42, 1-20.
4. Brehmer, B. (1992). Dynamic decision making: Human control of complex systems. Acta Psychologica, 81, 211-241.
5. Buchner, A. (1995). Basic topics and approaches to the study of complex problem solving. In P. A. Frensch & J. Funke (Eds.), Complex problem solving: The European perspective (pp. 27-63). Hillsdale, NJ: Erlbaum.
6. Buchner, A., & Funke, J. (1993). Finite-state automata: Dynamic task environments in problem-solving research. The Quarterly Journal of Experimental Psychology, 46, 83-118.
7. Bhner, M., Krçner, S., & Ziegler, M. (2008). Working memory, visual-spatial-intelligence and their relationship to problem-solving. Intelligence, 36, 672-680.
8. Dçrner, D., Kreuzig, H. W., Reither, F., & Studel, T. (Eds.). (1983). Lohausen. Vom Umgang mit Unbestimmtheit und Komplexitt [Lohhausen. On dealing with uncertainty and complexity]. Bern, Switzerland: Huber.
9. Eid, M., Lischetzke, T., Nussbeck, F. W., & Trierweiler, L. I. (2003). Separating trait effects from trait-specific method effects in multitrait-multimethod models: A multipleindicator CT-C(M-1) model. Psychological Methods, 8, 38-60.
10. Fischer, A., Greiff, S., & Funke, J. (2012). The process of solving complex problems. Journal of Problem Solving, 4, 19-42.
11. Funke, J. (2001). Dynamic systems as tools for analysing human judgement. Thinking & Reasoning, 7, 69-89.
12. Funke, J. (2010). Complex problem solving: A case for complex cognition? Cognitive Processing, 11, 133-142.
13. Gonzalez, C., Vanyukov, P., & Martin, M. K. (2005). The use of microworlds to study dynamic decision making. Computers in Human Behavior, 21, 273-286.
14. Greiff, S., Fischer, A., Wstenberg, S., Sonnleitner, P., Brunner, M., & Martin, R. (2013). A multitrait-multimethod study of assessment instruments for complex problem solving. Intelligence, 41(5), 579-596.
15. Greiff, S., & Funke, J. (2009). Measuring complex problem solving: the MicroDYN approach. In F. Scheuermann & J. Bjçrnsson (Eds.), The transition to computer-based assessment: New approaches to skills assessment and implications for large-scale testing (pp. 157-163). Luxembourg: Office for Official Publications of the European Communities.
16. Greiff, S., & Wstenberg, S. (2014). Assessment with microworlds using MicroDYN: Measurement invariance and latent mean comparisons: Psychometric properties across several student samples and blue-collar workers. European Journal of Psychological Assessment. Advance online publication. doi: 10.1027/1015-5759/a000194
17. Greiff, S., Wstenberg, S., & Funke, J. (2012). Dynamic problem solving: A new assessment perspective. Applied Psychological Measurement, 36, 189-213.
18. Greiff, S., Wstenberg, S., Molnr, G., Fischer, A., Funke, J., & Csap, B. (2013). Complex problem solving in educational contexts -something beyond g: Concept, assessment, measurement invariance, and construct validity. Journal of Educational Psychology, 105, 364-379.
19. Griffin, P., McGaw, B., & Care, E. (Eds.). (2012). Assessment and teaching of 21st century skills.New York, NY: Springer.
20. Heller, K. A., & Perleth, C. (2000). Kognitiver Fähigkeitstest für 4. bis 12. Klassen, Revision [Cognitive Abilities Test (CogAT; Thorndike, L. & Hagen, E., 1954-1986) -German adapted version]. Gçttingen, Germany: Beltz-Test.
21. Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6, 1-55.
22. Levy, S. T., & Wilensky, U. (2011). Mining students inquiry actions for understanding of complex systems. Computers & Education, 56, 556-573.
23. Little, T. D., Cunningham, W. A., Shahar, G., & Widaman, K. F. (2002). To parcel or not to parcel: Exploring the question, weighing the merits. Structural Equation Modeling: A Multidisciplinary Journal, 9, 151-173.
24. McElhaney, K. W., & Linn, M. C. (2011). Investigations of a complex, realistic task: Intentional, unsystematic, and exhaustive experimenters. Journal of Research in Science Teaching, 48, 745-770.
25. Muthn, B. O., du Toit, S. H. C., & Spisic, D. (1997). Robust inference using weighted least squares and quadratic estimating equations in latent variable modeling with categorical and continuous outcomes. Unpublished technical report. Retrieved from http://pages.gseis.ucla.edu/faculty/muthen/articles/Article075.pdf
26. Muthn, L. K., & Muthn, B. O. (2012). Mplus users guide (7th ed.). Los Angeles, CA: Muthn & Muthn.
27. Novick, L. R., & Bassok, M. (2005). Problem solving. In K. J. Holyoak & R. G. Morrison (Eds.), The Cambridge handbook of thinking and reasoning (pp. 321-349). New York, NY: Cambridge University Press.
28. Novick, M. R. (1966). The axioms and principal results of classical test theory. Journal of Mathematical Psychology, 3, 1-18.
29. OECD. (2013). PISA 2012 assessment and analytical framework. Paris, France: Organisation for Economic Cooperation and Development.
30. OECD. (2014). PISA 2012 Results: Creative problem solving (Volume V). Paris, France: OECD Publishing.
31. Osman, M. (2010). Controlling uncertainty: A review of human behavior in complex dynamic environments. Psychological Bulletin, 136, 65-86.
32. Raven, J., Raven, J. C., & Court, J. H. (1998). Manual for Ravens progressive matrices and vocabulary scales. Oxford, UK: Oxford Psychologists Press.
33. Rich, E. (2008). Automata, computability and complexity: Theory and applications. Upper Saddle River, NJ: Prentice Hall.
34. Rollett, W. (2008). Strategieeinsatz, erzeugte Information und Informationsnutzung bei der Exploration und Steuerung komplexer dynamischer Systeme [Use of strategy, generated information and use of information when exploring and controlling complex dynamic systems]. Berlin, Germany: Lit.
35. Scherer, R., & Tiemann, R. (2012). Factors of problem-solving competency in a virtual chemistry environment: The role of metacognitive knowledge about strategies. Computers & Education, 59, 1199-1214.
36. Schweizer, F., Wstenberg, S., & Greiff, S. (2013). Validity of the MicroDYN approach: Complex problem solving predicts school grades beyond working memory capacity. Learning and Individual Differences, 24, 42-52.
37. Segerer, R., Marx, A., & Marx, P. (2012). Unlçsbare Items im KFT 4-12+R [Unsolvable items in the CFT 4-12+R]. Diagnostica, 58, 45-50.
38. Sonnleitner, P., Brunner, M., Greiff, S., Funke, J., Keller, U., Martin, R., . . . Latour, T. (2012). The Genetics Lab: Acceptance and psychometric characteristics of a computer-based microworld assessing complex problem solving. Psychological Test and Assessment Modeling, 54, 54-72.
39. Sonnleitner, P., Keller, U., Martin, R., & Brunner, M. (2013). Students complex problem-solving abilities: Their structure and relations to reasoning ability and educational success. Intelligence, 41, 289-305.
40. Sternberg, R. J., & Frensch, P. A. (1991). Complex problem solving: Principles and mechanisms. Hillsdale, NJ: Erlbaum.
41. Tschirgi, J. E. (1980). Sensible reasoning: A hypothesis about hypotheses. Child Development, 51, 1-10.
42. Vollmeyer, R., Burns, B. D., & Holyoak, K. J. (1996). The impact of goal specificity on strategy use and the acquisition of problem structure. Cognitive Science, 20, 75-100.
43. Wittmann, W., & S, H.-M. (1999). Investigating the paths between working memory, intelligence, knowledge, and complex problem-solving performances via Brunswik symmetry. In P. L. Ackerman, P. C. Kyllonen, & R. D. Roberts (Eds.), Learning and individual differences: Process, trait, and content determinants (pp. 77-108). Washington, DC: American Psychological Association.
44. Wstenberg, S., Greiff, S., & Funke, J. (2012). Complex problem solving -More than reasoning? Intelligence, 40, 1-14.
45. Zinbarg, R. E., Revelle, W., Yovel, I., & Li, W. (2005). Cronbachs a, Revelles b, and Mcdonalds xH: Their relations with each other and two alternative conceptualizations of reliability. Psychometrika, 70, 123-133.