Assessment with microworlds using MicroDYN: Measurement invariance and latent mean comparisons - Psychometric properties across several student samples and blue-collar workers

European Journal of Psychological Assessment

Volumn 30, Issue 4, 2014, Pages 304-314

  Greiff, Samuel ^a   Wüstenberg, Sascha ^a  

^a UNIVERSITY OF LUXEMBOURG   (Luxembourg)

Author keywords

Assessment; Latent means; Measurement invariance; MicroDYN; Microworld

Indexed keywords

EID: 84909583175     PISSN: 10155759     EISSN: 21512426     Source Type: Journal    
DOI: 10.1027/1015-5759/a000194     Document Type: Article

References (40)

1. Asato, M. R., Sweeney, J. A., & Luna, B. (2006). Cognitive processes in the development of TOL performance. Neuropsychologia, 44, 2259-2269.
2. Bollen, K. A. (1989). Structural equations with latent variables. New York, NY: Wiley.
3. Bühner, M., Kröner, S., & Ziegler, M. (2008). Working memory, visual-spatial intelligence and their relationship to problem-solving. Intelligence, 36, 672-680.
4. Byrne, B. M., & Stewart, S. M. (2006). The MACS approach to testing for multigroup invariance of a second-order structure: A walk through the process. Structural Equation Modeling, 13, 287-321.
5. Cheung, G. W., & Rensvold, R. B. (2002). Evaluating good-ness- of-fit indexes for testing measurement invariance. Structural Equation Modeling, 9, 233-255.
6. Clariana, R., & Wallance, P. (2002). Paper-based versus computer-based assessment: Key factors associated with test mode effect. British Journal of Educational Technology, 33, 593-602.
7. Danner, D., Hagemann, D., Schankin, A., Hager, M., & Funke, J. (2011). Beyond IQ. A latent state trait analysis of general intelligence, dynamic decision making, and implicit learning. Intelligence, 39, 323-334.
8. Dörner, D., & Wearing, A. J. (1995). Complex problem solving: Toward a (computer-simulated) theory. In P. A. Frensch & J. Funke (Eds.), Complex problem solving: The European perspective (pp. 65-99). Hillsdale, NJ: Erlbaum.
9. French, B., & Finch, W. (2006). Confirmatory factor analytic procedures for the determination of measurement invariance. Structural Equation Modeling, 13, 378-402.
10. Funke, J. (2001). Dynamic systems as tools for analysing human judgement. Thinking and Reasoning, 7, 69-89.
11. Funke, J., & Frensch, P. A. (2007). Complex problem solving: The European perspective - 10 years after. In D. H. Jonassen (Ed.), Learning to solve complex scientific problems (pp. 25-47). New York, NY: Erlbaum.
12. Gardner, P. H., & Berry, D. C. (1995). The effect of different forms of advice on the control of a simulated complex system. Applied Cognitive Psychology, 9, 55-79.
13. Gonzalez, C., Thomas, R. P., & Vanyukov, P. (2005). The relationships between cognitive ability and dynamic decision making. Intelligence, 33, 169-186.
14. Gregorich, S. E. (2006). Do self-report instruments allow meaningful comparisons across diverse population groups? Testing measurement invariance using the confirmatory factor analysis framework. Medical Care, 44, 78-94.
15. Greiff, S., Wüstenberg, S., & Funke, J. (2012). Dynamic Problem Solving: A new measurement perspective. Applied Psychological Measurement, 36, 189-213.
16. Greiff, S., Wüstenberg, S., Molnár, G., Fischer, A., Funke, J., & Csapó, B. (2013). Complex problem solving in educational settings - something beyond g: Concept, assessment, measurement invariance, and construct validity. Journal of Educational Psychology, 105, 364-379.
17. Johnson, M., & Green, S. (2006). On-line mathematics assessment: The impact of mode on performance and question answering strategies. Journal of Technology, Learning, and Assessment, 4, 1-34.
18. Klahr, D., & Dunbar, K. (1988). Dual space search during scientific reasoning. Cognitive Science, 12, 1-48.
19. Kluge, A. (2008). Performance assessment with microworlds and their difficulty. Applied Psychological Measurement, 32, 156-180.
20. Kröner, S., Plass, J. L., & Leutner, D. (2005). Intelligence assessment with computer simulations. Intelligence, 33, 347-368.
21. Kyllonen, P. C. (2009). New constructs, methods, and directions for computer-based assessment. In F. Scheuermann & J. Björnsson (Eds.), The transition to computer-based assessment (pp. 151-156). Luxembourg: Office for Official Publications of the European Communities.
22. Mayer, R. E., & Wittrock, M. C. (2006). Problem solving. In P. A. Alexander & P. H. Winne (Eds.), Handbook of educational psychology (pp. 287-303). Mahwah, NJ: Erlbaum.
23. Muthén, B. O. (1998-2004). Mplus technical appendices. Los Angeles, CA: Muthén & Muthén.
24. Muthén, B. O., & Muthén, L. K. (2007). MPlus. Los Angeles, CA: Muthén & Muthén.
25. Muthén, L. K., & Muthén, B. O. (2010). Mplus user's guide (6th ed.). Los Angeles, CA: Muthén & Muthén.
26. Novick, L. R., & Bassok, M. (2005). Problem solving. In K. J. Holyoak & R. G. Morrison (Eds.), The Cambridge handbook of thinking and reasoning (pp. 321-349). Cambridge, NY: University Press.
27. OECD. (2010). PISA 2012 field trial problem solving framework Retrieved from http://www.oecd.org/dataoecd/8/42/46962005.pdf.
28. OECD. (2011). PISA 2009. Students on line. Digital technologies and performance. Paris: OECD Publishing.
29. Raven, J. (2000). Psychometrics, cognitive ability, and occupational performance. Review of Psychology, 7, 51-74.
30. Rigas, G., Carling, E., & Brehmer, B. (2002). Reliability and validity of performance measures in microworlds. Intelligence, 30, 463-480.
31. Rohde, T. E., & Thompson, L. A. (2007). Predicting academic achievement with cognitive ability. Intelligence, 35, 83-92.
32. Scheuermann, F., & Björnsson, J. (2009). The transition to computer-based assessment. Luxembourg: Office for Official Publications of the European Communities.
33. Schmidt, F. L., & Hunter, J. E. (1998). The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings. Psychological Bulletin, 124, 262-274.
34. Schweizer, F., Wüstenberg, S., & Greiff, S. (2013). Do complex problem solving dimensions measure something over and above working memory capacity? Learning and Individual Differences, 24, 42-52.
35. Sonnleitner, P., Brunner, M., Greiff, S., Funke, J., Keller, U., Martin, R., . . . Latour, T. (2012). The Genetics Lab. Acceptance and psychometric characteristics of a computer-based microworld to assess Complex Problem Solving. Psychological Test and Assessment Modeling, 54, 54-72.
36. Van der Linden, W. J., & Glas, C. A. W. (2000). Computerized adaptive testing. Dordrecht, The Netherlands: Kluwer.
37. Veenman, M. V. J., & Elshout, J. J. (1994). Differential effects of instructional support in simulation environments. Instructional Science, 22, 363-383.
38. Wenke, D., Frensch, P. A., & Funke, J. (2005). CPS and intelligence: Empirical relation and causal direction. In R. J. Sternberg & J. E. Pretz (Eds.), Cognition and intelligence: Identifying the mechanisms of the mind (pp. 160-187). New York, NY: Cambridge University Press.
39. Wüstenberg, S., Greiff, S., & Funke, J. (2012). Complex Problem Solving - More than reasoning? Intelligence, 40, 1-14.
40. Wüstenberg, S., Greiff, S., Molnár, G., & Funke, J. (2014). Cross-national gender differences in complex problem solving and their determinants. Learning and Individual Differences, 29, 18-29.