Measuring Performance in Dynamic Decision Making: Reliability and Validity of the Tailorshop Simulation

Journal of Individual Differences

Volumn 32, Issue 4, 2011, Pages 225-233

  Danner, Daniel ^a   Hagemann, Dirk ^a   Holt, Daniel V ^a   Hager, Marieke ^a   Schankin, Andrea ^a   Wüstenberg, Sascha ^a   Funke, Joachim ^a  

^a UNIVERSITY OF HEIDELBERG   (Germany)

Author keywords

Complex problem solving; Dynamic decision making; Reliability; Tailorshop; Validity

Indexed keywords

EID: 80054895658     PISSN: 16140001     EISSN: None     Source Type: Journal    
DOI: 10.1027/1614-0001/a000055     Document Type: Article

References (31)

1. Baker E.L., O'Neil H.F. Measuring problem solving in computer environments: Current and future states. Computers in Human Behavior 2002, 18:609-622. 10.1016/S0747-5632(02)00019-5.
2. Barth C.M., Funke J. Negative affective environments improve complex solving performance. Cognition and Emotion 2010, 24:1259-1268. 10.1080/02699930903223766.
3. Dörner D. On the difficulty people have in dealing with complexity. Simulation and Gaming 1980, 11:87-106.
4. Dörner D., Kreuzig H.W. Problemlösefähigkeit und Intelligenz [Problem solving ability and intelligence]. Psychologische Rundschau 1983, 34:185-192.
5. Dörner D., Kreuzig H.W., Reither F., Stäudel T. Lohausen. Vom Umgang mit Unbestimmtheit und Komplexität] Lohausen. On dealing with uncertainty and complexity[ 1983, Hans Huber, Bern.
6. Funke J. Einige Bemerkungen zu Problemen der Problemlöseforschung oder: Ist Testintelligenz doch ein Prädiktor [Issues in problem solving research: Is test intelligence a predictor after all?]. Diagnostica 1983, 29:283-302.
7. Gonzalez C., Vanyukov P., Martin M.K. The use of microworlds to study dynamic decision making. Computers in Human Behavior 2005, 21:273-286. 10.1016/j.chb.2004.02.014.
8. Higgins D.M., Peterson J.B., Pihl R.O., Lee A.G.M. Prefrontal cognitive ability, intelligence, Big Five personality, and the prediction of advanced academic and workplace performance. Journal of Personality and Social Psychology 2007, 93:298-319. 10.1037/0022-3514.93.2.298.
9. Kersting M. Zur Konstrukt- und Kriteriumsvalidität von Problemlöseszenarien anhand der Vorhersage von Vorgesetztenurteilen über die berufliche Bewährung [On the construct and criterion validity of problem-solving scenarios based on the prediction of supervisor assessment of job performance]. Diagnostica 2001, 47:67-76. 10.1026//0012-1924.47.2.67.
10. Kluwe R.H., Misiak C., Haider H. The control of complex systems and performance in intelligence tests. Intelligence: Reconceptualization and measurement 1991, 227-244. Erlbaum, Hillsdale, NJ. H.A.H. Rowe (Ed.).
11. Kröner S., Plass J.L., Leutner D. Intelligence assessment with computer simulations. Intelligence 2005, 33:347-368. 10.1016/j.intell.2005.03.002.
12. Lord F.M., Novick M.R. Statistical theories of mental test scores 1968, Addison-Wesley, Oxford.
13. Meng X., Rosenthal R., Rubin D.B. Comparing correlated correlation coefficients. Psychological Bulletin 1992, 111:172-175. 10.1037/0033-2909.111.1.172.
14. Muthén L.K., Muthén B.O. Mplus user's guide 2007, Muthén & Muthén, Los Angeles, CA. 5th ed.
15. Putz-Osterloh W. Über die Beziehung zwischen Testintelligenz und Problemlöseerfolg [The relationship between test intelligence and problem solving success]. Zeitschrift für Psychologie mit Zeitschrift für Angewandte Psychologie 1981, 189:79-100.
16. Putz-Osterloh W., Lüer G. The predictability of complex problem solving by performance on an intelligence test. Zeitschrift für Experimentelle und Angewandte Psychologie 1981, 28:309-334.
17. Putz-Osterloh W., Bott B., Köster K. Modes of learning in problem solving: Are they transferable to tutorial systems?. Computers in Human Behavior 1990, 6:83-96. 10.1016/0747-5632(90)90032-c.
18. Raven J.C., Court J.H., Raven J. Manual for Raven's Progressive Matrices and Mill Hill Vocabulary Scales. Advanced progressive matrices 1994, Oxford Psychologists Press, Oxford.
19. Rigas G., Carling E., Brehmer B. Reliability and validity of performance measures in microworlds. Intelligence 2002, 30:463-480. 10.1016/s0160-2896(02)00121-6.
20. Roszkowski M.J., Grable J.E. Gender differences in personal income and financial risk tolerance: How much of a connection?. The Career Development Quarterly 2010, 58:270-275.
21. Steyer R. Models of classical psychometric test theory as stochastic measurement models: Representation, uniqueness, meaningfulness, identifiability, and testability. Methodika 1989, 3:25-60.
22. Strohschneider S. Zur Stabilität und Validität von Handeln in komplexen Realitätsbereichen [On the stability and validity of complex problem-solving behavior]. Sprache & Kognition 1986, 5:42-48.
23. Süß H.-M. Intelligenz, Wissen und Problemlösen [Intelligence, knowledge and problem-solving 1996, Hogrefe, Göttingen.
24. Süß H.-M. Intelligenz und komplexes Problemlösen: Perspektiven für eine Kooperation zwischen differentiell-psychometrischer und kognitionspsychologischer Forschung [Intelligence and complex problem solving. Perspectives for a cooperation between differential-psychometric and cognition-psychological research]. Psychologische Rundschau 1999, 50:220-228. 10.1026//0033-3042.50.4.220.
25. Süß H.-M., Kersting M., Oberauer K. Zur Vorhersage von Steuerungsleistungen an computersimulierten Systemen durch Wissen und Intelligenz [On the predictability of control terformance on computer-simulated systems by knowledge and intelligence]. Zeitschrift für Differentielle und Diagnostische Psychologie 1993, 14:189-203.
26. Wirth J., Funke J. Dynamisches Problemlösen: Entwicklung und Evaluation eines neuen Messverfahrens zum Steuern komplexer Systeme. Problemlösekompetenz von Schülerinnen und Schülern 2005, 55-72. Verlag für Sozialwissenschaften, Wiesbaden. E. Klieme, D. Leutner, J. Wirth (Eds.).
27. Wirth J., Klieme E. Computer-based Assessment of Problem Solving Competence. Assessment in Education: Principles, Policy and Practice 2003, 10:329-345. 10.1080/0969594032000148172.
28. Wittmann W. Multivariate reliability theory: Principles of symmetry and successful validation strategies. Handbook of multivariate experimental psychology 1988, 505-560. Plenum, New York. 2nd ed. J.R. Cattell (Ed.).
29. Wittmann W., Hattrup K. The relationship between performance in dynamic systems and intelligence. Systems Research and Behavioral Science 2004, 21:393-409. 10.1002/sres.653.
30. Wittmann W.W., Süß H.-M. Investigating the paths between working memory, intelligence, knowledge, and complex problem-solving performances via Brunswik symmetry. Learning and individual differences: Process, trait, and content determinants 1999, 77-108. American Psychological Association, Washington, DC. P.L. Ackerman, P.C. Kyllonen, R.D. Roberts (Eds.).
31. Zaccaro S.J., Mumford M.D., Connelly M.S., Marks M.A., Gilbert J.A. Assessment of leader problem-solving capabilities. The Leadership Quarterly 2000, 11:37-64. 10.1016/S1048-9843(99)00042-9.