Complex problem solving in educational contexts-something beyond g: Concept, assessment, measurement invariance, and construct validity

Journal of Educational Psychology

Volumn 105, Issue 2, 2013, Pages 364-379

  Greiff, Samuel ^a   Wüstenberg, Sascha ^a   Molnár, Gyöngyvér ^b   Fischer, Andreas ^c   Funke, Joachim ^c   Csapó, Beno ^b  

^a UNIVERSITY OF LUXEMBOURG   (Luxembourg)

^b UNIVERSITY OF SZEGED   (Hungary)

^c UNIVERSITY OF HEIDELBERG   (Germany)

Author keywords

Complex problem solving; Education; General mental ability; Intelligence; MicroDYN

Indexed keywords

EID: 84878645531     PISSN: 00220663     EISSN: None     Source Type: Journal    
DOI: 10.1037/a0031856     Document Type: Article

References (77)

1. Alspaugh, J. W., & Harting, R. D. (1995). Transition effects of school grade-level organization on student achievement. Journal of Research & Development in Education, 28(3), 145-149.
2. American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (1999). Standards for educational and psychological testing (3rd ed). Washington, DC: AERA.
3. Benson, N., Hulac, D. M., & Kranzler, J. H. (2010). Independent examination of the Wechsler Adult Intelligence Scale-Fourth Edition (WAIS-IV): What does the WAIS-IV measure. Psychological Assessment, 22(1), 121-130. doi:10.1037/a0017767
4. Binet, A., & Simon, T. (1904). Méthodes nouvelles pour le diagnostic du niveau intellectuel des anomaux [New methods for assessing the intellectual level of anormal individuals]. L'Année Psychologique, 11, 191-244. doi:10.3406/psy.1904.3675
5. Bollen, K. A. (1989). Structural equations with latent variables. New York: Wiley.
6. Buchner, A. (1995). Basic topics and approaches to the study of complex problem solving. In P. A. Frensch & J. Funke (Eds.), Complex problem solving: The European perspective (pp. 27-63). Hillsdale, NJ: Erlbaum.
7. Byrne, B. M., & Stewart, S. M. (2006). The MACS approach to testing for multigroup invariance of a second-order structure: A walk through the process. Structural Equation Modeling, 13(2), 287-321. doi:10.1207/s15328007sem1302_7
8. Byrnes, J. P. (2001). Minds, brains, and education. New York, NY: Guilford Press.
9. Carroll, J. B. (2003). The higher-stratum structure of cognitive abilities: Current evidence supports g and about ten broad factors. In H. Nyborg (Ed.), The scientific study of general intelligence: Tribute to Arthur R. Jensen (pp. 5-21). Amsterdam, the Netherlands: Pergamon.
10. Chen, F. F. (2008). What happens if we compare chopsticks with forks. The impact of making inappropriate comparisons in cross-cultural research. Journal of Personality and Social Psychology, 95, 1005-1018. doi:10.1037/a0013193
11. Chen, Z., & Klahr, D. (1999). All other things being equal: Acquisition and transfer of the control of variables strategy. Child Development, 70(5), 1098-1120. doi:10.1111/1467-8624.00081
12. Cheung, G. W., & Rensvold, R. B. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling, 9, 233-255. doi:10.1207/S15328007SEM0902_5
13. Danner, D., Hagemann, D., Schankin, A., Hager, M., & Funke, J. (2011). Beyond IQ: A latent state trait analysis of general intelligence, dynamic decision making, and implicit learning. Intelligence, 39(5), 323-334. doi:10.1016/j.intell.2011.06.004
14. Davis-Kean, P. E. (2005). The influence of parent education and family income on child achievement. The indirect role of parental expectations and the home environment. Journal of Family Psychology, 19(2), 294-304. doi:10.1037/0893-3200.19.2.294
15. Deary, I. J., Whalley, L. J., & Crawford, J. R. (2004). An "instantaneous" estimate of a lifetime's cognitive change. Intelligence, 32, 113-119. doi:10.1016/j.intell.2003.06.001
16. Diaz, L., & Heining-Boynton, A. L. (1995). Multiple intelligences, multiculturalism, and the teaching of culture. International Journal of Educational Research, 23, 607-617. doi:10.1016/0883-0355(96)80440-X
17. Dörner, D. (1986). Diagnostik der operativen Intelligenz [Assessment of operative intelligence]. Diagnostica, 32(4), 290-308.
18. Dörner, D. (1990). The logic of failure. In D. E. Broadbent, J. T. Reason, & A. D. Baddeley (Eds.), Human factors n hazardous situations (pp. 15-36). New York, NY: Oxford University Press.
19. Dörner, D., & Kreuzig, W. (1983). Problemlösefähigkeit und Intelligenz [Problem solving ability and intelligence]. Psychologische Rundschau, 34, 185-192.
20. Duncker, K. (1945). On problem solving. Psychological Monographs, 58(5, Whole No. 270).
21. Fischer, A., Greiff, S., & Funke, J. (2012). The process of solving complex problems. Journal of Problem Solving, 4(1), 19-42.
22. Funke, J. (2001). Dynamic systems as tools for analysing human judgement. Thinking & Reasoning, 7(1), 69-89. doi:10.1080/13546780042000046
23. Funke, J. (2010). Complex problem solving: A case for complex cognition. Cognitive Processing, 11, 133-142. doi:10.1007/s10339-009-0345-0
24. Funke, J., & Frensch, P. A. (2007). Complex problem solving: The Euro- pean perspective-10 years after. In D. H. Jonassen (Ed.), Learning to solve complex scientific problems (pp. 25-47). New York, NY: Erlbaum.
25. Gardner, P. H., & Berry, D. C. (1995). The effect of different forms of advice on the control of a simulated complex system. Applied Cognitive Psychology, 9(7), S55-S79. doi:10.1002/acp.2350090706
26. Goleman, D. (1995). Emotional intelligence: Why it can matter more than IQ. New York, NY: Bantam Books.
27. Greeno, J. G., & Simon, H. A. (1988). Problem solving and reasoning. In R. C. Atkinson, R. J. Herrnstein, G. Lindzey, & R. D. Luce (Eds.), Stevens' handbook of experimental psychology (2nd ed., Vol. 2, pp. 589-672). Hillsdale, NJ: Erlbaum.
28. Greiff, S. (2012). Individualdiagnostik der Problemlösefähigkeit [Diagnostics of problem solving ability on an individual level]. Münster, Germany: Waxmann.
29. Greiff, S., Wüstenberg, S., & Funke, J. (2012). Dynamic problem solving: A new measurement perspective. Applied Psychological Measurement, 36(3), 189-213. doi:10.1177/0146621612439620
30. Gustafsson, J. E. (1984). A unifying model for the structure of intellectual abilities. Intelligence, 8, 179-203. doi:10.1016/0160-2896(84)90008-4
31. Holyoak, K. J. (1985). The pragmatics of analogical transfer. In G. H. Bower (Ed.), The psychology of learning and motivation (pp. 59-87). New York, NY: Academic Press.
32. Hu, L., & Bentler, P. M. (1999). Cutoffcriteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1-55. doi:10.1080/10705519909540118
33. Jensen, A. R. (1998). The g factor and the design of education. In R. J. Sternberg & W. M. Williams (Eds.), Intelligence, instruction, and assessment (pp. 111-131). Mahwah, NJ: Erlbaum.
34. Kihlstrom, J. F., & Cantor, N. (2011). Social intelligence. In R. J. Sternberg & S. C. Barry (Eds.), The Cambridge handbook of intelligence (pp. 564-581). New York, NY: Cambridge University Press. doi:10.1017/CBO9780511977244.029
35. Kröner, S., Plass, J. L., & Leutner, D. (2005). Intelligence assessment with computer simulations. Intelligence, 33(4), 347-368. doi:10.1016/j.intell.2005.03.002
36. Kyllonen, P. C. (2009). New constructs, methods, and directions for computer-based assessment. In F. Scheuermann & J. Björnsson (Eds.), The transition to computer-based assessment-Lessons learned from large-scale surveys and implications for testing (pp. 151-156). Luxembourg City, Luxembourg: Office for Official Publications of the European Communities.
37. Lievens, F., & Chan, D. (2010). Practical intelligence, emotional intelligence, and social intelligence. In J. L. Farr & N. T. Tippins (Eds.), Handbook of employee selection (pp. 339-359). New York, NY: Routledge.
38. Little, T. D., Cunningham, W. A., Shahar, G., & Widaman, K. F. (2002). To parcel or not to parcel: Exploring the question, weighing the merits. Structural Equation Modeling, 9(2), 151-173. doi:10.1207/S15328007SEM0902_1
39. Lynch, M., & Macbeth, D. (1998). Demonstrating physics lessons. In J. G. Greeno & S. V. Goldman (Eds.), Thinking practices in mathematics and science learning (pp. 269-297). Mahwah, NJ: Erlbaum.
40. Mansfield, R. S., Busse, T. V., & Krepelka, E. J. (1978). The effectiveness of creativity training. Review of Educational Research, 48, 517-536.
41. Markman, A. B. (1999). Knowledge representation. Mahwah, NJ: Erlbaum. Mayer, R. E. (2003). Learning and instruction. Upper Saddle River, NJ: Prentice Hall.
42. Mayer, R. E. (2003). Learning and instruction. Upper Saddle River, NJ: Prentice Hall
43. Mayer, R. E., & Wittrock, M. C. (2006). Problem solving. In P. A. Alexander & P. H. Winne (Eds.), Handbook of educational psychology (pp. 287-303). Mahwah, NJ: Erlbaum.
44. McGrew, K. S. (2009). CHC theory and the Human Cognitive Abilities Project: Standing on the shoulders of the giants of psychometric intelligence research. Intelligence, 37, 1-10. doi:10.1016/j.intell.2008.08.004
45. Molnár, G., & Csapó, B. (2007, August 28-September 1). Constructing complex problem solving competency scales by IRT models using data of different age groups. Abstract submitted at the 12th Biennial EARLI Conference, Budapest, Hungary.
46. Muthén, L. K., & Muthén, B. O. (2010). Mplus user's guide (6th ed.). Los Angeles, CA: Author.
47. Myrberg, E., & Rosen, E. (2008). A path model with mediating factors of parents' education on students' reading achievement in seven countries. Educational Research and Evaluation, 14(6), 507-520. doi:10.1080/13803610802576742
48. Nagy, J. (2008). Renewing elementary education. In K. Fazekas, J. Köllö, & J. Varga (Eds.), Green book for renewal of public education in Hungary (pp. 61-80). Budapest, Hungary: Ecostat.
49. Neisser, U., Boodoo, G., Bouchard, T. J., Jr., Boykin, A. W., Brody, N., Ceci, S. J., Urbina, S. (1996). Intelligence: Knowns and unknowns. American Psychologist, 51, 77-101. doi:10.1037/0003-066X.51.2.77
50. Newell, A., & Simon, H. A. (1972). Human problem solving. Englewood Cliffs, NJ: Prentice Hall.
51. Novick, L. R., & Bassok, M. (2005). Problem solving. In K. J. Holyoak & R. G. Morrison (Eds.), The Cambridge handbook of thinking and reasoning (pp. 321-349). Cambridge, England: Cambridge University Press.
52. Novick, L. R., Hurley, S. M., & Francis, M. (1999). Evidence for abstract, schematic knowledge of three spatial diagram representations. Memory & Cognition, 27, 288-308. doi:10.3758/BF03211413
53. Organisation for Economic Co-operation and Development. (2004). Problem solving for tomorrow's world. First measures of cross-curricular competencies from PISA 2003. Paris, France: OECD.
54. Organisation for Economic Co-operation and Development. (2009). PISA 2009 results: What students know and can do: Student performance in reading, mathematics, and science (Vol. 1). Paris, France: OECD.
55. Organisation for Economic Co-operation and Development. (2010). PISA 2012 problem solving framework [draft for field trial]. Paris, France: OECD.
56. Putz-Osterloh, W. (1981). Ü ber die Beziehung zwischen Testintelligenz und Problemlöseerfolg [On the relation between test intelligence and problem solving success]. Zeitschrift für Psychologie, 189, 79-100.
57. Raven, J. C. (1962). Advanced progressive matrices, Set II. London, England: H. K. Lewis.
58. Raven, J. (2000). Psychometrics, cognitive ability, and occupational performance. Review of Psychology, 7, 51-74.
59. Ree, H. M., & Carretta, T. R. (2002). g2K. Human Performance, 15, 3-24.
60. Reeve, C. L., & Hakel, M. D. (2002). Asking the right questions about g. Human Performance, 15, 47-74. doi: 10.1080/08959285.2002.9668083
61. Rigas, G., Carling, E., & Brehmer, B. (2002). Reliability and validity of performance measures in microworlds. Intelligence, 30, 463-480. doi: 10.1016/S0160-2896(02)00121-6
62. Rindermann, H., Flores-Mendoza, C., & Mansur-Alves, M. (2010). Reciprocal effects between fluid and crystallized intelligence and their dependence on parents' socioeconomic status and education. Learning and Individual Differences, 20(5), 544-548. doi:10.1016/j.lindif.2010.07.002
63. Rutter, M., & Maughan, B. (2002). School effectiveness findings 1979-2002. Journal of School Psychology, 40(6), 451-475. doi:10.1016/S0022-4405(02)00124-3
64. Salthouse, T. A., & Davis, H. P. (2006). Organization of cognitive abilities and neuropsychological variables across the lifespan. Developmental Review, 26, 31-54. doi:10.1016/j.dr.2005.09.001
65. Smith, M. C., & Reio, T. G. (2006). Adult development, schooling, and the transition to work. In P. A. Alexander & P. H. Winne (Eds.), Handbook of educational psychology (pp. 115-138). Mahwah, NJ: Erlbaum.
66. Smith, S. S. (2006). Examining the long-term impact of achievement loss during the transition to high school. Journal of Secondary Gifted Education, 17(4), 211-221.
67. Sonnleitner, P., Brunner, M., Greiff, S., Funke, J., Keller, U., Martin, R., Latour, T. (2012). The Genetics Lab. Acceptance and psychometric characteristics of a computer-based microworld to assess complex problem solving. Psychological Test and Assessment Modeling, 54(1), 54-72.
68. Spearman, C. (1927). The abilities of man. Their nature and measurement. New York, NY: Macmillan.
69. Sternberg, R. J. (2000). The holey grail of general intelligence. Science, 289, 399-401. doi:10.1126/science.289.5478.399
70. Sternberg, R. J. (2009). Toward a triachic theory of human intelligence. In R. J. Sternberg, J. C. Kaufman, & E. L. Grigorenko (Eds.), The essential Sternberg: Essays on intelligence, psychology, and education (pp. 38-70). New York, NY: Springer. (Original work published 1984)
71. Sugrue, B. (1995). A theory-based framework for assessing domainspecific problem-solving ability. Educational Measurement: Issues and Practice, 14(3), 29-35. doi:10.1111/j.1745-3992.1995.tb00865.x
72. Taub, G. E., & McGrew, K. S. (2004). A confirmatory factor analysis of Cattell-Horn-Carroll theory and cross-age invariance of the Woodcock- Johnson Tests of Cognitive Abilities III. School Psychology Quarterly, 19(1), 72-87. doi:10.1521/scpq.19.1.72.29409
73. Ullman, J. B. (2007). Structural equation modeling. In B. G. Tabachnick & L. S. Fidell (Eds.), Using multivariate statistics (pp. 676-780). Boston, MA: Pearson.
74. Wechsler, D. (2008). Wechsler Adult Intelligence Scale 4th edition. San Antonio, TX: Pearson Asssessment.
75. Weiß, R. H. (2006). Grundintelligenztest Skala 2-Revision (CFT 20-R) [Culture Fair Intelligence Test 20-R-Scale 2]. Göttingen, Germany: Hogrefe.
76. Wigfield, A., Byrnes, J. P., & Eccles, J. S. (2006). Development during early and middle adolescence. In P. A. Alexander & P. H. Winne (Eds.), Handbook of educational psychology (pp. 87-113). Mahwah, NJ: Erlbaum.
77. Wüstenberg, S., Greiff, S., & Funke, J. (2012). Complex problem solving. More than reasoning? Intelligence, 40, 1-14. doi:10.1016/j.intell.2011.11.003