The assessment of 21st century skills in industrial and organizational psychology: Complex and collaborative problem solving

Industrial and Organizational Psychology

Volumn 8, Issue 2, 2015, Pages 238-268

  Neubert, Jonas C ^a,b   Mainert, Jakob ^a   Kretzschmar, André ^a   Greiff, Samuel ^a  

^a UNIVERSITY OF LUXEMBOURG   (Luxembourg)

^b BRANDENBURG UNIVERSITY OF TECHNOLOGY   (Germany)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84964461863     PISSN: 17549426     EISSN: 17549434     Source Type: Journal    
DOI: 10.1017/iop.2015.14     Document Type: Note

References (139)

1. Adkins, C. L., Russell, C. J., & Werbel, J. D. (1994). Judgments of fit in the selection process: The role of work value congruence. Personnel Psychology, 47, 605-623. doi:10.1111/j.1744-6570.1994.tb01740.x
2. Aguinis, H., & Pierce, C. A. (2008). Enhancing the relevance of organizational behavior by embracing performance management research. Journal of Organizational Behavior, 29, 139-145. doi:10.1002/job.493
3. Allred, B. B., Snow, C. C., & Miles, R. E. (1996). Characteristics of managerial careers in the 21st century. Academy of Management Executive, 10(4), 17-27. doi:10.5465/AME.1996.3145316
4. Anakwe, U. P., Hall, J. C., & Schor, S. M. (2000). Knowledge-related skills and effective career management. International Journal of Manpower, 21, 566-579. doi:10.1108/01437720010379024
5. Anderson, N. (2007). The practitioner-researcher divide revisited: Strategic-level bridges and the roles of IWO psychologists. Journal of Occupational and Organizational Psychology, 80, 175-183. doi:10.1348/096317907'187237
6. Anderson, N., Herriot, P., & Hodgkinson, G. P. (2001). The practitioner-researcher divide in industrial, work and organizational (IWO) psychology: Where are we now, and where do we go from here? Journal of Occupational and Organizational Psychology, 74, 391-411. doi:10.1348/096317901167451
7. Annett, J., & Duncan, K. (1967). Task analysis and training design. Occupational Psychology, 41, 211-221.
8. Armstrong, S. J., Cools, E., & Sadler-Smith, E. (2012). Role of cognitive styles in business and management: Reviewing 40 years of research: Role of cognitive styles in business and management. International Journal of Management Reviews, 14, 238-262. doi:10.1111/j.1468-2370.2011.00315.x
9. Arthur, M. B., & Rousseau, D. M. (1996). A career lexicon for the 21st century. Academy of Management Perspectives, 10(4), 28-39. doi:10.5465/AME.1996.3145317
10. Arun, N., Coyle, P. T., & Hauenstein, N. (2012). Learning agility: Still searching for clarity on a confounded construct. Industrial and Organizational Psychology: Perspectives on Science and Practice, 5, 290-293. doi:10.1111/j.1754-9434.2012.01447.x
11. Autor, D. H., Katz, L. F., & Kearney, M. S. (2006). The polarization of the U. S. labor market. American Economic Review, 96(2), 189-194. doi:10.1257/000282806777212620
12. Autor, D. H., Levy, F., & Murnane, R. J. (2003). The skill content of recent technological change:An empirical exploration. TheQuarterly Journal of Economics, 118, 1279-1333. doi:10.1162/003355303322552801
13. Barrick, M. R., & Mount, M. K. (1991). The Big Five personality dimensions and job performance: A meta-analysis. Personnel Psychology, 44, 1-26. doi:10.1111/j.1744-6570.1991.tb00688.x
14. Bass, B. M. (1974). The substance and the shadow. American Psychologist, 29, 870-886. doi:10.1037/h0037515
15. Bass, B. M., & Avolio, B. J. (1990). Transformational leadership development:Manual for the multifactor leadership questionnaire. Palo Alto, CA: Consulting Psychologists Press
16. Baumgarten, D., Geishecker, I., & Görg, H. (2010). Offshoring, tasks, and the skill-wage pattern. CEGE Discussion Paper, 98. Retrieved from http://ssrn.com/abstract=1574784. doi:10.2139/ssrn.1574784
17. Becker, S. O., Ekholm, K., & Muendler, M.-A. (2013). Offshoring and the onshore composition of tasks and skills. Journal of International Economics, 90, 91-106. doi:10.1016/j.jinteco.2012.10.005
18. Bird, A. (1994). Careers as repositories of knowledge: A new perspective on boundaryless careers. Journal of Organizational Behavior, 15, 325-344. doi:10.1002/job.4030150404
19. Brannick, M. T., Levine, E. L., & Morgeson, F. P. (2007). Job and work analysis: Methods, research, and applications for human resource management (2nd ed.). Los Angeles, CA: Sage.
20. Brannick, M. T., & Prince, C. (1997). An overview of team performance measurement. In M. T. Brannick, E. Salas, & C. Prince (Eds.), Team performance assessment and measurement: Theory, methods, and applications (pp. 3-16). Mahwah, NJ: Erlbaum.
21. Braune, R., & Foshay, W. R. (1983). Towards a practical model of cognitive/information processing task analysis and schema acquisition for complex problem-solving situations. Instructional Science, 12, 121-145.
22. Brehmer, B. (1992). Dynamic decision making: Human control of complex systems. Acta Psychologica, 81, 211-241.
23. Brouwers, S. A., & Van De Vijver, F. J. R. (2012). Intelligence 2.0 in I-O psychology: Revival or contextualization? Industrial and Organizational Psychology: Perspectives on Science and Practice, 5, 158-160. doi:10.1111/j.1754-9434.2012.01422.x
24. Bruch, H., Spychala, A., & Wiegel, J. L. (2013). Wege zur hochleistungsorganisation: Wie die Graubündner Kantonalbank energie mobilisiert und erhält [Pathways to a high performance organization: How the Graubündner Kantonalbankmobilizes andmaintains energy]. Personalführung, 2013(2), 20-27.
25. Buchanan, D. A., & Huczynski, A. (2010). Organizational behaviour (7th ed.). Harlow, United Kingdom: Pearson Education.
26. Buchner, A. (1995). Basic topics and approaches to the study of complex problem solving. In P. A. Frensch & J. Funke (Eds.), Complex problem solving: The European perspective (pp. 27-63). Hillsdale, NJ: Erlbaum.
27. Caplan, R. D. (1987). Person-environment fit theory and organizations: Commensurate dimensions, time perspectives, andmechanisms. Journal ofVocational Behavior, 31, 248-267. doi:10.1016/0001-8791(87)90042-X
28. Cappelli, P. (2008). Talent management for the twenty-first century. Harvard Business Review, 86(3), 74-81.
29. Cascio, W. F. (1995). Whither industrial and organizational psychology in a changing world of work? American Psychologist, 50, 928-939. doi:10.1037/0003-066X.50.11.928
30. Cascio, W. F. (1998). Managing human resources: Productivity, quality of work life, profits. Boston, MA: Irwin McGraw-Hill
31. Cascio, W. F., & Aguinis, H. (2008). Research in industrial and organizational psychology from 1963 to 2007: Changes, choices, and trends. Journal of Applied Psychology, 93, 1062-1081. doi:10.1037/0021-9010.93.5.1062
32. Chan, D. (1996). Cognitive misfit of problem-solving style at work: A facet of personorganization fit. Organizational Behavior and Human Decision Processes, 68, 194-207. doi:10.1006/obhd.1996.0099
33. Chatman, J. A. (1989). Improving interactional organizational research: A Model of person-organization fit. Academy of Management Review, 14, 333-349. doi:10.5465/AMR.1989.4279063
34. Clifford, J. P. (1994). Job analysis: Why do it, and how should it be done? Public Personnel Management, 23, 321-340. doi:10.1177/009102609402300211
35. Cohen, D. J. (2007). The very separate worlds of academic and practitioner publications in human resource management: Reasons for the divide and concrete solutions for bridging the gap. Academy of Management Journal, 50, 1013-1019. doi:10.5465/AMJ.2007.27151946
36. Cohen, M. D., March, J. G., & Olsen, J. P. (1972). "A garbage can model of organizational choice." Administrative Science Quarterly, 17, 1-25.
37. Collings, D. G., & Mellahi, K. (2009). Strategic talent management: A review and research agenda. Human Resource Management Review, 19, 304-313. doi:10.1016/j.hrmr.2009.04.001
38. Danner, D., Hagemann, D., Holt, D. V., Hager, M., Schankin, A., Wüstenberg, S., & Funke, J. (2011). Measuring performance in dynamic decision making. Journal of Individual Differences, 32, 225-233.
39. Day, D. V., & Lord, R. G. (1988). Executive leadership and organizational performance: Suggestions for a new theory and methodology. Journal of Management, 14, 453-464. doi:10.1177/014920638801400308
40. De Meuse, K. P., Dai, G., & Hallenbeck, G. S. (2010). Learning agility: A construct whose time has come. Consulting Psychology Journal: Practice and Research, 62, 119-130. doi:10.1037/a0019988
41. DeRue, D. S., Ashford, S. J., & Myers, C. G. (2012). Learning agility: In search of conceptual clarity and theoretical grounding. Industrial and Organizational Psychology: Perspectives on Science and Practice, 5, 258-279. doi:10.1111/j.1754-9434.2012.01444.x
42. Dierdorff, E. C., & Wilson, M. A. (2003). A meta-analysis of job analysis reliability. Journal of Applied Psychology, 88, 635-646. doi:10.1037/0021-9010.88.4.635
43. Dries, N., Vantilborgh, T., & Pepermans, R. (2012). The role of learning agility and career variety in the identification and development of high potential employees. Personnel Review, 41, 340-358. doi:10.1108/00483481211212977
44. Dunnette, M. D. (1990). Blending the science and practice of industrial and organizational psychology:Where are we and where are we going? In M. D. Dunnette & L. M. Hough (Eds.), Handbook of industrial and organizational psychology (2nd ed., Vol. 1, pp. 1-27). Palo Alto, CA: Consulting Psychologists Press.
45. Edwards, J. R. (1991). Person-job fit: A conceptual integration, literature review, and methodological critique. In C. L. Cooper& I. T. Robertson (Eds.), International Review of Industrial/Organizational Psychology (Vol. 9, pp. 383-357). London, United Kingdom: Wiley.
46. Eichinger, R. W., & Lombardo, M. M. (2004). Learning agility as a prime indicator of potential. Human Resource Planning, 27(4), 12-15.
47. Fischer, A., Greiff, S., & Funke, J. (2012). The process of solving complex problems. Journal of Problem Solving, 4(1), 19-42. doi:10.7771/1932-6246.1118
48. Fleishman, E. A. (1989).Manual for the Leadership Opinion Questionnaire (Rev. ed.). Rosemont, IL: National Computer Systems.
49. Fleishman, E. A., & Reilly, M. E. (1992). Handbook of human abilities: Definitions, measurements, and job task requirements. Palo Alto, CA: Consulting Psychologists Press
50. Funke, J. (2001). Dynamic systems as tools for analysing human judgement. Thinking & Reasoning, 7, 69-89. doi:10.1080/13546780042000046
51. Funke, J. (2010). Complex problem solving: A case for complex cognition? Cognitive Processing, 11, 133-142. doi:10.1007/s10339-009-0345-0
52. Gonzalez, C., Lerch, J. F., & Lebiere, C. (2003). Instance-based learning in dynamic decision making. Cognitive Science, 27, 591-635.
53. Gonzalez, C., Vanyukov, P., & Martin, M. K. (2005). The use of microworlds to study dynamic decision making. Computers in Human Behavior, 21, 273-286.
54. Goos, M., & Manning, A. (2007). Lousy and lovely jobs: The rising polarization of work in Britain. Review of Economics and Statistics, 89, 118-133. doi:10.1162/rest.89.1.118
55. Goos, M., Manning, A., & Salomons, A. (2009). Job polarization in Europe. The American Economic Review, 99(2), 58-63.
56. Gottfredson, L. S. (1997). Why g matters: The complexity of everyday life. Intelligence, 24, 79-132. doi:10.1016/S0160-2896(97)90014-3
57. Greenhaus, J. H., Callanan, G. A., & Kaplan, E. (1995). The role of goal setting in career management. International Journal of Career Management, 7(5), 3-12. doi:10.1108/09556219510093285
58. Greiff, S. (2012). From interactive to collaborative problem solving: Current issues in the Programme for International Student Assessment. Review of Psychology, 19, 111-121.
59. Greiff, S., Fischer, A., Wüstenberg, S., Sonnleitner, P., Brunner, M., & Martin, R. (2013). A multitrait-multimethod study of assessment instruments for complex problem solving. Intelligence, 41, 579-596. doi:10.1016/j.intell.2013.07.012
60. Greiff, S., Kretzschmar, A., Müller, J. C., Spinath, B., & Martin, R. (2014). Computer-based assessment of complex problem solving in educational contexts and how it is influenced by students' level of information and communication technology literacy. Journal of Educational Psychology, 106, 666-680. doi:10.1037/a0035426
61. Greiff, S., & Neubert, J. C. (2014). On the relation of complex problem solving, personality, fluid intelligence, and academic achievement. Learning and Individual Differences, 36, 37-48. doi:10.1016/j.lindif.2014.08.003
62. Greiff, S., & Wüstenberg, S. (2014). Assessment with microworlds using MicroDYN: Measurement invariance and latent mean comparisons: Psychometric properties across several student samples and blue-collar workers. European Journal of Psychological Assessment, 30, 304-314. doi:10.1027/1015-5759/a000194
63. Greiff, S., Wüstenberg, S., & Funke, J. (2012). Dynamic problem solving: A new assessment perspective. Applied Psychological Measurement, 36, 189-213. doi:10.1177/0146621612439620
64. Greiff, S., Wüstenberg, S., Molnár, G., Fischer, A., Funke, J., & Csapó, B. (2013). Complex problem solving in educational contexts-Something beyond g: Concept, assessment, measurement invariance, and construct validity. Journal of Educational Psychology, 105, 364-379. doi:10.1037/a0031856
65. Griffin, P., McGaw, B., & Care, E. (Eds.). (2012). Assessment and teaching of 21st century skills. New York, NY: Springer
66. Grossman, G. M., & Rossi-Hansberg, E. (2008). Trading tasks:Asimple theory of offshoring. American Economic Review, 98(5), 1978-1997. doi:10.1257/aer.98.5.1978
67. Hall, D. T., & Mirvis, P. H. (1996). The new protean career. In D. T. Hall (Ed.), The career is dead-Long live the career: A relational approach to careers (pp. 15-45). San Francisco, CA: Jossey-Bass.
68. Harvey, D. F., & Bowin, R. B. (1996). Human resource management: An experiential approach. Englewood Cliffs, NJ: Prentice-Hall
69. Hogan, J., & Holland, B. (2003). Using theory to evaluate personality and job-performance relations: A socioanalytic perspective. Journal of Applied Psychology, 88, 100-112. doi:10.1037/0021-9010.88.1.100
70. Holland, J. L. (1985). Making vocational choices: A theory of vocational personalities and work environments. Odessa, FL: Psychological Assessment Resources
71. Hunter, J. E., & Schmidt, F. L. (1996). Intelligence and job performance: Economic and social implications. Psychology, Public Policy, and Law, 2, 447-472. doi:10.1037/1076-8971.2.3-4.447
72. Jacobs, T. O., & Jaques, E. (1987). Leadership in complex systems. In J. Zeidner (Ed.), Human productivity enhancement (Vols. 1-2, pp. 7-65). New York, NY: Praeger.
73. Jones, G., & Schneider, W. J. (2006). Intelligence, human capital, and economic growth: A Bayesian averaging of classical estimates (BACE) approach. Journal of Economic Growth, 11, 71-93. doi:10.1007/s10887-006-7407-2
74. Keane, A. J., & Nair, P. B. (2001). Problem solving environments in aerospace design. Advances in Engineering Software, 32, 477-487. doi:10.1016/S0965-9978(00)00108-3
75. Kirton, M. (1976). Adaptors and innovators: A description and measure. Journal of Applied Psychology, 61, 622-629. doi:10.1037/0021-9010.61.5.622
76. Klein, C., DeRouin, R. E., & Salas, E. (2006). Uncovering workplace interpersonal skills: A review, framework, andresearchagenda. In G. P. Hodgkinson & J. K. Ford(Eds.), International review of industrial and organizational psychology (pp. 79-126). West Sussex, England:Wiley.
77. Klein, G. (2008). Naturalistic decision making. Human Factors: The Journal of the Human Factors and Ergonomics Society, 50, 456-460. doi:10.1518/001872008'288385
78. Klein, G., Orasanu, J., Calderwood, R., & Zsambok, C. E. (1993). Decision making in action: Models and methods. Norwood, NJ:Ablex
79. Kristof, A. L. (1996). Person-organization fit: An integrative review of its conceptualizations, measurement, and implications. Personnel Psychology, 49, 1-49. doi:10.1111/j.1744-6570.1996.tb01790.x
80. Levine, E. L., Ash, R. A., Hall, H., & Sistrunk, F. (1983). Evaluation of job analysis methods by experienced job analysts. The Academy of Management Journal, 26, 339-348. doi:10.2307/255981
81. Levine, E. L., Sistrunk, F., McNutt, K. J., & Gael, S. (1988). Exemplary job analysis systems in selected organizations: A description of process and outcomes. Journal of Business and Psychology, 3, 3-21. doi:10.1007/BF01016745
82. Lewis, R. E., & Heckman, R. J. (2006). Talent management: A critical review. Human Resource Management Review, 16, 139-154. doi:10.1016/j.hrmr.2006.03.001
83. Lombardo, M. M., & Eichinger, R. W. (2000). High potentials as high learners. Human Resource Management, 39, 321-329. doi:10.1002/1099-050X(200024)39:4<321::AIDHRM4>3.0. CO;2-1
84. Marshall-Mies, J. C., Fleishman, E. A., Martin, J. A., Zaccaro, S. J., Baughman, W. A., & McGee, M. L. (2000). Development and evaluation of cognitive and metacognitive measures for predicting leadership potential. The Leadership Quarterly, 11, 135-153. doi:10.1016/S1048-9843(99)00046-6
85. McCormick, E. J., Jeanneret, P. R., & Mecham, R. C. (1972). A study of job characteristics and job dimensions as based on the Position Analysis Questionnaire (PAQ). Journal of Applied Psychology, 56, 347-368. doi:10.1037/h0033099
86. McEntire, L. E., Dailey, L. R., Osburn, H. K., & Mumford, M. D. (2006). Innovations in job analysis: Development and application ofmetrics to analyze job data. Human Resource Management Review, 16, 310-323. doi:10.1016/j.hrmr.2006.05.004
87. Middleton, H. (2002). Complex problem solving in a workplace setting. International Journal of Educational Research, 37, 67-84. doi:10.1016/S0883-0355(02)00022-8
88. Mumford, M. D., Mobley, M. I., Reiter-Palmon, R., Uhlman, C. E., & Doares, L. M. (1991). Process analytic models of creative capacities. Creativity Research Journal, 4, 91-122. doi:10.1080/10400419109534380
89. Mumford, M. D., Zaccaro, S. J., Harding, F. D., Jacobs, T. O., & Fleishman, E. A. (2000). Leadership skills for a changing world: Solving complex social problems. The Leadership Quarterly, 11, 11-35. doi:10.1016/S1048-9843(99)00041-7
90. Murphy, K. R., & Saal, F. E. (1990). What should we expect from scientist-practitioners? In K. R. Murphy & F. E. Saal (Eds.), Psychology in organizations: Integrating science and practice (pp. 49-66). Hillsdale, NJ: Erlbaum.
91. Naikar, N., Moylan, A., & Pearce, B. (2006). Analysing activity in complex systems with cognitive work analysis: Concepts, guidelines and case study for control task analysis. Theoretical Issues in Ergonomics Science, 7, 371-394. doi:10.1080/14639220500098821
92. National Center for O*NET Development. (2009). New and emerging occupations of the 21st century: Updating the O*NET-SOC taxonomy [O*NET Reports and Documents]. Raleigh, NC: Author.
93. National Research Council. (2012). Education for life and work: Developing transferable knowledge and skills in the 21st century. Washington, DC: The National Academies Press
94. Novick, L. R., & Bassok, M. (2005). Problem solving. In K. J. Holyoak & R. G. Morrison (Eds.), The Cambridge handbook of thinking and reasoning (pp. 321-349). New York, NY: Cambridge University Press.
95. O'Neil, H. F., Chuang, S., & Chung, G. K. W. K. (2004). Issues in the computer-based assessment of collaborative problem solving (CSE Report No. 620). Los Angeles, CA: National Center for Research on Evaluation, Standards, and Student Testing.
96. O'Reilly, C. A., Chatman, J. A., & Caldwell, D. F. (1991). People and organizational culture: A profile comparison approach to assessing person-organization fit. Academy ofManagement Journal, 34, 487-516. doi:10.2307/256404
97. Organisation for Economic Co-operation and Development. (2013a). Employment by size class. In Entrepreneurship at a glance 2013. Paris, France: Author. Retrieved from http://www.keepeek.com/Digital-Asset-Management/oecd/industry-and-services/entrepreneurship-at-a-glance-2013/employment-by-size-class_entrepreneur_aag-2013-8-en#page1
98. Organisation for Economic Co-operation and Development. (2013b). OECD skills outlook 2013: First results from the survey of adult skills. Paris, France: Author. doi:10.1787/9789264204256-en
99. Organisation for Economic Co-operation andDevelopment. (2013c). PISA 2012 assessment and analytical framework. Paris, France:Author
100. Organisation for Economic Co-operation and Development. (2013d). PISA 2015 draft collaborative problem solving framework. Paris, France: Author. Retrieved from http://www.oecd.org/pisa/pisaproducts/Draft%20PISA%202015%20Collaborative%20Problem%20Solving%20Framework%20.pdf
101. Osman, M. (2010). Controlling uncertainty: A review of human behavior in complex dynamic environments. Psychological Bulletin, 136, 65-86. doi:10.1037/a0017815
102. Oswald, F. L., & Hough, L. (2012). I-O 2.0 From intelligence 1.5: Staying (just) behind the cutting edge of intelligence theories. Industrial and Organizational Psychology, 5, 172-175. doi:10.1111/j.1754-9434.2012.01425.x
103. Pearlman, K. (1980). Job families: A review and discussion of their implications for personnel selection. Psychological Bulletin, 87, 1-28. doi:10.1037/0033-2909.87.1.1
104. Putz-Osterloh, W. (1981). Über die beziehung zwischen Testintelligenz und problemlöseerfolg [The relation between test intelligence and problem solving success]. Zeitschrift Für Psychologie Mit Zeitschrift Für Angewandte Psychologie, 189(1), 79-100.
105. Rasmussen, J., Pejtersen, A. M., & Goodstein, L. P. (1994). Cognitive systems engineering. New York, NY:Wiley
106. Ready, D. A., & Conger, J. A. (2007). Make your company a talent factory. Harvard Business Review, 85(6), 68-77.
107. Reiter-Palmon, R., & Illies, J. J. (2004). Leadership and creativity: Understanding leadership from a creative problem-solving perspective. The Leadership Quarterly, 15, 55-77. doi:10.1016/j.leaqua.2003.12.005
108. Riding, R. J. (1997). On the nature of cognitive style. Educational Psychology, 17, 29-49. doi:10.1080/0144341970170102
109. Rupp, D. E., & Beal, D. (2007, July). Checking in with the scientist-practitionermodel:How are we doing? The Industrial-Organizational Psychologist, 45(1), 35-40.
110. Rynes, S. L., Colbert, A. E., & Brown, K. G. (2002). HR professionals' beliefs about effective human resource practices:Correspondence between research and practice. Human Resource Management, 41, 149-174. doi:10.1002/hrm.10029
111. Rynes, S. L., & Gerhart, B. (1990). Interviewer assessments of applicant "fit": An exploratory investigation. Personnel Psychology, 43, 13-35. doi:10.1111/j.1744-6570.1990.tb02004.x
112. Rynes, S. L., Giluk, T. L., & Brown, K. G. (2007). The very separate worlds of academic and practitioner periodicals in human resource management: Implications for evidence-based management. Academy of Management Journal, 50, 987-1008. doi:10.5465/AMJ.2007.27151939
113. Scherbaum, C. A., Goldstein, H. W., Yusko, K. P., Ryan, R., & Hanges, P. J. (2012). Intelligence 2.0: Reestablishing a research program on g in I-O psychology. Industrial and Organizational Psychology, 5, 128-148. doi:10.1111/j.1754-9434.2012.01419.x
114. Schippmann, J. S., Ash, R. A., Battista, M., Carr, L., Eyde, L. D., Hesketh, B., . . . Sanchez, J. I. (2000). The Practice of Competency Modeling. Personnel Psychology, 53, 703-740.
115. Schmid, U., Ragni, M., Gonzalez, C., & Funke, J. (2011). The challenge of complexity for cognitive systems. Cognitive Systems Research, 12, 211-218. doi:10.1016/j.cogsys.2010.12.007
116. Schmidt, F. L., & Hunter, J. E. (1998). The validity and utility of selectionmethods in personnel psychology: Practical and theoretical implications of 85 years of research findings. Psychological Bulletin, 124, 262-274. doi:10.1037/0033-2909.124.2.262
117. Schmidt, F. L., Hunter, J. E., Outerbridge, A. N., & Goff, S. (1988). Joint relation of experience and ability with job performance: Test of three hypotheses. Journal of Applied Psychology, 73, 46-57. doi:10.1037/0021-9010.73.1.46
118. Schneider, B., & Konz, A. M. (1989). Strategic job analysis. Human Resource Management, 28, 51-63. doi:10.1002/hrm.3930280104
119. Schooler, C., Mulatu, M. S., & Oates, G. (1999). The continuing effects of substantively complex work on the intellectual functioning of older workers. Psychology and Aging, 14, 483-506. doi:10.1037/0882-7974.14.3.483
120. Schweizer, F., Wüstenberg, S., & Greiff, S. (2013). Validity of theMicroDYNapproach:Complex problem solving predicts school grades beyond workingmemory capacity. Learning and Individual Differences, 24, 42-52. doi:10.1016/j.lindif.2012.12.011
121. Seibert, S. E., & Kraimer, M. L. (2001). The five-factor model of personality and career success. Journal of Vocational Behavior, 58, 1-21. doi:10.1006/jvbe.2000.1757
122. Shepherd, A. (2001). Hierarchical task analysis. NewYork, NY: Taylor & Francis
123. Siddique, C. M. (2004). Job analysis: A strategic human resource management practice. International Journal of Human Resource Management, 15, 219-244. doi:10.1080/0958519032000157438
124. Smith, G. F. (1997). Managerial problem solving: A problem-centered approach. In C. E. Zsambok& G. Klein (Eds.), Naturalistic decision making (pp. 371-380). Hillsdale, NJ: Erlbaum.
125. Sonnleitner, P., Brunner, M., Greiff, S., Funke, J., Keller, U., Martin, R., . . . Latour, T. (2012). The genetics lab: Acceptance and psychometric characteristics of a computerbased microworld assessing complex problem solving. Psychological Test and Assessment Modeling, 54, 54-72.
126. Sonnleitner, P., Keller, U., Martin, R., & Brunner, M. (2013). Students' complex problemsolving abilities: Their structure and relations to reasoning ability and educational success. Intelligence, 41, 289-305. doi:10.1016/j.intell.2013.05.002
127. Spitz-Oener, A. (2006). Technical change, job tasks, and rising educational demands: Looking outside the wage structure. Journal of Labor Economics, 24, 235-270.
128. Taylor, F. W. (1911). The principles of scientific management. NewYork, NY:Harper
129. Taylor, W. G. K. (1989). The Kirton Adaption-Innovation Inventory: A re-examination of the factor structure. Journal of Organizational Behavior, 10, 297-307. doi:10.1002/job.4030100402
130. Tushman, M., & O'Reilly, C. (2007). Research and relevance: Implications of Pasteur's quadrant for doctoral programs and faculty development. Academy ofManagement Journal, 50, 769-774. doi:10.5465/AMJ.2007.26279169
131. Vancouver, J. B., & Schmitt, N. W. (2006). An exploratory examination of personorganization fit: Organizational goal congruence. Personnel Psychology, 44, 333-352. doi:10.1111/j.1744-6570.1991.tb00962.x
132. Vargas Cortes, M. G., & Beruvides, M. G. (1996). An analysis of middle management work in non-steady conditions. Computers & Industrial Engineering, 31, 53-57. doi:10.1016/0360-8352(96)00077-0
133. Vicente, K. J. (1999). Cognitive work analysis: Toward safe, productive, and healthy computerbased work. Hoboken, NJ: Taylor & Francis
134. Warr, P., & Bunce, D. (1995). Trainee characteristics and the outcomes of open learning. Personnel Psychology, 48, 347-375. doi:10.1111/j.1744-6570.1995.tb01761.x
135. Weede, E., & Kämpf, S. (2002). The impact of intelligence and institutional improvements on economic growth. Kyklos, 55, 361-380. doi:10.1111/1467-6435.00191
136. Wüstenberg, S., Greiff, S., & Funke, J. (2012). Complex problem solving-More than reasoning? Intelligence, 40, 1-14.
137. Zaccaro, S. J., Mumford, M. D., Connelly, M. S., Marks, M. A., & Gilbert, J. A. (2000). Assessment of leader problem-solving capabilities. The Leadership Quarterly, 11, 37-64. doi:10.1016/S1048-9843(99)00042-9
138. Zaccaro, S. J., Mumford, M. D., Marks, M. A., Connelly, M. S., Threlfall, K. V., Gilbert, J. A., & Fleishman, E. A. (1997). Cognitive and temperament determinants of Army leadership (U. S. Army Research Institute for the Behavioral and Social Sciences Tech. Rep. No. MRI 95-4). Bethesda, MD: Management Research Institute.
139. Zsambok, C. E., & Klein, G. (Eds.). (1997). Naturalistic decision making. Hillsdale, NJ: Erlbaum