An updated review of the multidimensionality of training outcomes: New directions for training evaluation research

Learning, Training, and Development in Organizations

Volumn , Issue , 2009, Pages 135-165

  Ford, J Kevin ^a   Kraiger, Kurt ^b   Merritt, Stephanie M ^c  

^a MICHIGAN STATE UNIVERSITY   (United States)

^b Department of Environmental and Radiological Health Sciences   (United States)

^c UNIVERSITY OF MISSOURI ST LOUIS   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84917263802     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.4324/9780203878385     Document Type: Chapter

References (112)

1. Airasian, P. W., & Miranda, H. (2002). The role of assessment in the revised taxonomy. Theory Into Practice, 41, 249-254.
2. Ajzen, I. (2001). Nature and operation of attitudes. Annual Review of Psychology, 52, 27-58.
3. Amodio, D. M., Harmon-Jones, E., & Devine, P. G. (2003). Individual differences in the activation and control of affective race bias as assessed by startle eye-blink response and self-report. Journal of Personality and Social Psychology, 84(4), 738-753.
4. Anderson, J. R. (1982). Acquisition of a cognitive skill. Psychological Review, 89, 369-406.
5. Anderson, L. W., Krathwohl, D. R., Airasian, P. W., Cruikshank, D. A., Mayer, R. E., Pintrich, P. R., et al. (2001). A taxonomy for learning and assessing: A revision of Bloom's taxonomy of educational objectives. New York: Longman.
6. Baker, L., & Cerro, L. (2000). Assessing metacognition in children and adults. In G. Schraw & J. Impara (Eds.), Issues in the measurement of metacognition (pp. 99-145), Lincoln, NE: Buros Institute of Mental Measurements, University of Nebraska Press.
7. Bargh, J. A., Chaiken, S., Govender, R., & Pratto, F. (1992). The generality of the automatic activation effect. Journal of Personality and Social Psychology, 62, 893-912.
8. Bargh, J. A., & Chartrand, T. L. (2000). The mind in the middle: A practical guide to priming and automaticity research. In H. T. Reiss & C. M. Judd (Eds.), Handbook of research methods in social and personality psychology (pp. 253-285). New York: Cambridge University Press.
9. Bell, B. S., & Kozlowski, S. W. J. (2002a). Goal orientation and ability: Interactive effects on self-efficacy, performance, and knowledge. Journal of Applied Psychology, 87, 497-505.
10. Bell, B. S., & Kozlowski, S. W. J. (2002b). Adaptive guidance: Enhancing self-regulation, knowledge and performance in technology-based training. Personnel Psychology, 55, 267-306.
11. Bizer, G. Y., & Krosnick, J. A. (2001). Exploring the structure of strength-related features: The relation between attitude importance and attitude accessibility. Journal of Personality and Social Psychology, 81, 566-586.
12. Bloom, B. (1956). Taxonomy of learning objectives: The cognitive domain. New York: Donald McKay.
13. Bransford, J., Brown, A., & Cocking, R. (1999). How people learn: Brain, mind, experience, and school. Washington, DC: National Academy Press.
14. Breland, B. T., & Donovan, J. J. (2005). The role of state goal orientation in the goal establishment process. Human Performance, 18, 23-53.
15. Brendl, C. M., Markman, A. B., & Messner, C. (2001). How do indirect measures of evaluation work? Evaluating the inference of prejudice in the Implicit Association Test. Journal of Personality and Social Psychology, 81, 760-773.
16. Brophy, J. (2005). Goal theorists should move on from performance goals. Educational Psychologist, 40(3), 167-176.
17. Button, S. B., Mathieu, J. E., & Zajac, D. M. (1996). Goal orientation in organizational research: A conceptual and empirical foundation. Organizational Behavior and Human Decision Processes, 67, 26-48.
18. Cannon-Bowers, J. A., & Salas, E. (2001). Reflections on shared cognition. Journal of Organizational Behavior, 195-202.
19. Chaiken, S., & Stangor, C. (1987). Attitudes and attitude change. Annual Review of Psychology, 38, 575-630.
20. Champagne, A. B., Klopfer, L. E., Desena, A. T., & Squires, D. A. (1981). Structural representations of students' knowledge before and after science instruction. Journal of Research in Science Technology, 18, 97-111.
21. Chen, G., Gully, S. M., Whiteman, J., & Kilcullen, R. N. (2000). Examination of relationships among trait-like individual differences, state-like individual differences, and learning performance. Journal of Applied Psychology, 85, 835-847.
22. Colquitt, J. A., LePine, J. A., & Noe, R. A. (2000). Trainee attributes and attitudes revisited: A meta-analytic structural equation modeling analysis of research on training motivation. Journal of Applied Psychology, 85, 678-706.
23. Colquitt, J. A., & Simmering, M. J. (1998). Conscientiousness, goal orientation, and motivation to learn during the learning process: A longitudinal study. Journal of Applied Psychology, 83, 654-665.
24. Crano, W. D., & Prislin, R. (2006). Attitudes and persuasion. Annual Review of Psychology, 57, 345-374.
25. Curtis, M. B., & Davis, M. A. (2003). Assessing knowledge structure in accounting education: An application of Pathfinder associative networks. Journal of Accounting Education, 21, 185-195.
26. Daley, B. J., Shaw, C. R., Balistrieri, T., Glasenapp, K., & Piacentine, L. (1999). Concept maps: A strategy to teach and evaluate critical thinking. Journal of Nursing Education, 38, 42-47.
27. Davis, M. A., Curtis, M. B., & Tschetter, J. D. (2003). Evaluating cognitive training outcomes: Validity and utility of structural knowledge assessment. Journal of Business and Psychology, 18, 191-206.
28. Day, E. A., Arthur, W., & Gettman, D. (2001). Knowledge structures and the acquisition of a complex skill. Journal of Applied Psychology, 86, 1022-1033.
29. Devine, P. (2001). Implicit prejudice and stereotyping: How automatic are they? Introduction to the special section. Journal of Personality and Social Psychology, 81, 757-759.
30. Devine, P. G., Plant, E. A., Amodio, D. M., Harmon-Jones, E., & Vance, S. L. (2002). The regulation of explicit and implicit race bias: The role of motivations to respond without prejudice. Journal of Personality and Social Psychology, 82, 835-848.
31. DeShon, R. P., & Gillespie, J. Z. (2005). A motivated action theory account of goal orientation. Journal of Applied Psychology, 90, 1096-1127.
32. Dovidio, J. F., Kawakami, K., Johnson, C., Johnson, B., & Howard, A. (1997). On the nature of prejudice: Automatic and controlled processes. Journal of Experimental Social Psychology, 33, 510-540.
33. Dowson, M., & McInerney, D. M. (2004). The development and validation of the Goal Orientation and Learning Strategies Survey (GOALS-S). Educational and Psychological Measurement, 64, 290-310.
34. Dweck, C. S. (1986). Mental processes affecting learning. American Psychologist, 41, 1040-1048.
35. Dweck, C. S., & Elliott, E. S. (1983). Achievement motivation. In E. M. Hetherington, (Ed.), Handbook of child psychology (Vol. 4, pp. 643-691). New York: Wiley.
36. Eccles, D. W., & Tenenbaum, G. (2004). Why an expert team is more than a team of experts: A social-cognitive conceptualization of team coordination and communication in sport. Journal of Sport and Exercise Psychology, 26, 542-560.
37. Elliot, A. J., & Church, M. A. (1997). A hierarchical model of approach and avoidance achievement motivation. Journal of Personality, 71, 369-396.
38. Elliot, A. J., & Harackiewicz, J. M. (1996). Approach and avoidance goals and intrinsic motivation: A mediational analysis. Journal of Personality and Social Psychology, 70, 461-475.
39. Ellis, A. (2006). System breakdown: The role of mental models and transactive memory in the relationship between acute stress and team performance. Academy of Management Journal, 49, 576-589.
40. Fazio, R. H. (1993). Variability in the likelihood of automatic attitude activation: Data reanalysis and commentary on Bargh, Chaiken, Govender, and Pratto (1992). Journal of Personality and Social Psychology, 64, 753-758.
41. Fazio, R. H., Jackson, J. R., Dunton, B. C., & Williams, C. J. (1995). Variability in automatic activation as an unobtrusive measure of racial attitudes: A bona fide pipeline? Journal of Personality and Social Psychology, 69, 1013-1027.
42. Fazio, R. H., Sanbonmatsu, D. M., Powell, M. C., & Kardes, F. R. (1986). On the automatic activation of attitudes. Journal of Personality and Social Psychology, 50, 229-238.
43. Ferrer-Caja, E., & Weiss, M. R. (2002). Cross-validation of a model of intrinsic motivation with students enrolled in high school elective courses. Journal of Experimental Education, 71, 41-65.
44. Fisher, S. L., & Ford, J. K. (1998). Differential effects of learner effort and goal orientation on two learning outcomes. Personnel Psychology, 51, 397-420.
45. Ford, J. K., Smith, E. M., Weissbein, D. A., Gully, S. M., & Salas, E. (1998). Relationships of goal orientation, metacognitive activity, and practice strategies with learning outcomes and transfer. Journal of Applied Psychology, 83, 218-233.
46. Gagne, R. M. (1984). Learning outcomes and their effects: Useful categories of human performance. American Psychologist, 39, 377-385.
47. Gawronski, B., LeBel, E. P., & Peters, K. R. (2007). What do implicit measures tell us? Perspectives on Psychological Science, 2, 181-193.
48. Glaser, R., Lesgold, A., & Gott, S. (1986, July). Implications of cognitive psychology for measuring job performance. Paper presented for the Committee on the Performance of Military Personnel, National Academy of Sciences, Washington, DC.
49. Goldstein, I. L., & Ford, J. K. (2002). Training in organizations (4th ed.). Belmont, CA: Wadsworth/Thompson.
50. Greenwald, A. G., & Banaji, M. R. (1995). Implicit social cognition: Attitudes, self-esteem, and stereotypes. Psychological Review, 102, 4-27.
51. Greenwald, A. G., McGhee, D. E., & Schwartz, J. L. K. (1998). Measuring individual differences in implicit cognition: The implicit association test. Journal of Personality and Social Psychology, 74, 1464-1480.
52. Harackiewicz, J. M., Barron, K. E., Tauer, J. M., & Elliot, A. J. (2002). Predicting success in college: A longitudinal study of achievement goals and ability measures as predictors of interest and performance from freshman year through graduation. Journal of Educational Psychology, 94, 562-575.
53. Holbrook, A. L., Berent, M. K., Krosnick, J. A., Visser, P. S., & Boninger, D. S. (2005). Attitude importance and the accumulation of attitude-relevant knowledge in memory. Journal of Personality and Social Psychology, 88, 749-769.
54. Holladay, C. L., & Quiñones, M. A. (2003). Practice variability and transfer of training: The role of self-efficacy generality. Journal of Applied Psychology, 88, 1094-1103.
55. Holland, R. W., & van Knippenberg, A. (2003). From repetition to conviction: Attitude accessibility as a determinant of attitude certainty. Journal of Experimental Social Psychology, 39, 594-601.
56. Hsieh, A. T., & Chao, H. Y. (2004). A reassessment of the relationship between job specialization, job rotation and job burnout: An example of Taiwan's hightechnology industry. International Journal of Human Resource Management, 15, 1108-1123.
57. Ivancevich, J. M., & Gilbert, J. A. (2000). Diversity management: Time for a new approach. Public Personnel Management, 29, 75-92.
58. Johnson-Laird, P. (1983). Mental models. Cambridge, MA: Harvard University Press.
59. Jonassen, D. H. (1995). Operationalizing mental models: Strategies for assessing mental models to support meaningful learning and design-supportive learning environments. CSCL Proceedings, 182-186.
60. Kaplan, D. E., & Black, J. B. (2003). Mental models and computer based scientific inquiry learning: Effects of mechanistic cues on adolescent representation and reasoning about causal systems. Journal of Science Education & Technology, 12, 483-493.
61. Karpinski, A., & Hilton, J. L. (2001). Attitudes and the Implicit Association Test. Journal of Personality and Social Psychology, 81, 774-788.
62. Kawakami, K., Dion, K. L., & Dovidio, J. F. (1998). Racial prejudice and stereotype activation. Personality and Social Psychology Bulletin, 24, 407-416.
63. Klein, H. J., & Lee, S. (2006). The effects of personality on learning: The mediated role of goal setting. Human Performance, 19, 43-66.
64. Kozlowski, S. W. J., & Bell, B. S. (2006). Disentangling achievement orientation and goal setting: Effects on self-regulatory processes. Journal of Applied Psychology, 91, 900-916.
65. Kozlowski, S. W. J., DeShon, R. P., Schmidt, A. M., & Chambers, B. A. (2002). Effects of feedback and goal orientation on individual and team regulation, learning, and performance. In S. W. J. Kozlowski (Chair), Advances in training effectiveness: Traits, states, learning processes, and outcomes. Symposium presented at the 17th Annual Conference of the Society for Industrial and Organizational Psychology, Toronto, Ontario, Canada.
66. Kozlowski, S. W. J., Gully, S. M., Brown, K. G., Salas, E., Smith, E. M., & Nason, E. R. (2001). Effects of training goals and goal orientation traits on multidimensional training outcomes and performance adaptability. Organizational Behavior and Human Decision Processes, 85, 1-31.
67. Kraiger, K. (2003). Perspectives on training. In W. C. Borman, D. R. Ilgen, & R. J. Klimoski (Eds.), Comprehensive handbook of psychology: Volume 12. Industrial and organizational psychology (pp. 171-192). Hoboken, NJ: John Wiley.
68. Kraiger, K., & Ford, J. K. (2007). The history of training in industrial/organizational psychology. In L. Koppes (Ed.), The science and practice of industrial and organizational psychology: Historical aspects from the first 100 years. Mahwah, NJ: Lawrence Erlbaum Associates.
69. Kraiger, K., Ford, J. K., & Salas, E. D. (1993). Application of cognitive, skill-based, and affective theories of learning outcomes to new methods of training evaluation. Journal of Applied Psychology, 78, 311-328.
70. Kraiger, K., Salas, E., & Cannon-Bowers, J. (1995). Measuring knowledge organization as a method for assessing learning during training. Human Factors, 37, 804-816.
71. Krathwohl, D. R. (2002). A revision of Bloom's taxonomy: An overview. Theory Into Practice, 41, 212-218.
72. Krathwohl, D. R., Bloom, B. S., & Masia, B. B. (1964). Taxonomy of educational objectives: The classification of educational goals. White Plains, NY: Longman.
73. Krosnick, J. A., Boninger, D. S., Chuang, Y. C., Berent, M. K., & Carnot, C. G. (1993). Attitude strength: One construct or many related constructs? Journal of Personality and Social Psychology, 65, 1132-1151.
74. Krosnick, J. A., & Petty, R. E. (1995). Attitude strength: An overview. In R. E. Petty & J. A. Krosnick (Eds.), Attitude strength: Antecedents and consequences (pp. 1-24). Hillsdale, NJ: Lawrence Erlbaum Associates.
75. Kuhn, D. (2000). Metacognitive development. Current Directions in Cognitive Science, 9, 178-181.
76. Marks, M. A., Sabella, M. J., Burke, C., & Zaccaro, S. J. (2002). The impact of cross training on team effectiveness. Journal of Applied Psychology, 87, 3-13.
77. Marks, M. A., Zaccaro, S. J., & Mathieu, J. E. (2000). Performance implications for leader briefings and team-interaction training for team adaptation to novel environments. Journal of Applied Psychology, 85, 971-986.
78. Mathieu, J. E., Heffner, T. S., Goodwin, G. F., Salas, E., & Cannon-Bowers, J. A. (2000). The influence of shared mental models on team process and performance. Journal of Applied Psychology, 85, 273-283.
79. McConahay, J. B. (1986). Modern racism, ambivalence, and the Modern Racism Scale. In J. F. Dovidio & S. L. Gaertner (Eds.), Prejudice, discrimination, and racism (pp. 91-125). San Diego, CA: Academic Press.
80. McConnell, A. R., & Liebold, J. M. (2001). Relations between the Implicit Association Test, explicit racial attitudes, and discriminatory behavior. Journal of Experimental Social Psychology, 37, 435-442.
81. Means, B., & Gott, S. P. (1988). Cognitive task analysis as a basis for tutor development: Articulating abstract knowledge representations. In J. Psotka, L. D. Massey, & S. A. Mutter (Eds.), Intelligent tutoring systems: Lessons learned (pp. 35-57). Hillsdale, NJ: Lawrence Erlbaum Associates.
82. Midgley, C., Kaplan, A., Middleton, M., Maehr, M. L., Urdan, T., Anderman, L. H., et al. (1998). The development and validation of scales assessing students' achievement goal orientations. Contemporary Educational Psychology, 23, 113-131.
83. Mohammed, S., & Dumville, B. C. (2001). Team mental models in a team knowledge framework: Expanding theory and measurement across disciplinary boundaries. Journal of Organizational Behavior, 22, 89-106.
84. Olson, M. A., & Fazio, R. H. (2004). Reducing the influence of extrapersonal associations on the Implicit Association Test: Personalizing the IAT. Journal of Personality and Social Psychology, 86, 653-667.
85. Pfau, M., Compton, J., Parker, K. A., Wittenberg, E. M., An, C., Ferguson, M., et al. (2004). The traditional explanation for resistance versus attitude accessibility: Do they trigger distinct or overlapping processes of resistance? Human Communication Research, 30, 329-360.
86. Phillips, J. M., & Gully, S. M. (1997). Role of goal orientation, ability, need for achievement, and locus of control in the self-efficacy and goal-setting process. Journal of Applied Psychology, 82, 792-802.
87. Pintrich, P. (2002). The role of metacognitive knowledge in learning, teaching and assessing. Theory Into Practice, 41, 119-225.
88. Pintrich, P. R., Wolters, C., & Baxter, G. (2000). Assessing metacognition and self-regulated learning. In G. Schraw & J. Impara (Eds.), Issues in the measurement of metacognition (pp. 43-97). Lincoln, NE: Buros Institute of Mental Measurements.
89. Radosevich, D. J., Vaidyanathan, V. T., Yeo, S., & Radosevich, D. M. (2004). Relating goal orientation to self-regulatory processes: A longitudinal field test. Contemporary Educational Psychology, 29, 207-229.
90. Rouse, W. B., & Morris, N. M. (1986). On looking into the black box: Prospects and limits in the search for mental models. Psychological Bulletin, 100, 349-363.
91. Rudman, L. A., Ashmore, R. D., & Gray, M. L. (2001). "Unlearning" automatic biases: The malleability of implicit prejudice and stereotypes. Journal of Personality and Social Psychology, 81, 856-868.
92. Sandberg, J., Christoph, N., & Emans, B. (2001). Tutor training: A systematic investigation of tutor requirements and an evaluation of a training program. British Journal of Education Technology, 32, 69-90.
93. Schmidt, A., & Ford, K. (2003). Learning within a learner control training environment: The interactive effects of goal orientation and metacognitive instruction on learning outcomes. Personnel Psychology, 56, 405-429.
94. Schmidt, R. A., & Bjork, R. A. (1992). New conceptualizations of practice: Common principles in three paradigms suggest new concepts for training. Psychological Science, 3, 207-217.
95. Schvaneveldt, R. W., Durso, F. T., & Dearholt, D. W. (1985). Pathfinder: Scaling with network structures. Las Cruces: New Mexico State University, Computing Research Laboratory.
96. Shavelson, R. J., Lang, H., & Lewin, B. (1993). Indirect approaches to knowledge representation of high school science: On concept maps as potential authentic assessments in science. Los Angeles: National Center for Research on Evaluation, Standards, and Student Testing, U.S. Department of Education.
97. Smith, E. R., & Zarate, M. A. (1992). Exemplar-based model of social judgment. Psychological Review, 99, 3-21.
98. Smith-Jentsch, K., Mathieu, J., & Kraiger, K. (2005). Investigating linear and interactive effects of shared mental models on safety and efficiency in a field setting. Journal of Applied Psychology, 90, 523-535.
99. Standage, M., Duda, J., & Ntoumanis, N. (2003). Predicting motivational regulations in physical education: The interplay between dispositional goal orientations, motivational climate and perceived competence. Journal of Sport Sciences, 21, 631-647.
100. Stevens, C. K., & Gist, M. E. (1997). Effects of self-efficacy and goal-orientation training on negotiation skill maintenance: What are the mechanisms? Personnel Psychology, 50, 955-978.
101. Stout, R. J., Kraiger, K., & Salas, E. (1997). The role of trainee knowledge structures in aviation team environments. International Journal of Aviation Psychology, 7, 235-250.
102. Tan, J. A., & Hall, R. J. (2005). The effects of social desirability bias on applied measures of goal orientation. Personality & Individual Differences, 38, 1891-1902.
103. VandeWalle, D. (1997). Development and validation of a work domain goal orientation instrument. Educational and Psychological Measurement, 57, 995-1015.
104. VandeWalle, D., Cron, W. L., & Slocum, J. W, Jr. (2001). The role of goal orientation following performance feedback. Journal of Applied Psychology, 86, 629-640.
105. Wallhead, T. L., & Ntoumanis, N. (2004). Effects of a sport education intervention on students' motivational responses in physical education. Journal of Teaching in Physical Education, 23, 4-18.
106. Wilson, T. D., Lindsey, S., & Schooler, T. Y. (2000). A model of dual attitudes. Psychological Review, 107, 101-126.
107. Wittenbrink, B., Judd, C. M., & Park, B. (1997). Evidence for racial prejudice at the implicit level and its relationship with questionnaire measures. Journal of Personality and Social Psychology, 72, 262-274.
108. Yeo, G. B., & Neal, A. (2004). A multilevel analysis of effort, practice, and performance: Effects of ability, conscientiousness, and goal orientation. Journal of Applied Psychology, 89, 231-247.
109. Yi, M. Y., & Davis, F. D. (2001). Improving computer training effectiveness for decision technologies: Behavior modeling and retention enhancement. Decision Sciences, 32, 521-545.
110. Yin, Y., Vanides, J., Ruiz-Primo, M., Ayala, C., & Shavelson, R. (2005). Comparison of two concept-mapping techniques: Implications for scoring, interpretation, and use. Journal of Research in Science Teaching, 42, 166-184.
111. Zweig, D., & Webster, J. (2004a). What are we measuring? An examination of the relationships between the big-five personality traits, goal orientation, and performance intentions. Personality and Individual Differences, 36, 1693-1708.
112. Zweig, D., & Webster, J. (2004b). Validation of a multidimensional measure of goal orientation. Canadian Journal of Behavioral Science, 36, 232-243.