Are we doing the right thing?: Food for thought on training evaluation and its context

Personnel Review

Volumn 39, Issue 2, 2010, Pages 162-177

  Giangreco, Antonio ^a   Carugati, Andrea ^a   Sebastiano, Antonio ^b  

^a IESEG SCHOOL OF MANAGEMENT   (France)

^b CREMS Centre for Research on Health Economics Social and Health Care Management Carlo Cattaneo University LIUC   (Italy)

Author keywords

Human resource management; Organizational behaviour; Training evaluation

Indexed keywords

EID: 75649091506     PISSN: 00483486     EISSN: None     Source Type: Journal    
DOI: 10.1108/00483481011017390     Document Type: Article

References (81)

1. Albernathy, D.J. (1999), “Thinking outside the evaluation box”, Training & Development, Vol. 53 No. 2, pp. 18-24.
2. Alliger, G.M. and Janak, E.A. (1989), “Kirkpatrick's levels of training criteria: 30 years later”, Personnel Psychology, Vol. 42, pp. 331-342.
3. Alliger, G.M., Tannenbaum, S.I., Bennett, W. Jr, Traver, H. and Shotland, A. (1997), “A meta-analysis of the relations among training criteria”, Personnel Psychology, Vol. 50, pp. 341-358.
4. Alvesson, M. (2000), “Social identity and the problem of loyalty in knowledge-intensive companies”, Journal of Management Studies, Vol. 37 No. 8, pp. 1101-23.
5. Andrews, A.B. (2004), “Start at the end: empowerment evaluation product planning”, Evaluation and Program Planning, Vol. 27 No. 3, pp. 275-285.
6. Arthur, W. Jr, Bennet, W. Jr, Edens, P.A. and Bell, S.T. (2003), “Effectiveness of training in organizations: a meta-analysis of design and evaluation features”, Journal of Applied Psychology, Vol. 88 No. 2, pp. 234-245.
7. Bates, R. (2004), “A critical analysis of evaluation practice: the kirkpatrick model and the principle of beneficence”, Evaluation and Program Planning, Vol. 27 No. 3, pp. 341-8.
8. Blanchard, N.P. and Thacker, J.W. (2007), Effective Training – Systems, Strategies, and Practices, 3rd ed., Pearson Prentice-Hall, Upper Saddle River, NJ.
9. Blanchard, N.P., Thacker, J.W. and Way, S.A. (2000), “Training evaluation: perspectives and evidence”, International Journal of Training and Development, Vol. 4 No. 4, pp. 295-304.
10. Brinkerhoff, R.O. (2003), The Success Case Method, Berrett-Koehler, San Francisco, CA.
11. Brown, K. and Gerhardt, M.W. (2002), “Formative evaluation: an integrative practice model and case study”, Personnel Psychology, Vol. 55, pp. 951-983.
12. Brown, K.G. (2005), “An examination of the structure and nomological network of trainee reactions: a closer look at ‘smile sheets’”, Journal of Applied Psychology, Vol. 90 No. 5, pp. 991-1001.
13. Brown, R.E. and Reed, C.S. (2002), “An integral approach to evaluating outcome evaluation training”, American Journal of Evaluation, Vol. 23 No. 1, pp. 1-17.
14. Bushnell, D.S. (1990), “Input, process, output: a model for evaluating training”, Training and Development Journal, Vol. 44 No. 3, pp. 41-43.
15. Cannon-Bowers, J.A., Salas, E., Tannenbaum, S.I. and Mathieu, J.E. (1995), “Toward theoretically based principles of training effectiveness: a model and initial empirical investigation”, Military Psychology, Vol. 7, pp. 141-164.
16. Cascio, W.F. (1999), Costing Human Resources: the Financial Impact of Behavior in Organizations, 4th ed., Kent Publishing, Boston, MA.
17. Clement, R.W. (1982), “Testing the hierarchy theory of training evaluation: an expanded role for trainee reactions”, Public Personnel Management Journal, pp. 176-184.
18. Costa, G. and Giannecchini, M. (2005), Risorse Umane, Mc Graw-Hill, Milan.
19. Cullen, J.C., Sawzin, S.A., Sisson, G.R. and Swanson, R.A. (1978), “Cost effectiveness: a model for assessing the training investment”, Training and Development Journal, Vol. 1, pp. 24-29.
20. Dixon, N.M. (1990), “The relationship between trainee responses on participant reaction forms and posttest scores”, Human Resources Development Quarterly, Vol. 1 No. 2, pp. 129-137.
21. Eseryel, D. (2002), “Approaches to evaluation of training: theory and practice”, Educational Technology & Society, Vol. 5 No. 2, pp. 93-98.
22. Fitz-Enz, J. (1994), “Yes … you can weigh training's value”, Training, Vol. 31 No. 7, pp. 54-58.
23. Friedman, T.L. (2005), The World is Flat: A Brief History of the 21st Century, Farrar, Straus and Giroux, New York, NY.
24. Gallie, D. and White, M. (1993), Employee Commitment and Skills Revolution, PSI Publishing, London.
25. Giangreco, A., Sebastiano, A. and Peccei, R. (2008), “Trainees' reactions to training: an analysis of the factors affecting overall satisfaction with training”, The International Journal of Human Resources Management.
26. Gray, C. and Lawless, N. (2000), “Innovations in the distance development of SME management skills”, European Journal of Open, Distance and E-Learning, pp. 1-18.
27. Green, F. (1997), Review of Information on the Benefits of Training for Employers, Research Report No. 7, DfEE, Sheffield.
28. Hamblin, A.C. (1974), Evaluation and Control of Training, Mc Graw-Hill, New York, NY.
29. Holton, F.H. (1996), “The flawed four-level evaluation model”, Human Resources Development Quarterly, Vol. 7 No. 1, pp. 5-20.
30. Hook, K. and Bunce, D. (2001), “Immediate learning in organisational computer training as a function of training intervention affective reaction, and session impact measures”, Applied Psychology: An International Review, Vol. 50 No. 3, pp. 436-454.
31. Kaufman, R., Keller, J. and Watkins, R. (1995), “What works and what doesn't: evaluation beyond Kirkpatrick”, Performance & Instructions, Vol. 35 No. 2, pp. 8-12.
32. Kirkpatrick, D.L. (1959 a), “Techniques for evaluating training programs: part 1 – reactions”, Journal of ASTD, Vol. 13 No. 11, pp. 3-9.
33. Kirkpatrick, D.L. (1959 b), “Techniques for evaluating training programs: part 2 – learning”, Journal of ASTD, Vol. 13 No. 12, pp. 21-26.
34. Kirkpatrick, D.L. (1960 a), “Techniques for evaluating training programs: part 3 – behavior”, Journal of ASTD, Vol. 14 No. 1, pp. 13-18.
35. Kirkpatrick, D.L. (1960 b), “Techniques for evaluating training programs: part 1 – results”, Journal of ASTD, Vol. 14 No. 12, pp. 28-32.
36. Kirkpatrick, D.L. (1967), “Evaluation of training”, in Ed. Craig, R.L. and Ed. Bittel, L.R. (Eds), Training and Development Handbook, Mc Graw-Hill, New York, NY.
37. Kirkpatrick, D.L. (1996 a), “Great ideas revisited”, Training and Development, Vol. 50, pp. 54-59.
38. Kirkpatrick, D.L. (1996 b), “Invited reaction: reaction to Holton article”, Human Resources Development Quarterly, Vol. 7 No. 1, pp. 23-25.
39. Kraiger, K. (2002), “Decision-based evaluation”, in Ed. Kraiger, K. (Ed.), Creating, Implementing, and Maintaining Effective Training and Development: State-of-the-art Lessons for Practice, Jossey-Bass, San Francisco, CA.
40. Kraiger, K., Mc Linden, D. and Casper, W.J. (2004), “Collaborative planning for training impact”, Human Resources Management, Vol. 43 No. 4, pp. 337-351.
41. Lee, S.H. and Pershing, J.A. (2002), “Dimensions and design criteria for developing training reactions evaluations”, Human Resources Development International, Vol. 5 No. 2, pp. 175-197.
42. Lilly, F. (2001), “Four steps to computing training ROI”, working paper, SPHR, Vancouver.
43. Lorenzet, S.J., Salas, E. and Tannenbaum, S.I. (2005), “Benefits from mistakes: the impact of guided errors on learning, performance, and self-efficacy”, Human Resource Development Quarterly, Vol. 16 No. 3, pp. 301-322.
44. Mc Evoy, G.M. (1997), “Organizational change and outdoor management education”, Human Resource Management, Vol. 36 No. 2, pp. 235-250.
45. Mc Killip, J. (2001), “Case studies in job analysis and training evaluation”, International Journal of Training and Development, Vol. 5 No. 4, pp. 283-9.
46. Malone, T.W. and Laubacher, R.J. (1998), “The dawn of the e-Lance economy”, Harvard Business Review, Vol. 76 No. 5, pp. 144-152.
47. Mann, S. and Robertson, I.T. (1996), “What should training evaluations evaluate?”, Journal of European Industrial Training, Vol. 20 No. 9, pp. 14-20.
48. Molenda, M., Pershing, J.R. and Reigeluth, C.M. (1996), “Designing instructional systems”, in Ed. Craig, R.L. (Ed.), The ASTD Training and Development Handbook: a Guide to Human Resource Development, Mc Graw-Hill, New York, NY.
49. Morgan, R.B. and Casper, W.J. (2000), “Examining the factor structure of participant reactions to training: a multidimensional approach”, Human Resources Development Quarterly, Vol. 11 No. 3, pp. 301-317.
50. Noe, R.A. (1986), “Trainees' attributes and attitudes: neglected influences on training effectiveness”, The Academy of Management Review, Vol. 11 No. 4, pp. 736-749.
51. Olsen, J.H. (1998), “The evaluation and enhancement of training transfer”, International Journal of Training and Development, Vol. 2 No. 1, pp. 61-75.
52. Pershing, J.A. and Pershing, J.L. (2001), “Ineffective reaction evaluation”, Human Resources Development Quarterly, Vol. 12 No. 1, pp. 73-90.
53. Phillips, J.J. (1997), Handbook of Training Evaluation and Measurement Methods, Gulf Publishing, Houston, TX.
54. Phillips, J.J. (2003), Return on Investment in Training and Performance Improvement Programs, Butterworth-Heinemann, Boston, MA.
55. Plant, R.A. and Ryan, R.J. (1992), “Training evaluation: a procedure for validating an organization's investment in training”, Journal of European Industrial Training, Vol. 16 No. 10, pp. 22-31.
56. Rowold, J. (2007), “Individual influences on knowledge acquisition in a call center training context in Germany”, International Journal of Training and Development, Vol. 11 No. 1, pp. 21-34.
57. Russ-Eft, D.F., Dickinson, P.D. and Levine, R. (2005), “Instructor quality affecting emergency medical technician (EMT) preparedness: a LEADS project”, International Journal of Training and Development, Vol. 9 No. 4, pp. 256-270.
58. Sackett, P.R. and Mullen, E.J. (1993), “Beyond formal experimental design: towards an expanded view of the training evaluation process”, Personnel Psychology, Vol. 46 No. 3, pp. 613-627.
59. Santos, A. and Stuart, M. (2003), “Employee perceptions and their influence on training effectiveness”, Human Resources Management Journal, Vol. 13 No. 1, pp. 27-45.
60. Sculley, J. and Byrne, J.A. (1987), Odyssey. Pepsi to Apple … A Journey of Adventure, Ideas, and the Future, Harper & Row, New York, NY.
61. Senge, P.M. (1990), The Fifth Discipline: The Art and Practice of the Learning Organization, Century, London.
62. Seyler, D.L., Holton, E.F. III, Bates, R.A., Burnett, M.F. and Carvalho, M.A. (1998), “Factors affecting motivation to transfer training”, International Journal of Training and Development, Vol. 2 No. 1, pp. 2-16.
63. Shelton, S. and Alliger, G. (1993), “Who's afraid of level 4 evaluation? A practical approach”, Training & Development, June, pp. 43-46.
64. Simms, R.R. (1993), “Evaluating public sector training programs”, Public Personnel Management, Vol. 22 No. 4, pp. 591-615.
65. Sitzmann, T., Brown, K.G., Casper, W.J., Ely, K. and Zimmerman, R.D. (2008), “A review and meta-analysis of the nomological network of trainee reactions”, Journal of Applied Psychology.
66. Sugrue, B. (2003), State of the Industry: ASTD's Annual Review of US and International Trends in Workplace Learning and Performance, ASTD American Society for Training and Development, Alexandria, VA.
67. Sugrue, B. and Rivera, R.J. (2005), 2005 State of the Industry: ASTD Annual Review of Trends in Workplace Learning and Performance, ASTD American Society for Training and Development, Alexandra, VA.
68. Swanson, R.A. and Sleezer, C.M. (1987), “Training effectiveness evaluation”, Journal of European Industrial Training, Vol. 11 No. 4, pp. 7-16.
69. Talbot, C. (1992), “Evaluation and validation: a mixed approach”, Journal of European Industrial Training, Vol. 16 No. 5, pp. 26-32.
70. Tan, J.A., Hall, R.J. and Boyce, C. (2003), “The role of employee reactions in predicting training”, Human Resource Development Quarterly, Vol. 14 No. 4, pp. 397-411.
71. Tannenbaum, S.I. and Woods, S.B. (1992), “Determining a strategy for evaluating training: operating within organizational constraints”, Human Resources Planning, Vol. 15 No. 2, pp. 63-81.
72. Taylor, P.J., O'Driscoll, P. and Binning, J.F. (1998), “A new integrated framework for training needs analysis”, Human Resource Management Journal, Vol. 8 No. 2, pp. 29-50.
73. Toplis, J. (2001), “Training evaluations – reflections on the first steps”, Professional News, pp. 354-367.
74. Tracy, J.B., Tannenbaum, S.I. and Kavanaugh, M.J. (1995), “Applying trained skills on the job: the importance of work environment”, Journal of Applied Psychology, Vol. 80, pp. 239-252.
75. Tracy, J.B., Hinkin, T.R., Tannenbaum, S. and Mathieu, J.E. (2001), “The influence of individual characteristics and the work environment on varying levels of training outcomes”, Human Resources Development Quarterly, Vol. 12 No. 1, pp. 5-22.
76. Tyler, K. (2002), “Evaluating evaluations”, HRM Magazine, June, pp. 85-93.
77. Van Buren, M.E. and Erskine, W. (2002), The 2002 ASTD State of Industry Report, ASTD American Society of Training and Development, Alexandria, VA.
78. Wang, G.G. and Wilcox, D. (2006), “Training evaluation: knowing more than is practiced”, Advances in Developing Human Resources, Vol. 8 No. 4, pp. 528-530.
79. Warr, P., Allan, C. and Birdi, K. (1999), “Predicting THRMee levels of training outcome”, Journal of Occupational and Organizational Psychology, Vol. 72, pp. 351-375.
80. Warr, P., Bird, M. and Rackham, N. (1970), Evaluation of Management Training: a Practical Framework, with Cases, for Evaluating Training Needs and Results, Gower, London.
81. Worthen, B.R. and Sanders, J.R. (1987), Educational Evaluation, Longman, New York, NY.