Dimensions and design criteria for developing training reaction evaluations

Human Resource Development International

Volumn 5, Issue 2, 2002, Pages 175-197

  Lee, Sung Heum ^a   Pershing, James A ^b  

^a Hanyang University   (South Korea)

^b INDIANA UNIVERSITY   (United States)

Author keywords

Evaluation; Level; Reaction Evaluation; Reactionnaire Abuses; Reactionnaire Design; Training Program Evaluation

Indexed keywords

EID: 84993001242     PISSN: 13678868     EISSN: 14698374     Source Type: Journal    
DOI: 10.1080/13678860110053678     Document Type: Article

References (79)

1. Alliger, G. M. and Janak, E. A. (1989) 'Kirkpatrick's levels of training criteria:thirty years later', Personnel Psychology 42(2):331-42.
2. American Society for Training and Development (1996) The 1996 American Society for Training and Development Report on Trends that Affect Corporate Learning & Performance, 2nd edn, Alexandria, VA:Author.
3. Antheil, J. H. and Casper, I. G. (1986) 'Comprehensive evaluation model:a tool for the evaluation of nontraditional educational programs', Innovative Higher Education 11(1):55-64.
4. Babbie, E.R. (1990) Survey Research Methods, 2nd edn, Belmont, CA:Wadsworth.
5. Ban, C. and Faerman, S. R. (1990) 'Issues in the evaluation of management training', Public Productivity & Management Review 13(3):271-86.
6. Basarab, D. J. (1994) 'Assessing the quality of training evaluation studies', Performance & Instruction 33(3):19-22.
7. Basarab, D. J. and Root, D. K. (1992) The Training Evaluation Process:A Practical Approach to Evaluating Corporate Training Programs, Boston, MA:Kluwer Academic.
8. Bassi, L. J. and Van Buren, M. E. (1999) 'The 1999 ASTD state of the industry report', Training & Development 53(1), a 27-page supplement.
9. Beer, V. (1990) 'Surveymania in HRD program evaluation', Performance Improvement Quarterly 3(3):2-11.
10. Beer, V. and Bloomer, A. C. (1986) 'Levels of evaluation', Educational Evaluation and Policy Analysis 8 (4):335-45.
11. Biner, P. M. (1993) 'The development of an instrument to measure student attitudes toward televised courses', The American Journal of Distance Education 7(1):62-73.
12. Bramley, P. (1996) Evaluating Training Effectiveness:Benchmarking your Training Activity against Best Practice, 2nd edn, London:McGraw-Hill.
13. Bramley, P. and Kitson, B. (1994) 'Evaluating training against business criteria', Journal of European Industrial Training 18(1):10-14.
14. Bramley, P. and Newby, A. C. (1984) 'The evaluation of training part I:clarifying the concept', Journal of European Industrial Training 8(6):10-16.
15. Brandenburg, D. C. (1982) 'Training evaluation:what's the current status?', Training and Development Journal 36(5):14, 16-22.
16. Brinkerhoff, R. O. (1989) 'Using evaluation to transform training', New Directions for Program Evaluation 44:5-20.
17. Campion, M. A. and Campion, J. E. (1987) 'Evaluation of an interview skills training program in a natural field setting', Personnel Psychology 40:675-91.
18. Cangelosi, J. S. (1991) Evaluating Classroom Instruction, New York:Longman.
19. Carnevale, A. P., Gainer, L. J. and Villet, J. (1990) Training in America:The Organization and Strategic Role of Training, San Francisco, CA:Jossey-Bass.
20. Clark, R. C. (2000) 'Four architectures of instruction', Performance Improvement 39(10):31-8.
21. Clement, R. W. (1981) 'Evaluating the effectiveness of management training:progress during the 1970's and prospects for the 1980's', Human Resource Management 20(4):8-13.
22. Clement, R. W. (1982) 'Testing the hierarchy theory of training evaluation:an expanded role for trainee reactions', Public Personnel Management Journal 12(2):176-84.
23. Dionne, P. (1996) 'The evaluation of training activities:a complex issue involving different stakes', Human Performance Development Quarterly 7(3):279-86.
24. Dixon, N. M. (1988) 'Meet training's goals without reaction forms', Personnel Journal 66(8):108,110,112,115.
25. Dixon, N. M. (1990) 'The relationship between trainee responses on participant reaction forms and posttest scores', Human Resources Development Quarterly 1(2):129-37.
26. Dubin, S. S., Mezack, M. and Neidig, R. (1974) 'Improving the evaluation of management development programs', Training and Development Journal 28(6):42-6.
27. Easterby-Smith, M. (1985) 'Turning course evaluation from an end to a means', Personnel Management 17(4):25-7.
28. Faerman, S. R. and Ban, C. (1993) 'Trainee satisfaction and training impact:issues in training evaluation', Public Productivity & Management Review 16(3):299-314.
29. Fisk, C. N. (1991) ASTD Trainer's Toolkit:Evaluation Instruments, San Diego, CA:Pfeiffer.
30. Forsyth, I., Jolliffe, A. and Stevens, D. (1995) Evaluating Course:Practical Strategies for Teachers, Lecturers and Trainers, London:Kogan Page.
31. Geroy, G. D. and Wright, P. C. (1988) 'Evaluation research:a pragmatic, program-focused, research strategy for decision-makers', Performance Improvement Quarterly 1(3):17-26.
32. Goldstein, I. L. (1974) Training:Program Development and Evaluation, Monterey, CA:Brooks/Cole.
33. Goldstein, I. L. (1993) Training in Organizations:Needs Assessment, Development, and Evaluation, 3rd edn, Pacific Grove, CA:Brooks/Cole.
34. Hale, J. (1998) 'Evaluation:it's time to go beyond levels, 1, 2, 3, and 4', Performance Improvement 37(2):30-4.
35. Hamblin, A. C. (1974) Evaluation and Control of Training, London:McGraw-Hill.
36. Holton, E. F. Ill (1996) 'The flawed four-level evaluation model', Human Resource Development Quarterly 7(1):5-21.
37. Kaufman, R., Keller, J. and Watkins, R. (1995) 'What works and what doesn't:evaluation beyond Kirkpatrick', Performance & Instruction 35(2):8-12.
38. Keller, J. M. (1996) Evaluating Diversity Training:17 Ready-to-Use Tools, San Diego, CA:Pfeiffer.
39. Kent, R. A. (1993) Marketing Research in Action, London:Routledge.
40. Kirkpatrick, D. L. (1959a) 'Techniques for evaluating training programs', Journal of the American Society of Training Directors 13(11):3-9.
41. Kirkpatrick, D. L. (1959b) 'Techniques for evaluating training programs:part 2 - learning', Journal of the American Society of Training Directors 13(12):21-6.
42. Kirkpatrick, D. L. (1960a) 'Techniques for evaluating training programs:part 3 - behavior', Journal of the American Society of Training Directors 14(1):13-18.
43. Kirkpatrick, D. L. (1960b) 'Techniques for evaluating training programs:part 4 - results', Journal of the American Society of Training Directors 14(2):28-32.
44. Kirkpatrick, D. L. (1983) 'Four steps to measuring training effectiveness', Personnel Administrator 28(11):19-21,23-5.
45. Kirkpatrick, D. L. (1988) A Practical Guide for Supervisory Training and Development, 2nd edn, Reading, MA:Addison-Wesley.
46. Kirkpatrick, D. L. (1994) Evaluating Training Programs:The Four Levels, San Francisco, CA:Berrett-Koehler.
47. Kusy, M. E. Jr. (1988) 'The effects of types of training evaluation on support of training among corporate managers', Performance Improvement Quarterly 1(2):23-30.
48. Lee, S. H. (1998) 'Making reaction evaluation a more useful tool in the evaluation of corporate training programs:reactionnaire dimensions and design criteria', unpublished doctoral dissertation, Indiana University, Bloomington.
49. Lindholm, T. R. (1953) 'Supervisory training and employee attitude', Journal of Industrial Training 7(6):17-20.
50. McGehee, W. and Thayer, P. W. (1961) Training in Business and Industry, New York:Wiley.
51. Molenda, M., Pershing, J. A. and Reigeluth, C. M. (1996) 'Designing instructional systems', in R. L. Craig (ed.) The ASTD Training and Development Handbook:A Guide to Human Resource Development, 4th edn, New York:McGraw-Hill, pp. 266-93.
52. Nadler, L. and Nadler, Z. (1994) Designing Training Programs:The Critical Events Model, 2nd edn, Houston, TX:Gulf.
53. Neuman, W. L. (1997) Social Research Methods:Qualitative and Quantitative Approaches, 3rd edn, Boston, MA:Allyn & Bacon.
54. Newby, A. C. (1992) Training Evaluation Handbook, San Diego, CA:Pfeiffer.
55. Newstrom, J. W. (1978) 'Catch-22:the problems of incomplete evaluation of training', Training and Development Journal 32(11):22-4.
56. Ostroff, C. (1991) 'Training effectiveness measures and scoring schemes:a comparison', Personnel Psychology 44(2):353-74.
57. Parker, B. L. (1986) 'Summative evaluation in training and development', Journal of Industrial Teacher Education 23(2):29-55.
58. Pershing, J. A. and Lee, S. H. (1997) 'Uses and abuses of "happy sheets:" how to design and use reactionnaires in evaluating training programs', paper presented at the 1997 International Conference of the International Society for Performance Improvement. Anaheim, CA.
59. Pershing, J. A. and Lee, S. H. (1998) 'Uses and abuses of level one evaluation:how to design and use reactionnaires in evaluating training programs', paper presented at the 1998 International Conference of the International Society for Performance Improvement. Chicago, IL.
60. Pershing, J. A., Lee, S. H. and Dye, K. (1997) 'Making level 1 evaluation a meaningful tool', paper presented at the 1997 International Conference of the American Society for Training and Development, Washington, DC.
61. Phillips, J. J. (1995) 'Return on investment:beyond the four levels', in E. Holton (ed.) Academy of Human Resource Development (AHRD) Conference Proceedings, St. Louis, MO (ERIC Document Reproduction Services No. ED 398 370).
62. Phillips, J. J. (1997a) Handbook of Training Evaluation and Measurement Methods:Proven Models and Methods for Evaluating any HRD Program, 3rd edn, Houston, TX:Gulf.
63. Phillips, J. J. (1997b) Return on Investment in Training and Performance Improvement Programs, Houston, TX:Gulf.
64. Phillips, S. (1993) Evaluation, London:Library Association.
65. Plant, R. A. and Ryan, R. J. (1994) 'Who is evaluating training?', Journal of European Industrial Training 18(5):27-30.
66. Pont, T. (1996) Developing Effective Training Skills:A Practical Guide to Designing and Delivering Group Training, 2nd edn, London:McGraw-Hill.
67. Richey, R. C. (1995) 'Trends in instructional design:emerging theory-based models', Performance Improvement Quarterly 8(3):96-110.
68. Robinson, D. G. and Robinson, J. C. (1989) Training for Impact:How to Link Training to Business Needs and Measure the Results, San Francisco, CA:Jossey-Bass.
69. Saari, L.M., Johnson, T.R., McLaughlin, S.D. and Zimmerle, D.M. (1988) "A survey of management training and education practices in U. S. companies', Personnel Psychology 41(4):731-43.
70. Schouborg, G. (1993) Flex:A Flexible Tool for Continuously Improving Tour Evaluation of Training Effectiveness. Amherst, MA:HRD.
71. Sheatsley, P. B. (1983) 'Questionnaire construction and item writing', in P. H. Rossi, J. D. Wright and A. B. Anderson (eds) Handbook of Survey Research, San Diego, CA:Academic, pp. 195-230.
72. Smith, M. E. and Brandenburg, D. C. (1991) 'Summative evaluation', Performance Improvement Quarterly 4(2):35-58.
73. Tannenbaum, S. I. and Yukl, G. (1992) 'Training and development in work organizations', in M. R. Rosenberg and L. W. Porter (eds) Annual Review of Psychology, PaIo Alto, CA:Annual Reviews, pp. 399-441.
74. Training (1996) 'Industry report:who's learning what?', 33(10):53-4, 57-9, 61, 63, 65.
75. Training (1999) 'Industry report 1999', 36(10):37-8.
76. Warr, P. and Bunce, D. (1995) 'Trainee characteristics and the outcomes of open learning', Personnel Psychology 48(2):347-75.
77. Warr, P., Bird, M. and Rackham, N. (1970) Evaluation of Management Training:A Practical Framework, with Cases, for Evaluating Training Meeds and Results, London:Gower.
78. Worthen, B. R. (1990) 'Program evaluation', in H. J. Walberg and G. D. Haertel (eds) The International Encyclopedia of Educational Evaluation, New York:Pergamon, pp. 42-7.
79. Worthen, B. R. and Sanders, J. R. (1987) Educational Evaluation:Alternative Approaches and Practical Guidelines, New York:Longman.