Differences in stakeholder perceptions about training evaluation: A concept mapping/pattern matching investigation

Evaluation and Program Planning

Volumn 23, Issue 2, 2000, Pages 211-230

  Michalski, Greg V ^a   Cousins, J Bradley ^b  

^a NORTEL NETWORKS   (Canada)

^b UNIVERSITY OF OTTAWA   (Canada)

Author keywords

Concept mapping and pattern matching; Organizational effectiveness; Power and politics; Stakeholder diversity; Training evaluation research

Indexed keywords

EID: 0000022480     PISSN: 01497189     EISSN: None     Source Type: Journal    
DOI: 10.1016/S0149-7189(00)00005-7     Document Type: Article

References (78)

1. Abernathy, D. J. (1999). Thinking outside the evaluation box. Training & Development, 53(2), 19-23.
2. Alkin, M. C. (1991). Evaluation theory development: II. In: M. W. McLaughlin, & D. C. Phillips, Evaluation and education: At quarter century (pp. 91-112). Chicago: University of Chicago Press Nineteenth Yearbook of the National Society for the Study of Education, Part II.
3. Alkin, M. C., Hofstetter, C. H., & Ai, X. (1998). Stakeholder concepts in program evaluation. In: A. Reynolds, & H. Walberg, Advances in educational productivity, vol. 7 (pp. 87-113). Greenwich, CT: JAI Press.
4. Alliger, G. M., & Janak, E. A. (1989). Kirkpatrick's levels of training criteria: Thirty years later. Personnel Psychology, 42, 331-342.
5. Altschuld, J. W., & Zheng, H. Y. (1995). Assessing the effectiveness of research organizations: An examination of multiple approaches. Evaluation Review, 19(2), 197-216.
6. Anderberg, M. R. (1973). Cluster analysis for applications. New York: Academic Press.
7. Bandura, A. (1986). Social foundations of thought and action: A social cognitive theory. Englewood Cliffs, NJ: Prentice-Hall.
8. Bassi, L., Benson, G., & Cheney, S. (1996). The top ten trends. Training & Development, 50(11), 29-33.
9. Brinkerhoff, R. O. (1989). Using evaluation to transform training. In: R. O. Brinkerhoff, Evaluating training programs in business and industry (pp. 5-20). San Francisco: Jossey-Bass.
10. Broad, M. L., & Newstrom, J. W. (1992). Transfer of training: Action-packed strategies to ensure high payoff from training investments. Reading, MA: Addison-Wesley.
11. Brown D.C. (1994). How managers and training professionals attribute causality for results: Implications for training evaluation. Unpublished Doctoral Dissertation, College of Education, University of Illinois, Urbana-Champaign.
12. Brown, J. S., & Duguid, P. (1991). Organizational learning and communities-of-practice: Toward a unified view of working, learning, and innovation. Organization Science, 2(1), 40-57.
13. Campbell, J. P. (1976). Contributions research can make in understanding organizational effectiveness. In: S. L. Spray, Organizational effectiveness: Theory, research, and utilization (pp. 29-45). Kent, OH: Kent State University Press.
14. Chelimsky, E. (1987). The politics of program evaluation. In: D. S. Cordray, H. S. Bloom, & R. J. Light, Evaluation practice in Review. New directions for program evaluation, vol. 34.
15. Concept Systems Inc. (1996a). Concept system: Facilitator training manual. Ithaca: Author.
16. Concept Systems Inc. [Computer software]. (1996b). Ithaca, NY: Author.
17. Conference Board Inc. (1997). The value of training in the era of intellectual capital: A research report (Report No. 1199-97-RR). New York: Author.
18. Connolly, T., Conlon, E. J., & Deutsch, S. J. (1980). Organizational effectiveness: A multiple-constituency approach. Academy of Management Review, 5(2), 211-217.
19. Cousins, J. B., & Earl, L. M. (1992). The case for participatory evaluation. Educational Evaluation and Policy Analysis, 14(4), 397-418.
20. Cousins, J. B., & Earl, L. M. (Eds.) (1995). Participatory evaluation in education: Studies in evaluation use and organizational learning. London: Falmer.
21. Cousins, J. B., & MacDonald, C. J. (1998). Conceptualizing outcomes for the evaluation of management training in technology-based companies: A participatory concept mapping application. Evaluation and Program Planning, 21(3), 333-344.
22. Davison, M. L. (1983). Multidimensional scaling. New York: Wiley.
23. Darrah, C. N. (1995). Workplace training, workplace learning: A case study. Human Organization, 54(1), 31-41.
24. Drucker, P. F. (1993a). Post-capitalist society. New York: Harper Business.
25. Drucker, P. F. (1993b). Professionals' productivity. Across the Board, 30(9), 50.
26. Drucker, P. F. (1995). Managing in a time of great change. New York: Truman Talley.
27. Everitt, B. (1980). Cluster Analysis (2nd). New York: Halsted Press/ Wiley.
28. Fitz-enz, J. (1988). Proving the value of training. Personnel, March, 17-23.
29. Geber, B. (1995). Does your training make a difference? Prove it!. Training, March, 27-34.
30. Gill, S. J. (1989). Using evaluation to build commitment to training. In: R. O. Brinkerhoff, Evaluating training programs in business and industry (pp. 35-43). San Francisco: Jossey-Bass.
31. Gold, N. (1983). Stakeholders and program evaluations: Characterizations and reflections. In: A. S. Bryk, Stakeholder-based evaluation. New directions for program evaluation, vol. 17 (pp. 63-72). San Francisco: Jossey-Bass.
32. Goleman, D. (1998). Working with emotional intelligence. New York: Bantam.
33. Guba, E. G., & Lincoln, Y. S. (1989). Fourth generation evaluation. Newbury Park: Sage.
34. Hinkin, T. R. (1998). A brief tutorial on the development of measures for use in survey questionnaires. Organizational Research Methods, 1(1), 104-121.
35. Kearsley, G. (1982). Costs, benefits, and productivity in training systems. Reading, MA: Addison.
36. Kirkpatrick, D. L. (Ed.) (1975). Evaluating training programs. Madison, WI: ASTD.
37. Kirkpatrick, D. L. (1994). Evaluating training programs: The four levels. San Francisco: Berrett-Koehler.
38. Kirkpatrick, D. L. (1996). Great ideas revisited. Training & Development, 50(1), 54-59.
39. Kolb, D. G., & Shepherd, D. M. (1997). Concept mapping organizational cultures. Journal of Management Inquiry, 6(4), 282-295.
40. Kruskal, J. B., & Wish, M. (1978). Multidimensional scaling. Beverly Hills, CA: Sage.
41. Lewis, T. (1996). A model for thinking about the evaluation of training. Performance Improvement Quarterly, 9(1), 3-22.
42. Likert, R. (1977). Measuring organizational performance. In: S. J. Carroll Jr, F. T. Paine, & J. B. Miner, The management process: Cases and readings (pp. 316-332). New York: Macmillan (Reprinted from Harvard Business Review, March-April, 1958).
43. Love, A. J. (1991). Internal evaluation: Building organizations from within. Newbury PK: Sage.
44. Mark, M. M., & Shotland, R. L. (1985). Stakeholder-based evaluation and value judgments. Evaluation Review, 9(5), 605-626.
45. Maxwell, J. A. (1996). Qualitative research design: An interactive approach. Thousand Oaks, CA: Sage.
46. McLinden, D. J. (1995). Proof, evidence, and complexity: Understanding the impact of training and development in business. Performance Improvement Quarterly, 8(3), 3-18.
47. McLinden, D. J., & Trochim, W. (1998). Getting to parallel: Assessing the return on expectations of training. Performance Improvement, 57(8), 21-26.
48. Miles, M. B., & Huberman, A. M. (1994). Qualitative data analysis, (2nd ed.). Thousand Oaks, CA: Sage.
49. Miller, T. I. (1994). Designing and conducting surveys. In: J. S. Wholey, H. P. Hatry, & K. E. Newcomer, Handbook of practical program evaluation (pp. 271-292). San Francisco: Jossey-Bass.
50. Mintzberg, H. (1983). Power in and around organizations. Englewood Cliffs, NJ: Prentice-Hall.
51. Moad, J. (1995). Calculating the real benefit of training. Datamation, April, 45-47.
52. Nadler, L. (1984). Human resource development. In: L. Nadler, Handbook of human resource development (pp. 1.1-1.47). New York: Wiley.
53. Palumbo, D. J. (Ed.) (1987). The politics of program evaluation. Newbury Park: Sage.
54. Patton, M. Q. (1997). Utilization-focused evaluation: The new century text, (3rd ed.). Thousand Oaks, CA: Sage.
55. Pepitone, J. S. (1995). Future training. Dallas: AddVantage Learning Press.
56. Pfeffer, J. (1981). Power in organizations. Marshfield, MA: Pitman.
57. Pfeffer, J. (1994). Competitive advantage through people: Unleashing the power of the work force. Boston: Harvard Business School Press.
58. Phillips, J. J. (1996). ROI: The search for best practices. Training & Development, 50(2), 42-47.
59. Phillips, J. J. (1991). Handbook of training evaluation and measurement methods, (2nd ed.). Houston: Gulf.
60. Rizzo-Michelin L.L. (1998). Concept Mapping in Evaluation Practice and Theory: A Synthesis of Current Empirical Research. Unpublished master's thesis, University of Ottawa, Canada.
61. Setze, R. (1997). Performance needs assessment in a corporate setting. In: Paper presented at the meeting of the American Evaluation Association, November, San Diego, CA.
62. Shulha, L. M., & Cousins, J. B. (1997). Evaluation use: Theory research and practice since 1986. Evaluation Practice, 18(3), 195-208.
63. Spray, S. L. (1976). Organizational effectiveness: Theory, research, and utilization introduction. In: S. L. Spray, Organizational effectiveness: Theory, research, and utilization (pp. 1-4). Kent, OH: Kent State University Press.
64. Stake, R. E. (1983). Stakeholder influence in the evaluation of cities-in schools. In: A. S. Bryk, Stakeholder-based evaluation. New directions for program evaluation, vol. 17 (pp. 15-30). San Francisco: Jossey-Bass.
65. Stewart, T. A. (1999). Intellectual capital: The new wealth of organizations. New York: Currency Doubleday.
66. Sumanth, D. J., Omachonu, V. K., & Beruvides, M. G. (1990). A review of the state-of-the-art research on white-collar/knowledge worker productivity. International Journal of Technology Management, 5, 337-355.
67. Tesoro, F. (1998). Implementing an ROI measurement process at Dell computer. Performance Improvement Quarterly, 11(4), 103-114.
68. Thompson, J. D. (1967). Organizations in action. New York: McGraw-Hill.
69. Toffler, A. (1990). Powershift: Knowledge, wealth and violence at the edge of the 21st century. New York: Bantam.
70. Trochim, W. (1985). Pattern matching, validity, and conceptualization in program evaluation. Evaluation Review, 9(5), 575-604.
71. Trochim, W. (Ed.) (1989a). Concept mapping for evaluation and planning (special issue). Evaluation and Program Planning, 12(1) Special Issue.
72. Trochim, W. (1989b). An introduction to concept mapping for planning and evaluation. Evaluation and Program Planning, 12(1), 1-16.
73. Trochim, W. (1989c). Concept mapping: Soft science or hard art? Evaluation and Program Planning, 12(1), 87-110.
74. Trochim, W. (1989d). Outcome pattern matching and program theory. Evaluation and Program Planning, 12(4), 355-366.
75. Trochim, W. (1990). Pattern matching and program theory. In: H. C. Chen, Theory-driven evaluation. New directions for program evaluation. San Francisco, CA: Jossey-Bass.
76. Trochim, W. (1993). The reliability of concept mapping. In: Paper presented at the Annual Meeting of the American Evaluation Association, Dallas, TX., November.
77. Weiss, C. H. (1983). The stakeholder approach to evaluation: origins and promise. In: A. S. Bryk, Stakeholder-based evaluation. New directions for program evaluation, vol. 17 (pp. 3-14). San Francisco: Jossey-Bass.
78. Weiss, C. H. (1987). Where politics and evaluation research meet. In: D. J. Palumbo, The politics of program evaluation (pp. 47-70). Newbury Park: Sage.