Formative evaluation: An integrative practice model and case study

Personnel Psychology

Volumn 55, Issue 4, 2002, Pages 951-983

  Brown, Kenneth G ^a   Gerhardt, Megan W ^a  

^a UNIVERSITY OF IOWA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 0036926101     PISSN: 00315826     EISSN: None     Source Type: Journal    
DOI: 10.1111/j.1744-6570.2002.tb00137.x     Document Type: Article

References (78)

1. Alliger G.M., Katzman S. (1997). Reconsidering training evaluation: Heterogeneity of variance as a training effect. In Ford J.K., Kozlowski S.W.J., Kraiger K., Salas E., Teachout M. (Eds.), Improving training effectiveness in work organizations (pp. 223-246). Mahwah, NJ: Erlbaum.
2. Alliger G.M., Tannenbaum S.I., Bennett W., Traver H. (1997). A meta-analysis of the relations among training criteria. Personnel Psychology, 50, 341-358.
3. Arvey R.D., Cole D.A. (1989). Evaluationg change due to training. In Goldstein I. & Associates (Eds.), Training and development in organizations (pp. 89-117). San Francisco: Jossey-Bass.
4. Baldwin T.T., Danielson C., Wiggenhorn W. (1997). The evolution of learning strategies in organizations: From employee development to business redefinition. Academy of Management Executive, 11, 47-58.
5. Bassi L.J., Benson G., Cheney S. (1996). The top ten trends. Training and Development Journal, 50(11), 28-42.
6. Beyer B.K. (1995). How to conduct a formative evaluation. Alexandria, VA: Association for Supervision and Curriculum Development.
7. Brinkerhoff R.O. (1987). Achieving results from training. San Francisco: Jossey-Bass.
8. Broad M.L., Newstrom J.W. (1992). Transfer of training: Action-packed strategies to ensure high payoff from training investments. Reading, M.A.: Addison-Wesley.
9. Brown J.L., Kiernan N.E. (2001). Assessing the subsequent effect of a formative evaluation on a program. Evaluation and Program Planning 24, 129-143.
10. Brown K.G. (2001). Using computers to deliver training: Which employees learn and why? Personnel Psychology, 54, 271-296.
11. Brown K.G., Ford J.K. (2002). Using computer technology in training: Building an infrastructure for learning. In Kraiger K. (Ed.) Creating, implementing, and managing effective training and development (pp. 192-233). San Francisco: Jossey-Bass.
12. Brown K.G., Milner K.R., Ford J.K. (2001). Repurposing instructor-led training into Web-based training: A case study and lesson learned. In Khan B. (Ed.), Web-based training. Englewood Cliffs, NJ: Educational Technology Publications.
13. Button S.B., Mathieu J.E., Zajac D.M. (1996). The development and psychometric evaluation of measures of learning goal and performance goal orientation. Organizational Behavior and Human Decision Processes, 67, 26-48.
14. Byrum D.C., (1992). Formative evaluation of computer courseware: An experimental comparison of two methods. Journal of Educational Computing Research 8, 69-80.
15. Chinen C., Boutin F. (1992). Cognitive style FD/FI as a learner selection criterion in formative evaluation: A qualitative analysis. Performance Improvement Quarterly, 5, 24-33.
16. Colquitt J.A., Simmering M.J. (1998), Conscientiousness, goal orientation, and motivation to learn during the learning process: A longitudinal study. Journal of Applied Psychology, 83, 654-665.
17. Dessinger J., Brown K.G., Reesman M., Elliot L. (1998). Evaluating at a distance: The IDL Group looks at training outcomes and transfer of training in interaetive satellite training, in Schreiber, D.A., Berge Z.L. (Eds.), Distance training: How innovative organizations are using technology to maximize foaming and meet business objectives (pp. 328-350). San Francisco: Jossey-Bass.
18. Dick W., Carey L. (1996). The systematic design of instruction (4th ed.). Reading, MA: Addison-Wesley.
19. Dipboye R.L. (1997). Organizational barriers to implementing a raional model of training. In Quiñones M.A., Ehrenstein A. (Eds,), Training for a rapidly changing workplace: Applications of psychological research (pp. 31-60). Washington, DC: American Psychological Association.
20. Dubin S.S. (1990). Maintaining competence through updating. In Willis S.L., Dubin S.S. (Eds.), Maintaining professional competence: Approaches to career enhancement, vitality, and success throughout a work life (pp. 9-43). San Francisco: Jossey-Bass.
21. Dweck C.S. (1986). Motivational processes affecting learning. American Psychologist, 41, 1040-1048.
22. Fisher S.L., Ford J.K. (1998). Differential effects of learner effort and goal orientation on two learning outcomes. Personnel Psychology, 51, 397-420.
23. Ford J.K., Wroten S.P. (1984). Introducing new methods for conducting training evaluation and for linking training evaluation to program redesign, Personnel Psychology, 37, 651-666.
24. Gagné R.M. (1996). Learning processes and instruction. Training Research Journal, 1, 17-28.
25. Geis G.L. (1987). Formative evaluation: Developmental testing and expert review. Performance and Instruction, 26, 1-8.
26. Goldstein I.L. (Ed.). (1989). Training and development in organizations. San Francisco: Jossey-Bass.
27. Goldstein I.L., Ford J.K. (2002). Training in organizations (4th ed.). Belmont, C.A.: Wadsworth.
28. Grove D.A., Ostroff C., (1990). Program evaluation, In Wexley K., Hinrichs J. (Eds.), Developing human resources. Washington, DC: BNA Books.
29. Holton E.F. (1996). The flawed four-level evaluation model. Human Resource Development Quarterly, 7, 5-21.
30. Huselid M.A. (1995). The impact of human resource management practices on turnover, productivity, and corporate financial performance. Academy of Management Journal, 38, 635-672.
31. Kirkpatrick D.L. (1974). Evaluation of training. In Craig R.L. (Ed.), Training and development handboak, (2nd ed. pp. 154-15-27). New York: McGraw Hill.
32. Kirkpatrick D.L. (1996). Evaluation. In Craig R.L. (Ed.), The ASTD training and development handbook (4th ed., pp. 294-312). New York: McGraw-Hill.
33. Kozlowski S.W.J., Brown, K.G., Weissbein D.A., Cannon-Bowers J., Salas E. (2000). A multilevel perspective on training effectiveness: Enhancing horizontal and vertical transfer. In Klein K.J., Kozlowski S.W.J. (Eds.), Multilevel theory, research, and methods in organizations (pp. 157-210). San Francisco: Jossey-Bass.
34. Kozlowski S.W.J., Gully S.M., Brown K.G., Salas E., Smith E.M., Nason E.R. (2001). Effects of training goals and goal orientation traits on multi-dimensional training outcomes and performance adaptability. Organizational Behavior and Human Decision Processes, 85, 1-31.
35. Kraiger K. (Ed.). (2002). Creating, implementing and managing effective training and development. San Francisco: Jossey-Bass.
36. Kraiger K., Ford J.K., Salas E. (1993). Application of cognitive, skill-based, and affective theories of learning outcomes to new methods of training evaluation. Journal of Applied Psychology, 78, 311-328.
37. Lee F., Sheldon K.M., Turban D.B. (in press). Personality and goal striving process: The influence of achievement goal patterns, goal level, and mental focus on performance and enjoyment. Journal of Applied Psychology.
38. LeMaistre K., Weston C. (1996). The priorities established among data sources when instructional designers revise written materials. Educational Technology Research and Development, 44, 61-70.
39. Martocchio J.J., Baldwin T.T. (1997). The evolution of strategic organizational training: New objectives and research agenda. In Ferris G.R. (Ed.), Research in personnel and human resources management, Vol. 15, pp. 1-46. Greenwich, CT: Jai Press.
40. Martocchio J.J., Judge T.A. (1997). Relationship between conscientiousness and learning employee training: Mediating influences of self-deception and self-efficacy. Journal of Applied Psychology, 82, 764-773.
41. Mathieu J.E., Leonard R.L. (1987). Applying utility concepts to a training program in supervisory skills: A time-based approach. Academy of Management Journal, 30, 316-335.
42. Medley-Mark V., Weston C. (1988). A comparison of student feedback obtained from three methods of formative evaluation of instructional materials. Instructional Science, 17, 3-27.
43. Moore M.G., Kearsley G. (1996). Distance education: A systems view. Belmont, C.A.: Wadsworth.
44. Morgan R.B., Casper W.J. (2000). Examining the factor structure of participant reactions to training: A multidimensional approach. Human Resource Development Quarterly, 11, 301-317.
45. Morrow C.C., Jarrett M.Q., Rupinski M.T. (1997). An investigation of the effect and economic utility of corporate-wide training. Personnel Psychology, 50, 91-119.
46. Nielson J. (1994a). Heuristic evaluation. In Nielson J., Mack R.L. (Eds.), Usability inspection methods (pp. 25-62). New York: Wiley.
47. Nielson J. (1994b). Guerilla H.C.I.: Using discount usability engineering to penetrate the intimidation barrier. In Bias R.G., Mayhew D.J. (Eds.), Cost-justifying usability (pp. 245-272). San Diego, CA: Academic Press.
48. Nixon E.K., Lee D. (2001). Rapid prototyping in the instructional design process. Performance Improvement Quarterly, 14, 95-116.
49. Noe R.A. (2002). Employee training and development (2nd ed.). Boston: McGraw-Hill.
50. Norman D.A. (1992). Turn signals are the facial expressions of automobiles. Reading, MA: Addison-Wesley.
51. Northrup P.T. (1995). Concurrent formative evaluation: Guidelines and implications for multimedia designers. Educational Technology, 35(6), 24-31.
52. Olian J.D., Durham C.C., Kristof-Brown A., Brown K.G., Pierce R.L., Kunder L. (1998). Designing management training and development for competitive advantage: Lessons from the best. Human Resource Planning 21(1), 20-31.
53. Phillips J.J. (1996). Measuring results in training. In Craig R.L. (Ed.), The ASTD training and development handbook (4th ed., pp. 313-341). New York: McGraw-Hill.
54. Raghuram S., Arvey R.D. (1994). Business strategy links with staffing and training practices. Human Resource Planning, 17, 55-73.
55. Reiguluth C.M. (1999). What is instructional-design theory and how is it changing? In Reiguluth C.M. (Ed.), Instructional-design theories and models (pp. 5-29). Mahwah, NJ: Erlbaum.
56. Robinson D.G., Robinson J.C. (1989). Training for impact How to link training to business needs and measure the results, San Francisco: Jossey-Bass.
57. Rossi P.H., Freeman H.E., Lipsey M.W. (1999). Evaluation: A systematic approach (6th ed.). Newbury Park, CA: Sage.
58. Rouiller J.Z., Goldstein I.L., (1993). The relationship between organizational transfer climate and positive transfer of training. Human Resource Development Quarterly, 4, 377-390.
59. Sackett P.R., Mullen E.J., (1993). Beyond format experimental design: Toward an expanded view of the training evaluation process. Personnel Psychology, 46, 613-627.
60. Salsa E., Cannon-Bowers J.A. (2001). The science of training: A decade of progress. Annual Review of Psychology, 52, 471-499.
61. Saroyan A. (1992). Differences in expert practice: A case from formative evaluation. Instructional Science, 21, 451-472.
62. Schmidt R., Bjork R.A. (1992). New conceptualizations of practice: Common principles in three paradigms suggest new concepts for training. Psychological Science, 3, 207-217.
63. Sherman T.M. (1977). The effects of student formative evaluation of instruction on teacher behavior. Journal of Educational Technology Systems, 6, 209-217.
64. Simon S.J., Werner J.M. (1996). Computer training through behavior modeling, self-paced, and instructional approaches: A field experiment. Journal of Applied Psychology, 81, 648-659.
65. Sonnenfeld J.A., Peiperl M.A. (1988). Staffing policy as a strategic response: A typology of career systems. Academy of Management Review, 13, 588-600.
66. Tannenbaum S.L. (1997). Enhancing continuous learning: Diagnostic findings from multiple companies. Human Resource Management, 36, 437-452.
67. Tannenbaum S.L., Woods S.B. (1992). Determining a strategy for evaluating training: Operating within organizational constraints. Human Resource Planning 15, 63-81.
68. Tannenbaum S.L., Yukl G. (1992). Training and development in work organizations. Annual Review of Psychology, 43, 399-441.
69. Tessmer M. (1994). Formative evaluation alternatives. Performance Improvement Quarterly, 7, 3-18.
70. Tessmer M., Wedman J. (1995). Context-sensitive instructional design models: A response to design research, studies, and criticism. Performance Improvement Quarterly, 8, 38-54.
71. Tracey J.B., Tannenbaum S.I., Kavanagh M.J. (1995). Applying trained skills on the job: The importance of the work environment. Journal of Applied Psychology, 80, 239-252.
72. Tripp S.D., Bichelmeyer B. (1990). Rapid prototyping: An alternative instructional design strategy. Educational Technology Research and Development, 38, 31-44.
73. Warr P., Bunce D. (1995). Trainee characteristics and the outcomes of open learning. Personnel Psychology, 48, 347-375.
74. Weston C., LeMaistre C., McAlpine L., Bordonaro T. (1997). The influence of participants in formative evaluation on the improvement of learning from written instructional materials. Instructional Science, 25, 369-386.
75. Weston C., McAlpine L., Bordonaro T. (1995). A model for understanding formative evaluation in instructional design. Educational Technology Research and Development, 43, 29-48.
76. White B.S., Branch R.M. (2001). Systematic pilot testing as a step in the instructional design process of corporate training and development. Performance Improvement Quarterly 14, 75-94.
77. Wognum I., Lam J.F. (2000). Stakeholder involvement in strategic HRD aligning: The impact on HRD effectiveness. International Journal of Training and Development, 4, 98-110.
78. Zohar D. (1980). Safety climate in industrial organization: Theoretical and applied implications. Journal of Applied Psychology, 65, 96-102.