Evaluating Ethics Education Programs: A Multilevel Approach

Ethics and Behavior

Volumn 25, Issue 1, 2015, Pages 37-60

  Mumford, Michael D ^a   Steele, Logan ^a   Watts, Logan L ^a  

^a UNIVERSITY OF OKLAHOMA   (United States)

Author keywords

engineering ethics; ethics education; evaluation; measures; RCR education; scientific ethics

Indexed keywords

EID: 84919848060     PISSN: 10508422     EISSN: None     Source Type: Journal    
DOI: 10.1080/10508422.2014.917417     Document Type: Article

References (85)

1. Aguinis, H., & Kraiger, K. (2009). Benefits of training and development for individuals and teams, organizations, and society. Annual Review of Psychology, 60, 451–474.
2. Anderson, M. S. (2010). Creating the ethical academy: A systems approach to understanding misconduct and empowering change. New York, NY: Routledge.
3. Antes, A. L., Brown, R. P., Murphy, S. T., Waples, E. P., Mumford, M. D., Connelly, S., & Devenport, L. D. (2007). Personality and ethical decision-making in research: The role of perceptions of self and others. Journal of Empirical Research on Human Research Ethics: International Journal, 2, 15–34.
4. Antes, A. L., Murphy, S. T., Waples, E. P., Mumford, M. D., Brown, R. P., Connelly, S., & Devenport, L. D. (2009). A meta-analysis of ethics instruction effectiveness in the sciences. Ethics & Behavior, 19, 379–402.
5. Antes, A. L., Thiel, C. E., Martin, L. E., Stenmark, C. K., Connelly, S., Devenport, L. D., & Mumford, M. D. (2012). Applying cases to solve ethical problems: The significance of positive and process-oriented reflection. Ethics & Behavior, 22, 113–130.
6. Antes, A. L., Wang, X., Mumford, M. D., Brown, R. P., Connelly, S., & Devenport, L. D. (2010). Evaluating the effects that existing instruction on responsible conduct of research has on ethical decision making. Academic Medicine, 85, 519–526.
7. Arthur, W., J., Bennett, W., J., Edens, P. S., & Bell, S. T. (2003). Effectiveness of training in organizations: A meta-analysis of design and evaluation features. Journal of Applied Psychology, 88, 234–245.
8. Arvey, R. D., Cole, D. A., Hazucha, J. F., & Hartanto, F. M. (1985). Statistical power of training evaluation designs. Personnel Psychology, 38, 493–507.
9. Baldwin, T. T., & Ford, J. K. (1988). Transfer of training: A review and directions for future research. Personnel Psychology, 41, 63–105.
10. Baldwin, T. T., Ford, J. K., & Blume, B. D. (2009). Transfer of training 1988–2008: An updated review and agenda for future research. International Review of Industrial and Organizational Psychology, 24, 41–70.
11. Bebeau, M. J., & Thoma, S. J. (1994). The impact of a dental ethics curriculum on moral reasoning. Journal of Dental Education, 58, 684–692.
12. Borman, W. C., & Motowidlo, S. J. (1993). Expanding the criterion domain to include elements of contextual performance. In N. Schmitt & W. C. Borman (Eds.), Personnel selection in organizations (pp. 71-98). San Francisco, CA: Jossey-Bass.
13. Braunschweiger, P., & Goodman, K. W. (2007). The CITI program: An international online resource for education in human subjects protection and the responsible conduct of research. Academic Medicine, 82, 861–864.
14. Brock, M. E., Vert, A., Kligyte, V., Waples, E. P., Sevier, S. T., & Mumford, M. D. (2008). Mental models: An alternative evaluation of a sensemaking approach to ethics instruction. Science and Engineering Ethics, 14, 449–472.
15. Campion, M. A., & Campion, J. E. (1987). Evaluation of an interviewee skills training program in a natural field experiment. Personnel Psychology, 40, 675–691.
16. Clarkeburn, H. (2002). A test for ethical sensitivity in science. Journal of Moral Education, 31, 439–453.
17. Clarkeburn, H., Downie, J. R., & Matthew, B. O. B. (2002). Impact of an ethics programme in a life sciences curriculum. Teaching in Higher Education, 7, 65–79.
18. Cook, T. D., & Campbell, D. T. (1979). Quasi-experimentation: Design & analysis issues for field settings. Boston, MA: Houghton Mifflin.
19. Council of Graduate Schools. (2012). Research and scholarly integrity in graduate education. Washington, DC: Author.
20. Crain, A. L., Martinson, B. C., & Thrush, C. R. (2013). Relationships between the survey of organizational research climate (SORC) and self-reported research practices. Science and Engineering Ethics, 19, 835–850.
21. Cullen, J. G., Sawzin, S. A., Sisson, G. R., & Swanson, R. A. (1978). Cost effectiveness: A model for assessing the training investment. Training and Development Journal, 32, 24–29.
22. Dalal, R. S. (2005). A meta-analysis of the relationship between organizational citizenship behavior and counterproductive work behavior. The Journal of Applied Psychology, 90, 1241–1255.
23. Dansereau, F., & Yammarino, F. J. (2002). Overview: The many faces of multi-level issues. Research in Multi Level Issues, 1, xiii–xix.
24. Davis, M. A., Curtis, M. B., & Tschetter, J. D. (2003). Evaluating cognitive training outcomes: Validity and utility of structural knowledge assessment. Journal of Business and Psychology, 18, 191–206.
25. Drake, M. J., Griffin, P. M., Kirkman, R., & Swann, J. L. (2005). Engineering ethical curricula: Assessment and comparison of two approaches. Journal of Engineering Education, 94, 223–231.
26. DuBois, J. M., & Dueker, J. M. (2009). Teaching and assessing the responsible conduct of research: A delphi consensus panel report. Journal of Research Administration, 40, 49–70.
27. DuBois, J. M., Dueker, J. M., Anderson, E. E., & Campbell, J. (2008). The development and assessment of an NIH-funded research ethics training program. Academic Medicine, 83, 596–603.
28. Ericsson, K. A., & Charness, N. (1994). Expert performance. American Psychologist, 49, 725–747.
29. Fisher, C. B., Fried, A. L., & Feldman, L. G. (2009). Graduate socialization in the responsible conduct of research: A national survey on the research ethics training experiences of psychology doctoral students. Ethics & Behavior, 19, 496–518.
30. Ford, J. K., Smith, E. M., Weissbein, D. A., Gully, S. M., & Salas, E. (1998). Relationships of goal orientation, metacognitive activity, and practice strategies with learning outcomes and transfer. Journal of Applied Psychology, 83, 218–233.
31. Ford, J. K., & Weissbein, D. A. (1997). Transfer of training: An updated review and analysis. Performance Improvement Quarterly, 10, 22–41.
32. Gawthrop, J. C., & Uhlemann, M. R. (1992). Effects of the problem-solving approach in ethics training. Professional Psychology: Research and Practice, 23, 38–42.
33. Goldstein, I. L. (1980). Training in work organizations. Annual Review of Psychology, 31, 229–272.
34. Griffin, R. P. (2010). Means and ends: Effective training evaluation. Industrial and Commercial Training, 42, 220–225.
35. Griffin, R. P. (2011). Workplace learning evaluation: A conceptual model and framework. Industrial and Commercial Training, 43, 172–178.
36. Gruys, M. L., & Sackett, P. R. (2003). Investigating the dimensionality of counterproductive work behavior. International Journal of Selection and Assessment, 11, 30–42.
37. Harkrider, L., MacDougall, A. E., Bagdasarov, Z., Johnson, J. F., Thiel, C. E., Mumford, M. D., … Devenport, L. D. (2013). Structuring case-based ethics training: How comparing cases and structured prompts influence training effectiveness. Ethics & Behavior, 23, 179–198.
38. Harkrider, L. N., Thiel, C. E., Bagdasarov, Z., Mumford, M. D., Johnson, J. F., Connelly, S., & Devenport, L. D. (2012). Improving case-based ethics training with codes of conduct and forecasting content. Ethics & Behavior, 22, 258–280.
39. Hauenstein, N. M., Findlay, R. A., & McDonald, D. P. (2010). Using situational judgment tests to assess training effectiveness: Lessons learned evaluating military equal opportunity advisor trainees. Military Psychology, 22, 262–281.
40. Heitman, E., & Bulger, R. E. (2005). Assessing the educational literature in the responsible conduct of research for core content. Accountability in Research, 12, 207–224.
41. Helton-Fauth, W., Gaddis, B., Scott, G., Mumford, M., Devenport, L., Connelly, S., & Brown, R. (2003). A new approach to assessing ethical conduct in scientific work. Accountability in Research: Policies and Quality Assurance, 10, 205–228.
42. Izzo, G. (2000). Compulsory ethics education and the cognitive moral development of salespeople: A quasi-experimental assessment. Journal of Business Ethics, 28, 223–241.
43. James, L., James, L., & Asch, D. (1990). The measuring of organizations: The role of cognition and values. In B. Schneider (Ed.), Organizational climate and culture (pp. 40–89). San Francisco, CA: Jossey-Bass.
44. Johnson, J. F., Bagdasarov, Z., Harkrider, L. N., MacDougall, A. E., Connelly, S., Devenport, L. D., & Mumford, M. D. (2013). The effects of note-taking and review on sensemaking and ethical decision-making. Ethics & Behavior, 23, 299–323.
45. Kirkpatrick, D. (1996). Revisiting Kirkpatrick’s four-level model. Training & Development, 50, 54–59.
46. Kirkpatrick, D. L., & Kirkpatrick, J. D. (2007). Implementing the four levels: A practical guide for effective evaluation of training programs. San Francisco, CA: Berrett-Koehler.
47. Kligyte, V., Marcy, R. T., Waples, E. P., Sevier, S. T., Godfrey, E. S., Mumford, M. D., & Hougen, D. F. (2008). Application of a sensemaking approach to ethics training in the physical sciences and engineering. Science and Engineering Ethics, 14, 251–278.
48. Kohlberg, L. (1984). Essays on moral development: Vol. 2. The psychology of moral development: The nature and validity of moral stages. San Francisco, CA: Harper & Row.
49. Kraiger, K., Ford, J. K., & Salas, E. (1993). Application of cognitive, skill-based, and affective theories of learning outcomes to new methods of training evaluation. Journal of Applied Psychology Monograph, 78, 311–328.
50. Loe, T. W., Ferrell, L., & Mansfield, P. (2000). A review of empirical studies assessing ethical decision making in business. Journal of Business Ethics, 25, 185–204.
51. MacDougall, A. E., Harkrider, L. N., Bagdasarov, Z., Johnson, J. F., Thiel, C. E., Peacock, J., … Connelly, S. (2014). Examining the effects of incremental case presentation and forecasting outcomes on case-based ethics instruction. Ethics & Behavior, 24, 126–150.
52. Mann, S., & Robertson, I. T. (1996). What should training evaluations evaluate? Journal of European Industrial Training, 20, 14–20.
53. Mecca, J. T., Medeiros, K. E., Giorgini, V., Gibson, C., Mumford, M. D., Connelly, S., & Devenport, L. D. (2014). The influence of compensatory strategies on ethical decision making. Ethics & Behavior, 24, 73–89.
54. Messick, S. (1989). Validity. In R. L. Linn (Ed.), Educational measurement (3rd ed.). New York, NY: Macmillan.
55. Mirabal, T. E. (1978). Forecasting future training costs. Training and Development Journal, 32, 78–87.
56. Morgan, R. B., & Casper, W. J. (2000). Examining the factor structure of participant reactions to training: A multidimensional approach. Human Resource Development Quarterly, 11, 301–317.
57. Mumford, M. D. (in press). Assessing the effectiveness of responsible conduct of research training: Key findings and viable procedures. In R. Nerem (Ed.), Responsible science. Washington, DC: National Academies of Science.
58. Mumford, M. D., Connelly, S., Brown, R. P., Murphy, S. T., Hill, J. H., Antes, A. L., … Devenport, L. D. (2008). A sensemaking approach to ethics training for scientists: Preliminary evidence of training effectiveness. Ethics & Behavior, 18, 315–339.
59. Mumford, M. D., Connelly, S., Murphy, S. T., Devenport, L. D., Antes, A. L., Brown, R. P., … Waples, E. P. (2009). Field and experience influences on ethical decision-making in the sciences. Ethics & Behavior, 19, 263–289.
60. Mumford, M. D., Devenport, L. D., Brown, R. P., Connelly, S., Murphy, S. T., Hill, J. H., & Antes, A. L. (2006). Validation of ethical decision making measures: Evidence for a new set of measures. Ethics & Behavior, 16, 319–345.
61. Mumford, M. D., Marks, M. A., Connelly, M. S., Zaccaro, S. J., & Reiter-Palmon, R. (2000). Development of leadership skills: Experience and timing. Leadership Quarterly, 11, 87–114.
62. Mumford, M. D., Murphy, S. T., Connelly, S., Hill, J. H., Antes, A. L., Brown, R. P., & Devenport, L. D. (2007). Environmental influences on ethical decision making: Climate and environmental predictors of research integrity. Ethics & Behavior, 17, 337–366.
63. Mumford, M. D., Waples, E. P., Antes, A. L., Brown, R. P., Connelly, S., Murphy, S. T., & Devenport, L. D. (2010). Creativity and ethics: The relationship of creative and ethical problem-solving. Creativity Research Journal, 22, 74–89.
64. National Academy of Engineering. (2009). Ethics education and scientific and engineering research. Washington, DC: National Academies Press.
65. National Academy of Sciences. (1992). Responsible science: Ensuring integrity of the research process. Washington, DC: National Academies Press.
66. National Institute of Medicine. (2002). Integrity in scientific research. Washington, DC: National Academies Press.
67. Ostroff, C. (1991). Training effectiveness measures and scoring schemes: A comparison. Personnel Psychology, 44, 353–374.
68. Pieters, H. M., Touw-Otten, F. W., & De Melker, R. A. (1994). Simulated patients in assessing consultation skills of trainees in general practice vocational training: A validity study. Medical Education, 28, 226–233.
69. Peterson, D. K. (2002). The relationship between unethical behavior and the dimensions of the ethical climate questionnaire. Journal of Business Ethics, 41, 313–326.
70. Podsakoff, P. M., MacKenzie, S. B., Moorman, R. H., & Fetter, R. (1990). Transformational leader behaviors and their effects on followers’ trust in leader, satisfaction, and organizational citizenship behaviors. The Leadership Quarterly, 1, 107–142.
71. Prince, M. (2004). Does active learning work? A review of the research. Journal of Engineering Education, 93, 223–231.
72. Rest, J. R. (1986). Moral development: Advances in research and theory. New York, NY: Praeger.
73. Saks, A. M., & Burke, L. A. (2012). An investigation into the relationship between training evaluation and the transfer of training. International Journal of Training and Development, 16, 118–127.
74. Salas, E., & Cannon-Bowers, J. A. (2001). The science of training: A decade of progress. Annual Review of Psychology, 52, 471–499.
75. Salas, E., Milham, L. M., & Bowers, C. A. (2003). Training evaluation in the military: Misconceptions, opportunities, and challenges. Military Psychology, 15, 3–16.
76. Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Boston, MA: Houghton Mifflin.
77. Siegel, D. H. (1983). Can research and practice be integrated in social work education? Journal of Education for Social Work, 19, 12–19.
78. Sonenshein, S. (2007). The role of construction, intuition, and justification in responding to ethical issues at work: The sensemaking-intuition model. Academy of Management Journal, 32, 1022–1040.
79. Stenmark, C. K., Antes, A. L., Thiel, C. E., Caughron, J. J., Wang, X., & Mumford, M. D. (2011). Consequences identification in forecasting and ethical decision-making. Journal of Empirical Research on Human Research Ethics, 6, 25–32.
80. Thiel, C. E., Bagdasarov, Z., Harkrider, L., Johnson, J. F., & Mumford, M. D. (2012). Leader ethical decision-making in organizations: Strategies for sensemaking. Journal of Business Ethics, 107, 49–64.
81. Thiel, C. E., Connelly, S., & Griffith, J. A. (2011). The influence of anger on ethical decision making: Comparison of a primary and secondary appraisal. Ethics & Behavior, 21, 380–403.
82. Thiel, C. E., Connelly, S., Harkrider, L., Devenport, L. D., Bagdasarov, Z., Johnson, J. F., & Mumford, M. D. (2013). Case-based knowledge and ethics education: Improving learning and transfer through emotionally rich cases. Science and Engineering Ethics, 19, 265–286.
83. Victor, B., & Cullen, J. B. (1988). The organizational bases of ethical work climate. Administrative Science Quarterly, 33, 101–125.
84. Waples, E. P., Antes, A. L., Murphy, S. T., Connelly, S., & Mumford, M. D. (2009). A meta-analytic investigation of business ethics instruction. Journal of Business Ethics, 87, 133–151.
85. Wells, D., & Schminke, M. (2001). Ethical development and human resources training: An integrative framework. Human Resource Management Review, 11, 135–158.