Improving undergraduate STEM education: The efficacy of discipline-based professional development

Science Advances

Volumn 3, Issue 2, 2017, Pages

  Manduca, Cathryn A ^a   Iverson, Ellen R ^a   Luxenberg, Michael ^b   Heather Macdonald, R ^c   McConnell, David A ^d   Mogk, David W ^e   Tewksbury, Barbara J ^f  

^a CARLETON COLLEGE   (United States)

^b Professional Data Analysts Inc   (United States)

^c The College of William and Mary   (United States)

^d NORTH CAROLINA STATE UNIVERSITY   (United States)

^e MONTANA STATE UNIVERSITY   (United States)

^f HAMILTON COLLEGE   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE; CUTTING TOOLS; EDUCATION; PROFESSIONAL ASPECTS; REGRESSION ANALYSIS; STUDENTS; SURVEYS; TEACHING;

CONTRIBUTING FACTOR; DIRECT OBSERVATIONS; INSTRUCTIONAL PRACTICES; LEARNING ABOUT TEACHINGS; OBSERVATION PROTOCOL; PROFESSIONAL DEVELOPMENT; PROFESSIONAL DEVELOPMENT PROGRAM; SCIENTIFIC RESEARCHES;

STEM (SCIENCE, TECHNOLOGY, ENGINEERING AND MATHEMATICS);

EDUCATION; GEOLOGY; HUMAN; QUESTIONNAIRE; STANDARDS; TEACHING; UNIVERSITY;

EARTH SCIENCES; FACULTY; HUMANS; SURVEYS AND QUESTIONNAIRES; TEACHING;

EID: 85027062298     PISSN: None     EISSN: 23752548     Source Type: Journal    
DOI: 10.1126/sciadv.1600193     Document Type: Article

References (62)

1. President’s Council of Advisors on Science and Technology, “Engage to excel: Producing one million additional college graduates with degrees in science, technology, engineering, and mathematics” (Executive Office of the President, 2012); http://files.eric.ed.gov/fulltext/ED541511.pdf.
2. National Science Foundation (NSF), “Shaping the future: New expectations for undergraduate education in science, mathematics, engineering, and technology” (nsf96139, NSF, 1996); http://nsf.gov/publications/pub_summ.jsp?ods_key=nsf96139.
3. S. Freeman, S. L. Eddy, M. McDonough, M. K. Smith, N. Okoroafor, H. Jordt, M. P. Wenderoth, Active learning increases student performance in science, engineering, and mathematics. Proc. Natl. Acad. Sci. U.S.A. 111, 8410–8415 (2014).
4. S. Singer, K. A. Smith, Discipline-based education research: Understanding and improving learning in undergraduate science and engineering. J. Eng. Educ. 102, 468–471 (2013).
5. S. L. Eddy, K. A. Hogan, Getting under the hood: How and for whom does increasing course structure work? CBE Life Sci. Educ. 13, 453–468 (2014).
6. D. C. Haak, J. HilleRisLambers, E. Pitre, S. Freeman, Increased structure and active learning reduce the achievement gap in introductory biology. Science 332, 1213–1216 (2011).
7. D. Ebert-May, T. L. Derting, J. Hodder, J. L. Momsen, T. M. Long, S. E. Jardeleza, What we say is not what we do: Effective evaluation of faculty professional development programs. Bioscience 61, 550–558 (2011).
8. National Research Council, Discipline-Based Education Research: Understanding and Improving Learning in Undergraduate Science and Engineering (The National Academies Press, 2012).
9. C. Henderson, A. Beach, N. Finkelstein, Facilitating change in undergraduate STEM instructional practices: An analytic review of the literature. J. Res. Sci. Teach. 48, 952–984 (2011).
10. K. Eagan, “Becoming more student centered? An examination of teaching practices across stem and non-stem disciplines between 2004 and 2014” (2016): https://sloan.org/storage/app/media/files/STEM_Higher_Ed/STEM_Faculty_Teaching_Practices.pdf.
11. C. A. Manduca, D. W. Mogk, B. Tewksbury, R. H. Macdonald, S. P. Fox, E. R. Iverson, K. Kirk, J. McDaris, C. Ormand, M. Bruckner, On the Cutting Edge: Teaching help for geoscience faculty. Science 327, 1095–1096 (2010).
12. Science Education Resource Center (SERC), “On the Cutting Edge” (SERC, 2016); http://serc.carleton.edu/NAGTWorkshops/index.html.
13. C. Wilson, “Status of the Geoscience Workforce 2014” (American Geosciences Institute, 2014).
14. SERC, “Overall Philosophy of Cutting Edge Workshop Design” (SERC, 2016); http://serc.carleton.edu/NAGTWorkshops/about/design.html.
15. C. Amundsen, M. Wilson, Are we asking the right questions? A conceptual review of the educational development literature in higher education. Rev. Educ. Res. 82, 90–126 (2012).
16. C. D’Avanzo, Post–vision and change: Do we know how to change? CBE Life Sci. Educ. 12, 373–382 (2013).
17. D. Ebert-May, T. L. Derting, T. P. Henkel, J. M. Maher, J. L. Momsen, B. Arnold, H. A. Passmore, Breaking the cycle: Future faculty begin teaching with learner-centered strategies after professional development. CBE Life Sci. Educ. 14, ar22 (2015).
18. C. A. Manduca, D. W. Mogk, N. Stillings, “Bringing research on learning to the geosciences” (SERC, 2004); http://serc.carleton.edu/files/research_on_learning/ROL0304_2004.pdf.
19. D. C. Edelson, Learning-for-use: A framework for the design of technology-supported inquiry activities. J. Res. Sci. Teach. 38, 355–385 (2001).
20. A. Kezar, S. Gehrke, “Communities of transformation and their work in achieving scale of STEM reform” (Pullias Center for Higher Education, 2015); www.uscrossier.org/pullias/wp-content/uploads/2016/01/communities-of-trans.pdf.
21. R. H. Macdonald, C. A. Manduca, D. W. Mogk, B. J. Tewksbury, Teaching methods in undergraduate geoscience courses: Results of the 2004 On the Cutting Edge survey of U.S. faculty. J. Geosci. Educ. 53, 237–252 (2005).
22. J. A. McLaughlin, E. Iverson, R. Kirkendall, C. Manduca, M. Bruckner, “On the Cutting Edge 2010 evaluation report” (SERC, 2010); https://serc.carleton.edu/files/NAGTWorkshops/2009_cutting_edge_evaluation_1265409435.pdf.
23. Rockman et al, “On The Cutting Edge Project: Year 3 independent evaluation report” (SERC, 2013); http://serc.carleton.edu/files/NAGTWorkshops/about/2013_external_evaluation_repor.pdf.
24. D. Sawada, M. Piburn, E. Judson, J. Turley, K. Falconer, R. Benford, I. Bloom, Measuring reform practices in science and mathematics classrooms: The Reformed Teaching Observation Protocol. Sch. Sci. Math. 102, 245–253 (2002).
25. SERC, “Classroom observation project: Understanding and improving our teaching” (SERC, 2016); http://serc.carleton.edu/NAGTWorkshops/certop/index.html.
26. E. F. Holton, Holton’s evaluation model: New evidence and construct elaborations. Adv. Develop. Hum. Resour. 7, 37–54 (2005).
27. E. W. L. Cheng, I. Hampson, Transfer of training: A review and new insights. Int. J. Manag. Rev. 10, 327–341 (2008).
28. J. Gitonga, “Transfer of learning in continuing medical education (CME): A conceptual model”, paper presented at the International Research Conference in the Americas of the Academy of Human Resource Development, Indianapolis, IN, 28 February to 4 March 2007.
29. S. M. Wilson, Professional development for science teachers. Science 340, 310–313 (2013).
30. L. M. Desimone, Improving impact studies of teachers’ professional development: Toward better conceptualizations and measures. Educ. Res. 38, 181–199 (2009).
31. M. S. Garet, A. C. Porter, L. Desimone, B. F. Birman, K. S. Yoon, What makes professional development effective? Results from a national sample of teachers. Am. Educ. Res. J. 38, 915–945 (2001).
32. M. Dancy, C. Henderson, Pedagogical practices and instructional change of physics faculty. Am. J. Phys. 78, 1056–1063 (2010).
33. J. E. Froyd, M. Borrego, S. Cutler, C. R. Henderson, M. J. Prince, Estimates of use of research-based instructional strategies in core electrical or computer engineering courses. IEEE Trans. Edu. 56, 393–399 (2013).
34. E. F. Redish, Teaching Physics with the Physics Suite (John Wiley and Sons, 2003).
35. E. Mazur, Peer Instruction: A User’s Manual (Prentice Hall, 1997).
36. T. J. Lund, M. Pilarz, J. B. Velasco, D. Chakraverty, K. Rosploch, M. Undersander, M. Stains, The best of both worlds: Building on the COPUS and RTOP observation protocols to easily and reliably measure various levels of reformed instructional practice. CBE Life Sci. Educ. 14, ar18 (2015).
37. American Association for the Advancement of Science (AAAS), “Describing and measuring undergraduate STEM teaching practice: A report from a national meeting on the measurement of undergraduate science, technology, engineering and mathematics (STEM) teaching” (AAAS, 2013); http://ccliconference.org/files/2013/11/Measuring-STEM-Teaching-Practices.pdf.
38. K. Eagan, E. B. Stolzenberg, J. B. Lozano, M. C. Aragon, M. R. Suchard, S. Hurtado, Undergraduate Teaching Faculty: The 2013-2014 HERI Faculty Survey (University of California, 2014).
39. W. Condon, E. R. Iverson, C. A. Manduca, C. Rutz, G. Willett, Faculty Development and Student Learning: Assessing the Connections (Scholarship of Teaching and Learning, Indiana Univ. Press, 2015).
40. D. L. Kirkpatrick, Evaluation of training, in Training and Development Handbook, R. L. Craig, L. R. Bittel, Eds. (McGraw-Hill, 1967), pp. 87–112.
41. S. Naquin, E. Holton, Motivation to improve work through learning in human resource development. Hum. Res. Dev. Int. 6, 355–370 (2003).
42. M. K. Smith, E. L. Vinson, J. A. Smith, J. D. Lewin, M. R. Stetzer, A campus-wide study of STEM courses: New perspectives on teaching practices and perceptions. CBE Life Sci. Educ. 13, 624–635 (2014).
43. J. Fairweather, “Linking evidence and promising practices in science, technology, engineering, and mathematics (STEM) undergraduate education” (The National Academies National Research Council Board of Science Education, 2009).
44. T. T. Baldwin, J. K. Ford, Transfer of training: A review and directions for future research. Person. Psychol. 41, 63–105 (1988).
45. M. J. Prince, M. Borego, S. Cutler, C. Henderson, J. E. Froyd, Use of research-based instructional strategies in core chemical engineering courses. Chem. Eng. Educ. 47, 27–37 (2013).
46. C. Henderson, M. Dancy, Barriers to the use of research-based instructional strategies: The influence of both individual and situational characteristics. Phys. Rev. S.T. Phys. Educ. Res. 3, 020102-1–020102-14 (2007).
47. S. E. Brownell, K. D. Tanner, Barriers to faculty pedagogical change: Lack of training, time, incentives, and… tensions with professional identity? CBE Life Sci. Educ. 11, 339–346 (2012).
48. E. Wenger-Trayner, B. Wenger-Trayner, “Communities of practice: A brief introduction” (Wenger-Trayner, 2016); http://wenger-trayner.com/introduction-to-communities-ofpractice.
49. E. F. Holton, R. A. Bates, W. E. A. Ruona, Development of a generalized learning transfer system inventory. Hum. Resource Dev. Q. 11, 333–360 (2000).
50. S. Gehrke, A. Kezar, STEM reform outcomes through communities of transformation. Change 48, 30–38 (2016).
51. K. L. Kenyon, M. E. Onorato, A. J. Gottesman, J. Hoque, S. G. Hoskins, Testing CREATE at community colleges: An examination of faculty perspectives and diverse student gains. CBE Life Sci. Educ. 15, 1–19 (2016).
52. M. A. Pelch, D. A. McConnell, Challenging instructors to change: A mixed methods investigation on the effects of material development on the pedagogical beliefs of geoscience instructors. Int. J. STEM Educ. 3, 1–18 (2016).
53. C. Rutz, J. Lauer-Glebov, Assessment and innovation: One darn thing leads to another. Assessing Writing 10, 80–99 (2005).
54. D. Russ-Eft, H. Preskill, Evaluating learning, performance, and change initiatives, in Evaluation in Organizations: A Systematic Approach to Enhancing Learning, Performance, and Change (Basic Books, ed. 2, 2009), chap. 3.
55. D. A. Budd, K. Kraft, D. A. McConnell, T. Vislova, Characterizing teaching in introductory geology courses: Measuring classroom practices. J. Geosci. Educ. 61, 461–475 (2013).
56. M. Q. Patton, Qualitative Research & Evaluation Methods: Integrating Theory and Practice (SAGE, 2015).
57. C. Henderson, R. Cole, J. Froyd, D. Gilbuena, R. K. Friedrichsen, C. Stanford, Designing Educational Innovations for Sustained Adoption: A How-to Guide for Education Developers Who Want to Increase the Impact of Their Work (Increase the Impact, 2015).
58. SERC, “On the Cutting Edge evaluation” (SERC, 2015); http://serc.carleton.edu/NAGTWorkshops/about/evaluation.html.
59. J. Gerring, Case Study Research: Principles and Practices (Cambridge Univ. Press, 2007).
60. SERC, “Activity design: Questions to consider when designing or reviewing an activity” (SERC, 2012); http://serc.carleton.edu/NAGTWorkshops/servicelearning/workshop10/activity_design/index.html.
61. B. Glaser, A. Strauss, Discovery of Grounded Theory: Strategies for Qualitative Research (Aldine, 1967).
62. D. Siegle, “Educational research basics: Excel spreadsheet to calculate instrument reliability estimates,” http://researchbasics.education.uconn.edu/excel-spreadsheet-tocalculate-instrument-reliability-estimates.