Reversing the Logic: An Outcomes-Based Student Typology for Determining "What Works" in Promoting an Array of Engineering-Related Student Learning Outcomes

Educational Evaluation and Policy Analysis

Volumn 36, Issue 2, 2014, Pages 145-169

  Knight, David B ^a  

^a VIRGINIA POLYTECHNIC INSTITUTE AND STATE UNIVERSITY   (United States)

Author keywords

assessment; cluster analysis; student learning; student learning outcomes; student typology

Indexed keywords

EID: 84899622878     PISSN: 01623737     EISSN: 19351062     Source Type: Journal    
DOI: 10.3102/0162373713504987     Document Type: Article

References (91)

1. Accreditation Board for Engineering and Technology (ABET). (2011). Criteria for accrediting engineering programs, 2012-2013. Retrieved from http://www.abet.org/accreditation-criteria-policies-documents/.
2. Abu El-HajT. R. A. (2003). Practicing for equity from the standpoint of the particular: Exploring the work of one urban teacher network. Teachers College Record, 105, 817-845. doi:http://www.tcrecord.org/11136.
3. Adams R. S.,Turns J.,Atman C. J.Educating effective engineering designers: The role of reflective practice.Design Studies. 2003;24:275-294.
4. Alon S.The evolution of class inequality in higher education: Competition, exclusion, and adaptation.American Sociological Review. 2009;74:731-755.
5. American Association of Colleges of Nursing. (2009). Standards for accreditation of baccalaureate and graduate degree nursing programs. Commission on Collegiate Nursing Accreditation. Retrieved from http://www.aacn.nche.edu/ccne-accreditation/standards09.pdf.
6. Ammon B. V.,Bowman J.,Mourad R.Who are our students?: Cluster analysis as a tool for understanding community college student populations.Journal of Applied Research in the Community College. 2008;16:32-44.
7. Anaya G.College impact on student learning: Comparing the use of self-reported gains, standardized test scores, and college grades.Research in Higher Education. 1999;40:499-526.
8. Armor D. J.Sociological methodology: 1973-1974. Costner H., ed. San Francisco, CA: Jossey-Bass; 1974:17-50.
9. Arum R.,Roksa J.Academically adrift: Limited learning on college campuses. Chicago, IL: University of Chicago Press; 2011:.
10. Association of American Colleges and Universities (AAC&U). (2012). Liberal Education and America's Promise (LEAP) essential learning outcomes. Retrieved from http://www.aacu.org/leap/vision.cfm.
11. Association to Advance Collegiate Schools of Business. (2011). Eligibility procedures and accreditation standards for business accreditation. Tampa, FL: AACSB International. Retrieved from http://www.aacsb.edu/accreditation/accounting/standards/Standards-accntg-Jan2012.pdf.
12. Astin A. W.Assessment for excellence: The philosophy and practice of assessment and evaluation in higher education. New York, NY: Macmillan; 1991:.
13. Astin A. W.What matters in college? Four critical years revisited. San Francisco, CA: Jossey-Bass; 1993:.
14. Bahr P. R.The bird's eye view of community colleges: A behavioral typology of first-time students based on cluster analytic classification.Research in Higher Education. 2010;51:724-749.
15. Barber J.Integration of learning: A grounded theory analysis of college students' learning.American Educational Research Journal. 2012;49:590-617.
16. Baruch Y.Response rate in academic studies-A comparative analysis.Human Relations. 1999;52:421-438.
17. Besterfield-Sacre M.,Shuman L. J.,Wolfe H.Modeling undergraduate engineering outcomes.International Journal of Engineering Education. 2002;18:128-139.
18. Bjorklund S. A.,Parente J. M.,Sathianathan D.Effects of faculty interaction and feedback on gains in student skills.Journal of Engineering Education. 2004;93:153-160.
19. Boix Mansilla V. B.,Duraising E. D.Targeted assessment of students' interdisciplinary work: An empirically grounded framework proposed.Journal of Higher Education. 2007;78:215-237.
20. Boughan K.The role of academic process in student achievement: An application of structural equations modeling and cluster analysis to community college longitudinal data.AIR Professional File. 2000;74:1-18.
21. Bowman N. A.Can 1st-year college students accurately report their learning and development?.American Educational Research Journal. 2010;47:466-496.
22. Brotman J. S.,Moore F. M.Girls and science: A review of four themes in the science education literature.Journal of Research in Science Teaching. 2008;45:971-1002.
23. Campbell C. M.,Cabrera A. F.How sound is NSEE? Investigating the psychometric properties of NSEE at a public, research-extensive institution.Review of Higher Education. 2011;35:77-104.
24. CampbellP. B.JollyE.HoeyL.PerlmanL. K. (2002). Upping the numbers: Using research based decision making to increase diversity in the quantitative disciplines. GE Foundation Report. Retrieved from http://www.campbell-kibler.com/upping_the_numbers.pdf.
25. Corrigan M. E.Beyond access: Persistence challenges and the diversity of low-income students.New Directions for Higher Education. 2003;121:25-34.
26. Cox B. E.,McIntosh K. L.,Reason R. D.,Terenzini P. T. Paper presented at the annual conference of the Association for the Study of Higher Education; 2010Indianapolis, IN; 2010.
27. Dempster A. P.,Laird N. M.,Rubin D. B.Maximum likelihood from incomplete data via the EM algorithm.Journal of the Royal Statistical Society: Series B. 1977;39:1-38.
28. DesJardins S.A comment on interpreting odds-ratios when logistic regression coefficients are negative.AIR Professional File. 2001;81 (Fall): 1-7.
29. Graham J. W.Missing data: Making it work in the real world.Annual Review of Psychology. 2009;60:549-576.
30. Hagedorn L. S.,Prather G. Paper presented at the 2005 Annual Forum of the Association for Institutional Research; 2005San Diego, CA; 2005.
31. Harper B. J.,Lambert A. D.,Lattuca L. R. Paper presented at the meeting of the Association for the Study of Higher Education; 2006Anaheim, CA; 2006.
32. Haüssler P.,Hoffmann L.An intervention study to enhance girls' interest, self-concept, and achievement in physics classes.Journal of Research in Science Teaching. 2002;39:870-888.
33. Hu S.,McCormick A. C.An engagement-based student typology and its relationship to college outcomes.Research in Higher Education. 2012;53:738-754.
34. HutchingsP. (2010, April). Opening doors to faculty involvement in assessment (Occasional Paper No. 4). National Institute for Learning Outcomes Assessment. Retrieved from http://www.learningoutcomeassessment.org/documents/PatHutchings_000.pdf.
35. Johnson D. W.,Johnson R.The socialization and achievement crisis: Are cooperative learning experiences the solution?.Applied Social Psychology Annual. 1983;4:119-164.
36. Kalton G.Introduction to survey sampling. Newbury Park, CA: SAGE; 1983:.
37. Knight D. B. Proceedings of the 118th Annual Conference of the American Society for Engineering Education; 2011Vancouver, BC, Canada; 2011.
38. Knight D. B.,Lattuca L. R.,Yin A. C.,Kremer G.,York T.,Ro H. K.An exploration of gender diversity in engineering programs: A curriculum and instruction-based perspective.Journal of Women and Minorities in Science and Engineering. 2012;18:55-78.
39. Kuh G. D.The national survey of student engagement: Conceptual framework and overview of psychometric properties. Bloomington: Indiana University Center for Postsecondary Research and Planning; 2004:.
40. Kuh G. D.Achieving accountability in higher education: Balancing public, academic, and market demands. Burke J. C., ed. San Francisco, CA: Jossey-Bass; 2005:148-172.
41. Kuh G. D.,Hu S.,Vesper N.They shall be known by what they do: An activities based typology of college students.Journal of College Student Development. 2000;41:228-244.
42. KuhG. D.IkenberryS. (2009). More than you think, less than we need: Learning outcomes assessment in American higher education: National Institute for Learning Outcomes Assessment. Retrieved from http://www.learningoutcomeassessment.org/NILOAsurveyresults09.htm.
43. Kuh G. D.,Kinzie J.,Schuh J. H.,Whitt E. J.Student success in college: Creating conditions that matter. San Francisco, CA: Jossey-Bass; 2005:.
44. Kuncel N. R.,Crede M.,Thomas L. L.The validity of self-reported grade point averages, class ranks, and test scores: A meta-analysis and review of the literature.Review of Educational Research. 2005;75:63-82.
45. Laing J.,Swayer R.,Noble J.Accuracy of self-reported activities and accomplishments of college-bound seniors.Journal of College Student Development. 1989;29:362-368.
46. Lattuca L. R.,Knight D. B.,Bergom I.Developing a measure of interdisciplinary competence.International Journal of Engineering Education. 2013;29:726-739.
47. Lattuca L. R.,Stark J. S.Shaping the college curriculum: Academic plans in context. San Francisco, CA: Jossey-Bass; 2009:.
48. Lattuca L. R.,Terenzini P. T.,Volkwein J. F.Engineering change: A study of the impact of EC2000. Baltimore, MD: ABET; 2006:.
49. Lattuca L. R.,Trautvetter L. C.,Codd S.,Knight D. B. Proceedings of the 118th Annual Conference of the American Society for Engineering Education; 2011Vancouver, BC, Canada; 2011.
50. Levin J.,Wyckoff J.Identification of student characteristics that predict persistence and success in an engineering college at the end of the sophomore year: Informing the practice of academic advising. University Park: Pennsylvania State University; 1990:.
51. Liaison Committee on Medical Education. (2011). Functions and structures of a medical school: Standards for accreditation of medical education programs leading to the M.D. degree. Liaison Committee on Medical Education. Retrieved from http://www.lcme.org/functions.pdf.
52. Long J. S.,Freese J.Regression models for categorical dependent variables using Stata. 2006). Regression models for categorical dependent variables using Stata (2nd ed.). College Station, TX: Stata Press.College Station, TX: Stata Press; 2006:.
53. McGrath J. E.,Martin J.,Kulka R. A.Judgment calls in research. Beverly Hills, CA: SAGE; 1982:.
54. Mentkowski M.Creating a context where institutional assessment yields educational improvement.Journal of General Education. 1991;40:255-283.
55. The engineer of 2020: Visions of engineering in the new century. Washington, DC: National Academies Press; 2004:.
56. Rising above the gathering storm: Energizing and employing America for a brighter economic future. Washington, DC: National Academies Press; 2007:.
57. Pace C. R.The credibility of student self-reports. Los Angeles: Center for the Study of Evaluation, University of California at Los Angeles; 1985:.
58. Palmer E.,Terenzini P. T.,Harper B. J.,McKenna A.,Merson D. S. Proceedings of the American Society for Engineering Education; 2011Vancouver, Canada; 2011.
59. Pascarella E. T.,Terenzini P. T.How college affects students: Findings and insights from twenty years of research. San Francisco, CA: Jossey-Bass; 1991:.
60. Pascarella E. T.,Terenzini P. T.How college affects students: Vol. 2. A third decade of research. San Francisco, CA: Jossey-Bass; 2005:.
61. Pawley A. L. Proceedings of the 2004 American Society for Engineering Education Annual Conference & Exposition; 2004Salt Lake City, UT; 2004.
62. Pike G. R.The relationship between self-reports of college experiences and achievement test scores.Research in Higher Education. 1995;36:1-22.
63. Porter S. R.Applications of intermediate/advanced statistics in institutional research. Coughlin M. A., ed. Tallahassee, FL: Association for Institutional Research; 2005:110-131.
64. Porter S. R.Do college student surveys have any validity?.Review of Higher Education. 2011;35:45-76.
65. Porter S. R.,Umbach P. D. Paper presented at the annual meeting of the Association for Institutional Research; 2001Long Beach, CA; 2001.
66. Porter S. R.,Umbach P. D.Student survey response rates across institutions: Why do they vary?.Research in Higher Education. 2006;47:229-247.
67. Quade D.Rank analysis of covariance.Journal of the American Statistical Association. 1967;62:1187-1200.
68. Raudenbush S. W.,Bryk A. S.Hierarchical linear models: Applications and data analysis methods. 2002). Hierarchical linear models: Applications and data analysis methods (2nd ed.). Newbury Park, CA: SAGE.Newbury Park, CA: SAGE; 2002:.
69. Reason R. D.,Cox B. E.,Lutovsky-Quaye B. R.,Terenzini P. T.Faculty and institutional factors that encourage faculty to promote student encounters with difference in first-year courses.Review of Higher Education. 2010;33:391-414.
70. Reason R. D.,Terenzini P. T.,Domingo R. J.First things first: Developing academic competence in the first year of college.Research in Higher Education. 2006;47:149-175.
71. Reason R. D.,Terenzini P. T.,Domingo R. J.Developing social and personal competence in the first year of college.Review of Higher Education. 2007;30:271-299.
72. Ro H. K.,Merson D.,Lattuca L. R.,Terenzini P. T. Proceedings of the 119th Annual Conference of the American Society for Engineering Education; 2012San Antonio, TX; 2012.
73. Seymour E.The loss of women from science, mathematics, and engineering undergraduate majors: An explanatory account.Science Education. 1995;79:437-473.
74. Shadish W. R.,Cook T. D.,Campbell D. T.Experimental and quasi-experimental designs for generalized causal inference. Boston, MA: Houghton Mifflin; 2002:.
75. Shavelson R. J.Measuring college learning responsibility: Accountability in a new era. Palo Alto, CA: Stanford University Press; 2010:.
76. Sheppard S.,Gilmartin S.,Chen H. L.,Donaldson K.,Lichtenstein G.,Eris Ö., Toye G.Exploring the engineering student experience: Findings from the Academic Pathways of People Learning Engineering Survey (APPLES, TR-10-01). Seattle, WA: Center for the Advancement for Engineering Education; 2010:.
77. Shuman L. J.,Besterfield-Sacre M.,McGourty J.The ABET "professional skills"-Can they be taught? Can they be assessed?.Journal of Engineering Education. 2005;94:41-55.
78. Steinley D.K-means clustering: A half-century synthesis.British Journal of Mathematical and Statistical Psychology. 2006;59:1-34.
79. Steinley D.Initializing K-means batch clustering: A critical evaluation of several techniques.Journal of Classification. 2007;24:99-121.
80. Strauss L. C.,Terenzini P. T.The effects of students' in-and out-of-class experiences on their analytical and group skills: A study of engineering education.Research in Higher Education. 2007;48:967-992.
81. Tabachnick B. G.,Fidell L. S.Using multivariate statistics. 2001). Using multivariate statistics (4th ed.). Boston, MA: Allyn & Bacon.Boston, MA: Allyn & Bacon; 2001:.
82. Terenzini P. T.,Reason R. D. Paper presented at the annual conference of the Association for the Study of Higher Education; 2005Philadelphia, PA; 2005.
83. Terenzini P. T.,Reason R. D. Paper presented at the annual conference of the Consortium of Higher Education Researchers (CHER); 2010Oslo, Norway; 2010.
84. Science, technology, engineering and mathematics trends and the role of federal programs: Statement of Cornelia M. Ashby, Director, Education, Workforce, and Income Security Issues (GAO-06-702T). Washington, DC: Author; 2006:.
85. VanDerLinden K.Credit student analysis: 1999 and 2000. Annapolis Junction, MD: Community College Press, American Association of Community Colleges; 2002:.
86. Van Horn P. S.,Green K. E.,Martinussen M.Survey response rates and survey administration in counseling and clinical psychology: A meta-analysis.Educational and Psychological Measurement. 2009;69:389-403.
87. Volkwein J. F.,Lattuca L. R.,Harper B. J.,Domingo R. J.Measuring the impact of professional accreditation on student experiences and learning outcomes.Research in Higher Education. 2007;48:251-282.
88. Ward J. H.Hierarchical grouping to optimize function.Journal of the American Statistical Association. 1963;58:236-244.
89. White House. (2011). Winning the future. Retrieved from www.whitehouse.gov/winning-the-future.
90. Whitt E. J.,Pascarella E. T.,Neisheim B. S. E.,Marth B. P.,Pierson C. T.Differences between women and men in objectively measured outcomes, and the factors that influence those outcomes, in the first three years of college.Journal of College Student Development. 2003;44:587-610.
91. Zhang G.,Anderson T. J.,Ohland M. W.,Thorndyke B. R.Identifying factors influencing engineering student graduation: A longitudinal and cross-institutional study.Journal of Engineering Education. 2004;93:313-320.