Self-Reported Learning Gains: A Theory and Test of College Student Survey Response

Research in Higher Education

Volumn 54, Issue 2, 2013, Pages 201-226

  Porter, Stephen R ^a  

^a NORTH CAROLINA STATE UNIVERSITY   (United States)

Author keywords

College students; Learning gains; Survey research; Validity

Indexed keywords

EID: 84874227568     PISSN: 03610365     EISSN: None     Source Type: Journal    
DOI: 10.1007/s11162-012-9277-0     Document Type: Article

References (48)

1. American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (1999). Standards for Educational and psychological testing. Washington, DC: American Educational Research Association.
2. Astin, A. W., & Lee, J. J. (2003). How risky are one-shot cross-sectional assessments of undergraduate students? Research in Higher Education, 44(6), 657-672.
3. Biglan, A. (1973a). The characteristics of subject matter in different academic areas. Journal of Applied Psychology, 57(3), 195-203.
4. Biglan, A. (1973b). Relationships between subject matter characteristics and the structure and output of university departments. Journal of Applied Psychology, 57(3), 204-213.
5. Bloom, H. S., Hill, C. J. J., Black, A. R., & Lipsey, M. (2008). Performance trajectories and performance gaps as achievement effect-size benchmarks for educational interventions. Journal of Research on Educational Effectiveness, 1, 289-328.
6. Bowman, N. A. (2010a). Assessing learning and development among diverse college students. New Directions for Instutional Research, 145, 53-71.
7. Bowman, N. A. (2010b). Can 1st-year students accurately report their learning and development? American Educational Research Journal, 47, 466-496.
8. Bowman, N. A. (2011a). Examining systematic errors in predictors of college student self-reported gains. New Directions for Instutional Research, 150, 7-19.
9. Bowman, N. A. (2011b). Validity of college self-reported gains at diverse institutions. Educational Researcher, 40, 22-24.
10. Bradburn, N., Rips, L. J., & Shevell, S. (1987). Answering autobiographical questions: The impact of memory and inference on surveys. Science, 123, 157-161.
11. Campbell, C. M., & Cabrera, A. F. (2011). How sound is NSSE?: Investigating the psychometric properties of NSSE at a public, research-extensive institution. Review of Higher Education, 35, 77-103.
12. Ewell, P., McClenney, K., & McCormick, A. C. (2011). Measuring engagement. Inside Higher Education. Crawfordsville: Wabash College.
13. Gottfredson, G. D., & Holland, J. L. (1996). Dictionary of Holland occupational codes. Lutz, FL: Psychological Assessment Resources Inc.
14. Groves, R. M. (2006). Nonresponse rates and nonresponse bias in household surveys. Public Opinion Quarterly, 70, 646-675.
15. Groves, R. M., Cialdini, R. B., & Couper, M. P. (1992). Understanding the decision to participate in a survey. Public Opinion Quarterly, 56(4), 475-495.
16. Groves, R. M., & Peytcheva, E. (2008). The impact of nonresponse rates on nonresponse bias: A meta-analysis. Public Opinion Quarterly, 72, 167-189.
17. Higher Education Data Sharing Consortium. (2012a). HEDS senior survey. Downloaded June 10, 2012, from http://www. hedsconsortium. org/storage/HEDS_Senior_Survey_Sample_02-23-2012. pdf.
18. Higher Education Research Institute. (2012b). College senior survey. Downloaded June 10, 2012, from http://www. heri. ucla. edu/researchers/instruments/FUS_CSS/2012CSS. PDF.
19. Holland, J. L. (1973). Making vocational choices: A theory of vocational personalities and work environment. Upper Saddle River, NJ: Prentice-Hall.
20. Huang, Y., & Healy, C. (1997). The relations of Holland-typed majors to students' freshman and senior work values. Research in Higher Education, 38, 455-477.
21. Keeter, S., Kennedy, C., Dimock, M., Best, J., & Craighill, P. (2006). Gauging the impact of growing nonresponse on estimates from a national RDD telephone survey. Public Opinion Quarterly, 70(5), 759-779.
22. Kuh, G. D. (2001). The national survey of student engagement: Conceptual framework and overview of psychometric properties. Bloomington, IN: Indiana University Center for Postsecondary Research.
23. Kuh, G. D., & Vesper, N. (2001). Do computers enhance or detract from student learning? Research in Higher Education, 42(1), 87-102.
24. Laird, T. F. N., Shoup, R., Kuh, G. D., & Schwarz, M. J. (2008). The effects of discipline on deep approaches to student learning and college outcomes. Research in Higher Education, 49(6), 469-494.
25. Lambert, A. D., Terenzini, P. T., & Lattuca, L. R. (2007). More than meets the eye: Curricular and programmatic effects on student learning. Research in Higher Education, 48(2), 141-168.
26. McCormick, A., Pike, G., Kuh, G., & Chen, P.-S. (2009). Comparing the utility of the 2000 and 2005 carnegie classification systems in research on students's college experiences and outcomes. Research in Higher Education, 50, 144-167. doi: 10. 1007/s11162-008-9112-9.
27. McCormick, A. C., & McClenney, K. (2012). Will these trees ever bear fruit? A response to the special issue on student engagement. Review of Higher Education, 35(2), 307-333.
28. National Survey of Student Engagement. (2012). Sample institutional report. Downloaded June 9, 2012, from http://nsse. iub. edu/_/?cid=402.
29. Pace, R. C. (1985). The credibility of student self-reports. Technical report, Center for the Study of Evaluation, University of California, Los Angeles.
30. Pascarella, E. T., Blaich, C., Martin, G. L., & Hanson, J. M. (2011). How robust are the findings of academically adrift? Change, 43(3), 20-24.
31. Pascarella, E. T., & Padgett, R. (2011). Using institution-level NSSE benchmarks to assess engagement in good practices: A cautionary note. Manuscript, University of Iowa.
32. Pascarella, E. T., & Wolniak, G. C. (2004). Change or not to change-Is there a question? A response to pike. Journal of College Student Development, 45(3), 353-355.
33. Pike, G. R. (2000). The influence of fraternity or sorority membership on students' college experiences and cognitive development. Research in Higher Education, 41(1), 117-139.
34. Pike, G. R. (2006). Students' personality types, intended majors, and college expectations: Further evidence concerning psychological and sociological interpretations of Holland's theory. Research in Higher Education, 47(7), 801-822.
35. Pike, G. R. (2011). Using college students' self-reported learning outcomes in scholarly research. New Directions for Instutional Research, 150, 41-58.
36. Pike, G. R. (2012, June 28). Personal communication.
37. Pike, G. R., & Killian, T. S. (2001). Reported gains in student learning: Do academic disciplines make a difference? Research in Higher Education, 42(4), 429-454.
38. Pike, G. R., Kuh, G., McCormick, A., Ethington, C., & Smart, J. (2011a). If and when money matters: The relationships among educational expenditures, student engagement and students' learning outcomes. Research in Higher Education, 51(1), 81-106.
39. Pike, G. R., Smart, J. C., & Ethington, C. A. (2011b). Differences in learning outcomes across academic environments: Further evidence concerning the construct validity of students' self-reports. Paper presented at the annual conference of the Association for the Study of Higher Education, Raleigh, NC.
40. Pike, G. R., Smart, J. C., & Ethington, C. A. (2012). The mediating effects of student engagement on the relationships between academic disciplines and learning outcomes: An extension of Holland's theory. Research in Higher Education, 53(5), 550-575.
41. Porter, S., & Umbach, P. D. (2006). Student survey response rates across institutions: Why do they vary? Research in Higher Education, 47, 229-247.
42. Porter, S. R. (2011a). Do college student surveys have any validity? Review of Higher Education, 35, 45-76.
43. Porter, S. R. (2011b). Student learning as a measure of quality in higher education. Context for Success Project. Seattle: Gates Foundation.
44. Rothstein, J. (2009). Student sorting and bias in value-added estimation: Selection on observables and unobservables. Education Finance and Policy, 4(4), 537-571.
45. Smart, J. C. (2010). Differential patterns of change and stability in student learning outcomes in Holland's academic environments: The role of environmental consistency. Research in Higher Education, 51, 468-482.
46. Smart, J. C., Feldman, K. A., & Ethington, C. A. (2000). Academic disciplines: Holland's theory and the study of college students and faculty. Nashville, TX: Vanderbilt University Press.
47. Tourangeau, R., Rips, L. J., & Rasinski, K. (2000). The psychology of survey response. Cambridge: Cambridge University Press.
48. Zhao, C.-M., & Kuh, G. D. (2004). Adding value: Learning communities and student engagement. Research in Higher Education, 45(2), 115-138.