Can 1st-year college students accurately report their learning and development?

American Educational Research Journal

Volumn 47, Issue 2, 2010, Pages 466-496

  Bowman, Nicholas A ^a  

^a UNIVERSITY OF NOTRE DAME   (United States)

Author keywords

College students; Self report; Student development; Student learning; Validity

Indexed keywords

EID: 77953714015     PISSN: 00028312     EISSN: None     Source Type: Journal    
DOI: 10.3102/0002831209353595     Document Type: Article

References (74)

1. ACT. (1991). CAAP technical handbook. Iowa City, IA: Author.
2. Anaya, G. (1999). College impact on student learning: Comparing the use of self- reported gains, standardized test scores, and college grades. Research in Higher Education, 40, 499-526.
3. Aronson, E., Ellsworth, P.C., Carlsmith, J.M., & Gonzales, M.H. (1989). Methods of research in social psychology (2nd ed.). New York: McGraw-Hill.
4. Astin, A.W. (1993). What matters in college? San Francisco: Jossey-Bass.
5. Astin, A., Astin, H., Boatsman, K., Bonous-Hammarth, M., Chambers, T., Goldberg, S.,et al.(1996). A social change model of leadership development: Guidebook (Version III). Los Angeles: University of California at Los Angeles, Higher Education Research Institute.
6. Baird, L.L. (1976). Using self-reports to predict student performance (Research Monograph No. 7). Princeton, NJ: College Board.
7. Baird, L.L. (1988). Value added: Using student gains as yardsticks of learning. In C. Adelman (Ed.), Performance and judgment: Essays on principles and practice in the assessment of student learning (pp. 205-216). Washington, DC: U.S. Department of Education, Office of Educational Research and Improvement.
8. Bebeau, M., & Thoma, S. (2003). Guide for the DIT-2. Minneapolis: University of Minnesota, Center for the Study of Ethical Development.
9. Berdie, R.F. (1971). Self-claimed and tested knowledge. Educational and Psychological Measurement, 31, 629-636.
10. Bowman, N.A. (in press-a). Dissonance and resolution: The non-linear effects of diversity courses on well-being and orientations toward diversity. Review of Higher Education.
11. Bowman, N.A. (in press-b). The development of psychological well-being among first-year college students. Journal of College Student Development.
12. Bowman, N.A., & Hill, P.L. (2009). Social desirability and self-reported gains. Unpublished data, University of Notre Dame.
13. Bowman, N.A., & Seifert, T. (2009). Can students accurately assess what affects their learning and development? Manuscript submitted for publication.
14. Burke, J. C. (Ed.). (2005). Achieving accountability in higher education: Balancing public, academic, and market demands. San Francisco: Jossey-Bass.
15. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Mahwah, NJ: Lawrence Erlbaum.
16. Cohen, J., Cohen, P., West, S.G., & Aiken, L.S. (2003). Applied multiple regression correlation analysis for the behavioral sciences (3rd ed.). Mahwah, NJ: Lawrence Erlbaum.
17. Cole, J., & Gonyea, R. (2008, November). Accuracy of self-reported SAT and ACT scores: Implications for research. Paper presented at the annual meeting of the Association for the Study of Higher Education, Jacksonville, FL.
18. Conway, M., & Ross, M. (1984). Getting what you want by revising what you had. Journal of Personality and Social Psychology, 47, 738-748.
19. Cooper, W. (1981). Ubiquitous halo. Psychological Bulletin, 90, 218-244.
20. Cronbach, L.J., & Furby, L. (1970). How should we measure "change"-or should we? Psychological Bulletin, 74, 68-80.
21. DuBois, P.H. (1957). Multivariate correlational analysis. New York: Harper.
22. Dumont, R.G., & Troelstrup, R.L. (1980). Exploring relationships between objective and subjective measures of instructional outcomes. Research in Higher Education, 12, 37-51.
23. Fuertes, J., Miville, M., Mohr, J., Sedlacek, W., & Gretchen, D. (2000). Factor structure and short form of the Miville-Guzman Universality-Diversity Scale. Measurement and Evaluation in Counseling and Development, 33, 157-169.
24. Gilbert, D. (2007). Stumbling on happiness. New York: Vintage.
25. Goethals, G.R., & Reckman, R.F. (1973). The perception of consistency in attitudes. Journal of Experimental Social Psychology, 9, 491-501.
26. Gonyea, R.M. (2005). Self-reported data in institutional research: Review and recommendations. In P. D. Umbach (Ed.), New directions for institutional research (Vol. 127, pp. 73-89). San Francisco: Jossey-Bass.
27. Krosnick, J.A. (1991). Response strategies for coping with the cognitive demands of attitude measures in surveys. Applied Cognitive Psychology, 5, 213-236.
28. Kuh, G.D., Hayek, J.C., Carini, R.M., Ouimet, J.A., Gonyea, R.M., & Kennedy, J. (2001). NSSE technical and norms report. Bloomington: Indiana University, Center for Postsecondary Research and Planning.
29. Long, J.S. (1997). Regression models for categorical and limited dependent variables. Thousand Oaks, CA: Sage.
30. Lord, F.M. (1956). The measurement of growth. Educational and Psychological Measurement, 16, 421-437.
31. Markus, G.B. (1986). Stability and change in political attitudes: Observed, recalled and explained. Political Behavior, 8, 21-44.
32. McFarland, C., & Ross, M. (1987). The relation between current impressions and memories of self and dating partners. Personality and Social Psychology Bulletin, 13, 228-238.
33. McNemar, Q. (1958). On growth measurement. Educational and Psychological Measurement, 18, 47-55.
34. National Center for Public Policy and Higher Education. (2004). Measuring up 2004: The state-by-state report card for higher education. San Jose, CA: Author.
35. Miville, M., Gelso, C., Pannu, R., Liu, W., Touradji, P., Holloway, P.,et al.(1999). Appreciating similarities and valuing differences: The Miville-Guzman Universality-Diversity Scale. Journal of Counseling Psychology, 46, 291-307.
36. Nisbett, R.E., & Wilson, T.D. (1977). Telling more than we know: Verbal reports on mental processes. Psychological Review, 84, 231-259.
37. Nunnally, J.C. (1982). The study of human change: Measurement, research strategies, and methods of analysis. In B. B. Wolman (Ed.), Handbook of developmental psychology (pp. 133-148). Englewood Cliffs, NJ: Prentice Hall.
38. Pace, C. (1984). Measuring the quality of college student experiences: An account of the development and use of the College Student Experiences Questionnaire. Los Angeles: Higher Education Research Institute.
39. Pace, C., & Friedlander, J. (1982). The meaning of response categories: How often is occasionally, often, and very often? Research in Higher Education, 17, 267-281.
40. Pascarella, E.T. (2001). Using student self-reported gains to estimate college impact: A cautionary tale. Journal of College Student Development, 42, 488-492.
41. Pascarella, E., Bohr, L., Nora, A., & Terenzini, P. (1995). Cognitive effects of 2-year and 4-year colleges: New evidence. Educational Evaluation and Policy Analysis, 17, 83-96.
42. Pascarella, E., Edison, M., Nora, A., Hagedorn, L., & Terenzini, P. (1996). Influences on students' openness to diversity and challenge in the first year of college. Journal of Higher Education, 67, 174-195.
43. Pascarella, E.T., & Terenzini, P.T. (2005). How college affects students: Vol. 2. A third decade of research. San Francisco: Jossey-Bass.
44. Pike, G.R. (1993). The relationship between perceived learning and satisfaction with college: An alternative view. Research in Higher Education, 34, 23-40.
45. Pike, G.R. (1994, November). The relationship between self-report and objective measures of student achievement. Paper presented at the annual meeting of the Association for the Study of Higher Education, Tucson, AZ.
46. Pike, G.R. (1995). The relationship between self-reports of college experiences and achievement test scores. Research in Higher Education, 36, 1-21.
47. Pike, G.R. (1996). Limitations of using students' self-reports of academic development as proxies for traditional achievement measures. Research in Higher Education, 37, 89-114.
48. Pike, G.R. (1999). The constant error of the halo in educational outcomes research. Research in Higher Education, 40, 61-86.
49. Pohlmann, J., & Beggs, D. (1974). A study of the validity of self-reported measures of academic growth. Journal of Educational Measurement, 11, 115-119.
50. Pronin, E., & Kugler, M.B. (2007). Valuing thoughts, ignoring behavior: The introspection illusion as a source of the blind spot bias. Journal of Experimental Social Psychology, 43, 565-578.
51. Raudenbush, S.W., & Bryk, A.S. (2002). Hierarchical linear models: Applications and data analysis methods (2nd ed.). Newbury Park, CA: Sage.
52. Reiter, H.L. (1980). The perils of partisan recall. Public Opinion Quarterly, 44, 385-388.
53. Rest, J., Narvaez, D., Thoma, S., & Bebeau, M. (1999). DIT2: Devising and testing a revised instrument of moral judgment. Journal of Educational Psychology, 91, 644-659.
54. Rogosa, D.R., Brandt, D., & Zimowski, M. (1982). A growth curve approach to the measurement of change. Psychological Bulletin, 90, 726-748.
55. Rogosa, D.R., & Willett, J.B. (1983). Demonstrating the reliability of the difference score in the measurement of change. Journal of Educational Measurement, 20, 335-343.
56. Ross, M. (1989). Relation of implicit theories to the construction of personal histories. Psychological Review, 96, 341-357.
57. Ryff, C.D. (1989). Happiness is everything, or is it? Explorations on the meaning of psychological well-being. Journal of Personality and Social Psychology, 57, 1069-1081.
58. Singer, J.D., & Willett, J.B. (2003). Applied longitudinal data analysis: Modeling change and event occurrence. New York: Oxford University Press.
59. Terenzini, P.T., Pascarella, E.T., & Lorang, W.G. (1982). An assessment of the academic and social influences on freshman year educational outcomes. Review of Higher Education, 5, 86-109.
60. Tourangeau, R., Rips, L., & Rasinski, K. (2000). The psychology of survey response. New York: Cambridge University Press.
61. Truxello, D.M., Seitz, R., & Bauer, T.N. (2008). The role of cognitive ability in self-efficacy and self-assessed test performance. Journal of Applied Social Psychology, 38, 903-918.
62. Tucker, L.R., Damarin, F., & Messick, S.A. (1966). A base-free measure of change. Psychometrika, 31, 457-473.
63. Tyree, T. (1998). Designing an instrument to measure socially responsible leadership using the social change model of leadership development. Unpublished doctoral dissertation, University of Maryland- College Park.
64. Umbach, P.D., & Kuh, G.D. (2006). Student experiences with diversity at liberal arts colleges: Another claim for distinctiveness. Journal of Higher Education, 77, 169-192.
65. University of Minnesota, Center for the Study of Ethical Development. (n.d.). New index (N2). Retrieved January 14, 2008, from http://www.centerforthestudyofethicaldevelopment.net/New%20Index.htm.
66. Upcraft, M. L., Gardner, J. N., & Barefoot, B. O. (Eds.). (2004). Challenging and supporting the first-year student. San Francisco: Jossey-Bass.
67. U.S. Department of Education. (2006). A test of leadership: Charting the future of U.S. higher education. Washington, DC: Author.
68. Vontress, C.E. (1986). Social and cultural foundations. In M. D. Lewis, R. Hayes, & J. A. Lewis (Eds.), An introduction to the counseling profession (pp. 215-250). Itasca, IL: Peacock.
69. Vontress, C.E. (1988). An existential approach to cross-cultural counseling. Journal of Multicultural Counseling and Development, 16, 78-83.
70. Werts, C.E., & Linn, R.L. (1970). A general linear model for studying growth. Psychological Bulletin, 73, 17-22.
71. Whitt, E.J., Edison, M., Pascarella, E.T., Nora, A., & Terenzini, P.T. (1999). Interactions with peers and objective and self-reported cognitive outcomes across 3 years of college. Journal of College Student Development, 40, 61-78.
72. Wilson, T.D. (2002). Strangers to ourselves. Cambridge, MA: Belknap Press of Harvard University Press.
73. Yalom, I.D. (1985). The theory and practice of group psychotherapy (2nd ed.). New York: Basic Books.
74. Zimmerman, D.W. (2009). The reliability of difference scores in populations and samples. Journal of Educational Measurement, 46, 19-42.