Psychometric properties of three new National Survey of Student Engagement based engagement scales: An item response theory analysis

Research in Higher Education

Volumn 50, Issue 8, 2009, Pages 775-794

  Carle, Adam C ^a   Jaffee, David ^a   Vaughan, Neil W ^a   Eder, Douglas ^a  

^a UNIVERSITY OF NORTH FLORIDA   (United States)

Author keywords

Community learning; Item response theory; Measurement; Student engagement; Student faculty interaction; Test reliability; Transformational learning

Indexed keywords

EID: 76049107589     PISSN: 03610365     EISSN: None     Source Type: Journal    
DOI: 10.1007/s11162-009-9141-z     Document Type: Article

References (43)

1. Astin, A. W. (1993). What matters in college: Four critical years revisited. San Francisco: Jossey-Bass.
2. Astin, A. W., & Sax, L. J. (1998). How undergraduates are affected by service participation. The Journal of College Student Development, 39(3), 251-263.
3. Astin, A. W., Vogelgesang, L. J., Ikeda, E. K., & Yee, J. A. (2000). How service learning affects students. Los Angeles: Higher Education Research Institute University of California.
4. Benjamini, Y., & Hochberg, Y. (1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. Journal of the Royal Statistical Society, Series B, 57, 289-300.
5. Bjorner, J. B., Smith, K. J., Stone, C., & Sun, X. (2007). IRTFIT: A macro for item fit and local dependence tests under IRT models. USA: QualityMetric Incorporated School of Education, University of Pittsburgh.
6. Carini, R. M., Kuh, G. D., & Klein, S. P. (2006). Student engagement and student learning: Testing the linkages. Research in Higher Education, 47, 1-32.
7. Chickering, A. W., & Gamson, Z. F. (1987). Seven principles for good practice in undergraduate education. AAHE Bulletin, 39, 3-7.
8. Crane, P. K., Gibbons, L. E., Jolley, L., & van Belle, G. (2006). Differential item functioning analysis with ordinal logistic regression techniques: DIFdetect and difwithpar. Medical Care, 44, S115-S123.
9. Cronbach, L. J., & Meehl, P. E. (1955). Construct validity in psychological tests. Psychological Bulletin, 52(4), 281-302.
10. Embretson, S., & Reise, S. P. (2000). Item response theory for psychologists. NJ: Lawrence.
11. Gonyea, R. M., Kinzie, J., Kuh, G. D., & Nelson Laird, T. F. (2008). High impact activities: What they are, why they work, and who benefits. Program presented at the American Association for Colleges and Universities annual meeting. Washington, DC.
12. Gordon, J., Ludlum, J., & Hoey, J. (2008). Validating NSSE against student outcomes: Are they related? Research in Higher Education, 49, 19-39.
13. Green, S. B., & Babyak, M. A. (1997). Control of type I errors with multiple tests of constraints in structural equation modeling. Multivariate Behavioral Research, 32(1), 39-51.
14. Hambleton, R. K., & Swaminathan, H. (1985). Item response theory. Boston, MA: Kluwer.
15. Hopkins, K. D. (1997). Educational and psychological measurement and evaluation. New York: Allyn & Bacon.
16. Hu, L., & Bentler, P. M. (1998). Fit indices in covariance structure modeling: Sensitivity to underparameterized model misspecification. Psychological Methods, 3(4), 424-453.
17. Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6(1), 1-55.
18. Jones, R. (2006). Identification of measurement differences between English and Spanish language versions of the mini-mental state examination: Detecting differential item functioning using MIMIC modeling. Medical Care, 44, S124-S133.
19. Kinzie, J., Evenbeck, S. (2008). Assessing Student Engagement in High-Impact Practices. Program presented at the NASPA IARC Conference Scottsdale, AZ.
20. Klein, S. P., Kuh, G. D., Chun, M., Hamilton, L., & Shavelson, R. (2005). An approach to measuring cognitive outcomes across higher education institutions. Research in Higher Education, 46(3), 251-276.
21. Kuh, G. D. (2001). The national survey of student engagement: Conceptual framework and overview of psychometric properties. Bloomington, IN: Indiana University, Center for Postsecondary Research.
22. Kuh, G. D. (2003). What we're learning about student engagement from NSSE. Change, 35, 24-32.
23. Kuh, G. D. (2005). 7 steps for taking student learning seriously. Trusteeship, 13, 20-24.
24. Kuh, G. D., Hayek, J. C., Carini, R. M., Ouimet, J. A., Gonyea, R. M., & Kennedy, J. (2001). NSSE technical and norms report. Bloomington, IN: Indiana University Center for Postsecondary Research and Planning.
25. Kuh, G. D., Kinzie, J., Schuh, J. H., Whitt, E. J., & Associates. (2005). Student success in college: Creating conditions that matter. San Francisco: Jossey-Bass.
26. McInnis, E. D. (2006). Nonresponse bias in student assessment surveys: A comparison of respondents and non-respondents of the national survey of student engagement at an independent comprehensive Catholic University. Unpublished doctoral dissertation, Marywood University, USA.
27. Muthén, L. K., & Muthén, B. O. (1998-2007). Mplus user's guide. (4ed.) Los Angeles, CA: Muthén & Muthén.
28. National Survey of Student Engagement (NSSE). (2006). NSSE 2006 codebook. Retrieved November 22, 2009, from http://www.nsse.iub.edu/pdf/2006_Institutional_Report/nsse_codebooks/NSSE%202006%20Codebook.pdf.
29. National Survey of Student Engagement (NSSE). (2008). Origins. Retrieved November 22, 2009, from http://nsse.iub.edu/html/origins.cfm.
30. Orlando, M., & Thissen, D. (2000). Likelihood-based item-fit indices for dichotomous item response theory models. Applied Psychological Measurement, 24(1), 50-64.
31. 2: An item fit index for use with dichotomous item response theory models. Applied Psychological Measurement, 27(4), 289-298.
32. Pace, C. R. (1984). Measuring the quality of college student experiences. Los Angeles: Los Angeles Center for the Study of Evaluation, University of California Los Angeles.
33. Pascarella, E. T., & Terenzini, P. T. (1991). How college affects students. San Francisco: Jossey-Bass.
34. Pedhazur, E. J., & Schmelkin, L. (1991). Measurement, design, and analysis: An integrated approach. Hillsdale, NJ: Lawrence Erlbaum Associates, Inc.
35. Pike, G. R. (2006a). The convergent and discriminant validity of NSSE Scalelet scores. Journal of College Student Development, 47, 551-564.
36. Pike, G. R. (2006b). The dependability of NSSE Scalelets for college- and department-level assessment. Research in Higher Education, 47, 177-195.
37. Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores. Psychometrika Monograph Supplement, 34(4, pt. 2), 100.
38. Stone, C. A. (2000). Monte carlo based null distribution for an alternative goodness-of-fit test statistic in IRT models. Journal of Educational Measurement, 37(1), 58-75.
39. Stone, C. A. (2003). Empirical power and type I error rates for an IRT fit statistic that considers the precision of ability estimates. Educational and Psychological Measurement, 63(4), 566-583.
40. Stone, C. A., & Zhang, B. (2003). Assessing goodness of fit of item response theory models: A comparison of traditional and alternative procedures. Journal of Educational Measurement, 40(4), 331-352.
41. Thissen, D., Chen, W., & Bock, D. (2002). MULTILOG 7. Chicago: Scientific software international.
42. Thissen, D., Steinberg, L., & Wainer, H. (1993). Differential item functioning. In P. W. Holland (Ed.), Detection of differential item functioning using the parameters of item response models (pp. 67-113). Hillsdale, NJ: Lawrence Erlbaum Associates.
43. Thorndike, R. M. (2004). Measurement and evaluation in psychology and education (7th ed.). Columbus, OH: Merrill Publishing Co/Prentice-Hall.