Outcomes Assessment in Higher Education: Challenges and Future Research in the Context of Voluntary System of Accountability

Educational Measurement: Issues and Practice

Volumn 30, Issue 3, 2011, Pages 2-9

  Liu, Ou Lydia ^a  

^a EDUCATIONAL TESTING SERVICE   (United States)

Author keywords

Accountability; Higher education; Outcomes assessment; Voluntary system of accountability

Indexed keywords

EID: 80053330239     PISSN: 07311745     EISSN: 17453992     Source Type: Journal    
DOI: 10.1111/j.1745-3992.2011.00206.x     Document Type: Article

References (43)

1. ACT (2008). 2008 Retention/Completion Summary Tables. Iowa City, IA ACT.
2. ACT (2009). CAAP guide to successful general education outcomes assessment. Iowa City, IA ACT.
3. Banta, T. (2008). Trying to clothe the emperor. Assessment Update, 20(2), 3-4, 16-17.
4. Borden, V. M. H., & Young, J. W. (2008). Measurement validity and accountability for student learning. In V. M. H. Borden & G. R. Pike (Eds.), Assessing and accounting for student learning: Finding a constructive path forward. San Francisco Jossey-Bass.
5. College Board (2010). Trends in college pricing. New York Author.
6. Council for Aid to Education (2007). CLA institutional report 2006-2007. New York Author.
7. Council for Higher Education Accreditation (2006). Accrediting organizations in the United States: How do they operate to assure quality? Washington, DC Council for Higher Education Accreditation.
8. Council for Higher Education Accreditation (2010). 2010-2011 directory of CHEA-recognized organizations. Washington, DC Council for Higher Education Accreditation.
9. Educational Testing Service (2007). MAPP user's guide. Princeton, NJ Author.
10. Educational Testing Service (2010). Market research of institutions that use outcomes assessment. Princeton, NJ Author.
11. Ewell, P. T. (1994). A policy guide for assessment: Making good use of the tasks in critical thinking. Princeton, NJ ETS.
12. Gorard, S. (2008). The value-added of primary schools: What is it really measuring? Educational Review, 60(2), 179-185.
13. Hendel, D. D. (1991). Evidence of convergent and discriminant validity in three measures of college outcomes. Educational and Psychological Measurement, 51(2), 351-358.
14. Klein, S. (1996). The costs and benefits of performance testing on the bar examination. The Bar Examiner, 65(3), 13-20.
15. Klein, S., Benjamin, R., Shavelson, R., & Bolus, R. (2007). The collegiate learning assessment: Facts and fantasies. Evaluation Review, 31(5), 415-439.
16. Klein, S., Freedman, D., Shavelson, R., & Bolus, R. (2008). Assessing school effectiveness. Evaluation Review, 32(6), 511-525.
17. Klein, S., Kuh, G., Chun, M., Hamilton, L., & Shavelson, R. (2005). An approach to measuring cognitive outcomes across higher-education institutions. Journal of Research on Higher Education, 46(3), 251-276.
18. Klein, S., Liu, O.L., Sconing, J., Bolus, R., Bridgeman, B., Kugelmass, H., Nemeth, A., Robbins, S., & Steedle, J. (September 29, 2009). Test Validity Study (TVS) Report. Supported by the Fund for Improvement of Postsecondary Education (FIPSE)
19. Lee, V. E. (2000). Using hierarchical linear modeling to study social contexts: The case of school effects. Educational Psychologist, 35(2), 125-141.
20. Liu, O. L. (2008). Measuring learning outcomes in higher education using the Measure of Academic Proficiency and Progress (MAPP™) (ETS Research Report No. RR-08-047). Princeton, NJ ETS.
21. Liu, O. L. (2009a). R&D connections: Measuring learning outcomes in higher education (Report No. RDC-10). Princeton, NJ ETS.
22. Liu, O. L. (2009b). Measuring value-added in higher education: Conditions and caveats. Results from using the Measure of Academic Proficiency and Progress (MAPP™). Assessment and Evaluation in Higher Education, 34(6), 1-14.
23. Liu, O. L. (2011). Value-added assessment in higher education: A comparison of two methods. Higher Education, 61(4), 445-461.
24. Lockwood, J. R., McCaffrey, D. F., & Hamilton, L. S. (2007). The sensitivity of value-added teacher effect estimates to different mathematics achievement measures. Journal of Educational Measurement, 44(1), 47-67.
25. Marr, D. (1995). Validity of the academic profile. Princeton, NJ Educational Testing Service.
26. National Institute for Learning Outcomes Assessment (NILOA) (2009). More than you think, less than we need: Learning outcomes assessment in American higher education. Urbana, IL NILOA.
27. Palardy, G. J. (2008). Differential school effects among low, middle, and high social class composition schools: A multilevel, multiple group latent growth curve analysis. School Effectiveness and School Improvement, 19, 21-49.
28. Rabe-Hesketh, S., & Skrondal, A. (2008). Multilevel and longitudinal modeling using Stata (2nd ed.). College Station, TX Stata Press.
29. Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models: Applications and data analysis methods (2nd ed.). Thousand Oaks, CA Sage.
30. Singer, J. (1998). Using SAS PROC MIXED to fit multilevel models, hierarchical models, and individual growth models. Journal of Education and Behavioral Statistics, 24(4), 323-355.
31. Stone, C. (1995). A course in probability and statistics. Belmont, CA Duxbury Press.
32. Sundre, D. L. (1997, April). Differential examinee motivation and validity: A dangerous combination. Paper presented at the meeting of the American Educational Research Association, Chicago, IL.
33. Sundre, D. L. (1999, April). Does examinee motivation moderate the relationship between test consequences and test performance? Paper presented at the meeting of the American Educational Research Association, Montreal, Canada (ERIC Document Reproduction Service No. ED432588).
34. Sundre, D. L., & Wise, S. L. (2003, April). Motivation filtering: An exploration of the impact of low examinee motivation on the psychometric quality of tests. Paper presented at the annual meeting of the National Council on Measurement in Education, Chicago, IL.
35. The Conference Board, Inc., the Partnership for 21st Century Skills, Corporate Voices for Working Families, and the Society for Human Resource Management (2006). Are they really ready to work? New York The Conference Board (, accessed May 5, 2009).
36. U.S. Department of Education, National Center for Education Statistics (2006). Digest of Education Statistics, 2005 (NCES 2006-030).
37. U.S. Department of Education, National Center for Education Statistics. (2008). Digest of Education Statistics, 2007 (NCES 2008-022).
38. Voluntary System of Accountability (2008). Background on learning outcomes measures (, accessed May 18, 2009).
39. Wainer, H. (1993). Measurement problems. Journal of Educational Measurement, 30, 1-21.
40. Wise, S. L., & DeMars, C. D. (2005). Low examinee effort in low-stakes assessment: Problems and potential solutions. Educational Assessment, 10(1), 1-17.
41. Wise, S. L., & Kong, X. (2005). Response time effort: A new measure of examinee motivation in computer-based tests. Applied Measurement in Education, 18(2), 163-183.
42. Wise, V. L., Wise, S. L., & Bhola, D. S. (2006). The generalizability of motivation filtering in improving test score validity. Educational Assessment, 11(1), 65-83.
43. Yeh, S. S., & Ritter, J. (2009). The cost-effectiveness of replacing the bottom quartile of novice teachers through value-added teacher assessment. Journal of Education Finance, 34(4), 426-451.