Do adjusted subscores lack validity? don't blame the messenger

Educational and Psychological Measurement

Volumn 71, Issue 5, 2011, Pages 789-797

  Sinharay, Sandip ^a   Haberman, Shelby J ^a   Wainer, Howard ^b  

^a EDUCATIONAL TESTING SERVICE   (United States)

^b NATIONAL BOARD OF MEDICAL EXAMINERS   (United States)

Author keywords

augmented subscore; subscores; validity

Indexed keywords

EID: 80053204392     PISSN: 00131644     EISSN: 15523888     Source Type: Journal    
DOI: 10.1177/0013164410391782     Document Type: Article

References (20)

1. American Educational Research Association, American Psychological Association, & National Council on Measurement in Education Standards for educational and psychological testing. Washington, DC: American Educational Research Association; 1999.
2. de la Torre J.,Patz R. J.Making the most of what we have: A practical application of multidimensional IRT in test scoring Journal of Educational and Behavioral Statistics. 2005:295-311.
3. Dwyer A.,Boughton K. A.,Yao L.,Steffen M.,Lewis D.A comparison of subscale score augmentation methods using empirical data. Paper presented at the annual meeting of the National Council on Measurement in Education; 2006San Francisco, CA; 2006.
4. Haberman S. J.When can subscores have value? Journal of Educational and Behavioral Statistics. 2008a:204-229.
5. Haberman S. J.Subscores and validity. Princeton, NJ: Educational Testing Services; 2008b:.
6. Haberman S. J.,Sinharay S.Reporting of subscores using multidimensional item response theory.Psychometrika. 2010;75:209-227.
7. Kane, M. T. (2006). Validation. In R. L. Brennan (Ed.), Educational measurement (4th ed., pp. 18-64). Westport, CT: Praeger.
8. Luecht R. M.Applications of multidimensional diagnostic scoring for certification and licensure tests. Paper presented at the meeting of the National Council on Measurement in Education; 2003Chicago, IL; 2003.
9. Lyren P.Reporting subscores from college admission tests.Practical Assessment, Research, and Evaluation. 2009;14:1-10.
10. Messick, S. (1989). Validity. In R. L. Linn (Ed.) Educational measurement (3rd ed., pp. 13-103). Washington, DC: National Council on Measurement in Education and American Council on Education.
11. Knowing what students know: The science and design of educational assessment. Washington, DC: National Academies Press; 2001:.
12. Puhan G.,Sinharay S.,Haberman S. J.,Larkin K.Comparison of subscores based on classical test theory. Applied Measurement in Education. 2010:1-20.
13. Reckase M. D.The past and future of multidimensional item response theory.Applied Psychological Measurement. 1997;21:25-36.
14. Sinharay S.How often do subscores have added value? Results from operational and simulated data Journal of Educational Measurement. 2010:150-174.
15. Skorupski W. P.,Carvajal J.A comparison of approaches for improving the reliability of objective level scores.Educational and Psychological Measurement. 2010;70:357-375.
16. Stone C. A.,Ye F.,Zhu X.,Lane S.Providing subscale scores for diagnostic information: A case study when the test is essentially unidimensional Applied Measurement in Education. 2010:63-86.
17. Wainer H.,Sheehan K.,Wang X.Some paths toward making praxis scores more useful Journal of Educational Measurement. 2000:113-140.
18. Wainer, H., Vevea, J. L., Camacho, F., Reeve, B. B., Rosa, K., Nelson, L., Swygert, K. A.,Thissen, D. (2001). Augmented scores-"Borrowing strength" to compute scores based on small numbers of items. In D. Thissen & H. Wainer (Eds.), Test scoring (pp. 343-387). Mahwah, NJ: Lawrence Erlbaum.
19. Yao L.,Boughton K. A.A multidimensional item response modeling approach for improving subscale proficiency estimation and classification.Applied Psychological Measurement. 2007;31:83-105.
20. Yen W. M.A Bayesian/IRT index of objective performance. Paper presented at the annual meeting of the Psychometric Society; 1987Montreal, Quebec, Canada; 1987.