Psychometric properties of scale scores and performance levels for performance assessments using polytomous IRT

Journal of Educational Measurement

Volumn 37, Issue 2, 2000, Pages 141-162

  Wang, Tianyou ^a   Kolen, Michael J ^b   Harris, Deborah J ^a  

^a American College Testing Incorporated   (United States)

^b UNIVERSITY OF IOWA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 0034195157     PISSN: 00220655     EISSN: None     Source Type: Journal    
DOI: 10.1111/j.1745-3984.2000.tb01080.x     Document Type: Article

References (28)

1. ACT (1995). Work Keys Assessments. Iowa City: ACT.
2. ACT (1997). Work Keys Test Descriptions. Iowa City: ACT.
3. Andrich, D. (1978). A rating formulation for ordered response categories. Psychometrika, 43, 561-573.
4. AERA, APA, & NCME (1985). Standards for educational and psychological testing. Washington, DC: American Psychological Association.
5. Birnbaum, A. (1968). Test scores, sufficient statistics, and the information structures of tests. In F. M. Lord & M. R. Novick, Statistical theories of mental test scores. Reading, Mass.: Addison-Wesley.
6. Bock, R. D. (1972). Estimating item parameters and latent ability when responses are scored in two or more nominal categories. Psychometrika, 37, 29-51.
7. Colton, D. A., Gao, X., & Kolen, M. J. (1997). Assessing the reliability of performance level scores using bootstrapping. ACT Research Report Series, 97-3. Iowa City, IA: ACT.
8. Feldt, L. S., & Brennan, R. L. (1989). Reliability. In R. L. Linn (Ed.), Educational Measurement (pp. 105-146). Phoenix, AZ: The Oryx Press.
9. Feldt, L. S., & Qualls, A. L. (1996). Estimation of measurement error variance at specific score levels. Journal of Educational Measurement, 33, 141-156.
10. Han, T., Kolen, M. J., & Pohlmann, J. (1997). A comparison among IRT true- and observed-score equatings and traditional equipercentile equating. Applied Measurement in Education, 10, 105-121.
11. Hanson, B. A. (1994). An extension of the Lord-Wingersky algorithm to polytomous items. Unpublished research note.
12. Hanson, B. A., & Brennan, R. L. (1990). An investigation of classification consistency indexes estimated under alternative strong true score models. Journal of Educational Measurement, 27, 345-359.
13. Huynh, H. (1976). On the reliability of decisions in domain-referenced testing. Journal of Educational Measurement, 13, 253-264.
14. Keats, J. A, & Lord, F. M. (1962). A theoretical distribution for the mental test scores. Psychometrika, 27, 59-72.
15. Kolen, M. J. (1988). Defining score scales in relation to measurement error. Journal of Educational Measurement, 25, 97-110.
16. Kolen, M. J., Hanson, B. A., & Brennan, R. L. (1992). Conditional CSEM of the scale scores for scale scores. Journal of Educational Measurement, 29, 285-307.
17. Kolen, M. J., Zeng, L., & Hanson, B. A. (1996). Conditional CSEM of the scale scores for scale scores using IRT. Journal of Educational Measurement, 33, 129-140.
18. Linacre, J. M., & Wright, B. D. (1993). FACETS. Chicago: MESA Press.
19. Lord, F. M. (1980). Applications of item response theory to practical testing problems. Hillsdale, NJ: Lawrence Erlbaum Associates, Inc., Publishers.
20. Lord, F. M., & Wingersky, M. S. (1984). Comparison of IRT true-score and equipercentile observed-score "equatings." Applied Psychological Measurement, 8, 452-461.
21. Masters, G. N. (1982). A Rasch model for partial credit scoring. Psychometrika, 47, 149-174.
22. Muraki, E. (1992). A generalized partial credit model: Application of an EM algorithm. Applied Psychological Measurement, 16, 159-176.
23. Muraki, E., & Bock, R. D. (1993). PARSCALE: IRT based test scoring and item analysis for graded open-ended exercises and performance tasks. Chicago: Scientific Software International.
24. Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores. Psychometrika monograph, No. 17.
25. Subkoviak, M. J. (1976). Estimating reliability from a single administration of a mystery test. Journal of Educational Measurement, 13, 265-276.
26. Subkoviak, M. R. (1984). Estimating the reliability of mastery-nonmastery classifications. In Berk, R. A. (Ed.). A guide to criterion-referenced test construction. Baltimore, MD: The Johns Hopkins University Press.
27. Thissen, D., Pommerich, M. Billeaud, K., & Williams, V.S.L. (1995). Item response theory for scores on tests including polytomous items with ordered responses. Applied Psychological Measurement, 19, 39-49.
28. Wang, T., Kolen, M. J., & Harris, D. J. (1996). Conditional standard errors, reliability and decision consistency of performance levels using polytomous IRT. Paper presented at the Annual Meeting of the American Educational Research Association, New York, April.