Validity issues in test speededness

Educational Measurement: Issues and Practice

Volumn 26, Issue 4, 2007, Pages 29-37

  Lu, Ying ^a,b   Sireci, Stephen G ^b  

^a EDUCATIONAL TESTING SERVICE   (United States)

^b UNIVERSITY OF MASSACHUSETTS   (United States)

Author keywords

Construct validity; Construct irrelevant variance; Item response time; Speededness; Standardized testing

Indexed keywords

EID: 36148954370     PISSN: 07311745     EISSN: 17453992     Source Type: Journal    
DOI: 10.1111/j.1745-3992.2007.00106.x     Document Type: Article

References (36)

1. American Educational Research Association, American Psychological Association, & National Council on Measurement in Education (1999). Standards for educational and psychological testing. Washington, DC : Author.
2. Bejar, I. I. (1985). Test speededness under number-right scoring: An analysis of the Test of English as a Foreign Language (Report No. ETS-RR-85-11). Princeton, NJ : Educational Testing Service. (ERIC Document Reproduction Service No. ED 263 128)
3. Cohen, A. S., Wollack, J. A., Bolt, D. M., & Mroch, A. A. (2002, April). A mixture Rasch model analysis of test speededness. Paper presented at the Annual Meeting of American Educational Research Association, New Orleans, LA.
4. College Entrance Examination Board (1984). The college board technical handbook for the scholastic aptitude test and achievement tests. New York : Author.
5. Cronbach, L. J., & Warrington, W. G. (1951). Time limit tests: Estimating their reliability and degree of speeding. Psychometrika, 14, 167 188.
6. Cook, L. L., & Douglass, J. B. (1982). Analysis of fit and vertical equating with the three-parameter model. Paper presented at the Annual meeting of American Educational Research Association, New York.
7. Dorans, N. J., Schmitt, A. P., & Bleistein, C. A. (1992). The standardization approach to assessing comprehensive differential item functioning. Journal of Educational Measurement, 29, 309 319.
8. Evans, F. R., & Reilly, R. R. (1972). A study of speededness as a source of test bias. Journal of Educational Measurement, 37, 123 131.
9. Gulliksen, H. (1950). Theory of mental tests. New York : John Wiley.
10. Hambleton, R. K., Swaminathan, H., & Rogers, H. J. (1991). Fundamentals of item response theory. Newbury Park, CA : Sage.
11. Lawrence, I. M. (1993). The effect of test speededness on subgroup performance (Report No. ETS-RR-93-49). Princeton, NJ : Educational Testing Service.
12. Lord, F. M. (1980). Applications of item response theory to practical testing problems. Hillsdale, NJ : Lawrence Erlbaum.
13. Morrison, E. J. (1960). On test variance and the dimensions of the measurement situation. Educational and Psychological Measurement, 20, 231 250.
14. Oppler, S. H., Davies, S. A., Lyons, B. D., Nathanson, L. B., & Chen, W. (2004, April). The effect of speededness on MCAT scores: An initial examination. Paper presented at the Annual Conference of the Society for Industrial and Organizational Psychology, Chicago, IL.
15. Oshima, T. C. (1994). The effect of speededness on parameter estimation in item response theory. Journal of Educational Measurement, 31 (3 200 219.
16. Rindler, S. E. (1979). Pitfalls in assessing test speededness. Journal of Educational Measurement, 16 (4 261 270.
17. Roskam, E. E. (1997). Models for speed and time-limit tests. In W. J. van der Linden & R. K. Hambleton (Eds. Handbook of modern item response theory (pp. 187 208). New York : Springer.
18. Rost, J. (1990) Rasch models in latent classes: An integration of two approaches to item analysis. Applied Psychological Measurement, 14, 271 282.
19. Schnipke, D. L. (1995, April). Assessing speededness in computer-based tests using item response times. Paper presented at the Annual Meeting of the National Council on Measurement in Education, San Francisco, CA.
20. Schnipke, D. L., & Scrams, D. J. (1997). Modeling item response times with a two-state mixture model: A new method of measuring speededness. Journal of Educational Measurement, 34 (3 213 232.
21. Sireci, S. G. (2005). Unlabeling the disabled: A perspective on flagging scores from accommodated test administrations. Educational Researcher, 34 (1 3 12.
22. Sireci, S. G., Scarpati, S., & Li, S. (2005). Test accommodations for students with disabilities: An analysis of the interaction hypothesis. Review of Educational Research, 75, 457 490.
23. Stafford, R. E. (1971). The speed quotient: A new descriptive statistic for tests. Journal of Educational Measurement, 8, 275 278.
24. Swineford, F. (1974). The test analysis manual (SR-74-06). Princeton, NJ : Educational Testing Service.
25. Thissen, D. (1983). Timed testing: An approach using item response theory. In D. J. Weiss (Ed. New horizons in testing: Latent trait test theory and computerized adaptive testing. New York : Academic Press.
26. Thompson, S., & Thurlow, M. (2002, June). Universally designed assessments: Better tests for everyone¡ Policy Directions, Number 14. Minneapolis, MND : National Center on Educational Outcomes.
27. Thompson, S., Thurlow, M., & Malouf, D. B. (2004). Creating better tests for everyone through universally designed assessments. Journal of Applied Testing Technology, 6 (1). Available at http://www.testpublishers.org/jattart. htm.
28. van der Linden, W. J., Scrams, D. J., & Schnipke, D. L. (1999). Using response-time constraints to control for differential speededness in computerized adaptive testing. Applied Psychological Measurement, 23 (3 195 210.
29. Verhelst, N. D., Verstralen, H. H. F. M., & Jansen, M. G. H. (1997). A logistic model for time-limit tests. In W. J. van der Linden & R. K. Hambleton (Eds. Handbook of modern item response theory (pp. 169 185). New York : Springer.
30. Wang, T., & Hanson, B. A. (2005). Development and calibration of an item response model that incorporates response time. Applied Psychological Measurement, 29 (5 323 339.
31. Wise, L. L. (1986, April). Latent trait models for partially speeded tests. Paper presented at the Annual Meeting of the American Educational Research Association, San Francisco.
32. Wise, S. L., & DeMars C. E. (2006). An application of item response time: The effort-moderated IRT model. Journal of Educational Measurement, 43 (1 19 38.
33. Yamamoto, K. (1990). HYBIL: A computer program to estimate HYBRID model parameters. Princeton, NJ : Educational Testing Service.
34. Yamamoto, K. (1995). Estimating the effects of test length and test time on parameter estimation using the HYBRID model (TOEFL Tech. Rep. No. TR-10). Princeton, NJ : Educational Testing Service.
35. Yen, W. M. (1981). Using simulation results to choose a latent trait model. Applied Psychological Measurement, 5, 245 262.
36. Zenisky, A. L., Hambleton, R. K., & Sireci, S. G. (2002). Identification and evaluation of local item dependencies in the Medical College Admissions Test. Journal of Educational Measurement, 39, 291 309.