Building and supporting a validity argument for a standards-based classroom assessment of English proficiency based on teacher judgments

Educational Measurement: Issues and Practice

Volumn 27, Issue 3, 2008, Pages 32-42

  Llosa, Lorena ^a  

^a NEW YORK UNIVERSITY   (United States)

Author keywords

English learners; Language proficiency; Teacher judgments; Validity

Indexed keywords

EID: 50849111054     PISSN: 07311745     EISSN: 17453992     Source Type: Journal    
DOI: 10.1111/j.1745-3992.2008.00126.x     Document Type: Article

References (43)

1. Arkoudis, S., & O'Loughlin, K. (2004). Tensions between validity and outcomes: Teacher assessment of written work of recently arrived immigrant ESL students. Language Testing, 21, 284 304.
2. Bachman, L. F. (2005). Building and supporting a case for test use. Language Assessment Quarterly, 2 (1 1 34.
3. Bachman, L. F. (2006). Linking validity and use in educational assessment. Paper presented at the National Council of Measurement in Education Annual Meeting in San Francisco, CA.
4. Beswick, J. F., Douglas Willms, J., & Sloat, E. A. (2005). A comparative study of teacher ratings of emergent literacy skills and student performance on a standardized measure. Education, 126 (1 116 137.
5. Breen, M. P., Barrat-Pugh, C., Derewianka, B., House, H., Hudson, C., Lumley, T., & Rohl, M. (1997). Profiling ESL Children: How teachers interpret and use national and state assessment frameworks. Canberra : Department of Employment, Education, Training, and Youth Affairs, Commonwealth of Australia.
6. Brindley, G. (1998). Outcomes-based assessment and reporting in language learning programmes: A review of the issues. Language Testing, 15 (1 45 85.
7. Brindley, G. (2001). Outcomes-based assessment in practice: Some examples and emerging insights. Language Testing, 18 (4 393 407.
8. Brookhart, S. M. (1991). Grading practices and validity. Educational Measurement: Issues and Practice, 10 (1 35 36.
9. Brookhart, S. M. (1994). Teachers' grading: Practice and theory. Applied Measurement in Education, 7, 279 301.
10. Cizek, G. J., Fitzgerald, S. M., & Rachor, R. E. (1995). Teachers' assessment practices: Preparation, isolation and the kitchen sink. Educational Assessment, 3 (2 159 179.
11. Cross, L. H., & Frary, R. B. (1999). Hodgepodge grading: Endorsed by students and teachers alike. Applied Measurement in Education, 12 (1 53 72
12. Davison, C. (2004). The contradictory culture of teacher-based assessment: ESL teacher assessment practices in Australian and Hong Kong secondary schools. Language Testing, 21, 305 334.
13. Dutro, E., & Valencia, E 2004). The relationship between state and district content standards: Issues of alignment, influence and utility. Education Policy Analysis Archives, 12 (45). Retrieved February 2007 from http://epaa.asu.edu/epaa/v12n45/.
14. Eckert, T. L., Dunn, E. K., Codding, R. S., Begeny, J. C., & Kleinmann, A. E. (2006). Assessment of mathematics and reading performance: An examination of the correspondence between direct assessment of student performance and teacher report. Psychology in the Schools, 43 (3 247 265.
15. Ferrara, S. (2004). An alternative approach to statewide assessment: The Nebraska Standards-based Teacher-led Assessment System [Special Issue]. Educational Measurement: Issues and Practice, 23, 2.
16. Gattullo, F. (2000). Formative assessment in ELT primary (elementary) classrooms: An Italian case study. Language Testing, 17, 278 288.
17. Gearhart, M., & Wolf, S. (1996). Issues in portfolio assessment: Providing evidence of writing competency (Part I: The purposes and processes of writing). Los Angeles : Center for Research on Evaluation, Standards, and Student Testing.
18. Gijsel, M. A. R., Bosman, A. M. T., & Verhoeven, L. (2006). Kindergarten risk factors, cognitive factors, and teacher judgments as predictors of early reading in Dutch. Journal of Learning Disabilities, 39 (6 558 571.
19. Gresham, F. M., MacMilan, D. L., & Bocian, K. M. (1997). Teachers as "Tests": Differential validity of teacher judgments in identifying students at-risk for learning difficulties. The School Psychology Review, 26 (1 47 60.
20. Hecht, S. A., & Greenfield, D. B. (2002). Explaining the predictive accuracy of teacher judgments of their students' reading achievement: The role of gender, classroom behavior, and emergent literacy skills in a longitudinal sample of children exposed to poverty. Reading and Writing: An Interdisciplinary Journal, 15, 789 809.
21. Hoge, R., & Coldarci, T. (1989). Teacher-based judgments of academic achievement: A review of literature. Review of Educational Research, 59, 297 313.
22. Kane, M. (1992). An argument-based approach to validity. Psychological Bulletin, 112 (3 527 535.
23. Kane, M. (2001). Current concerns in validity theory. Journal of Educational Measurement, 38, 319 342.
24. Kane, M. (2002). Validating high-stakes testing programs. Educational Measurement: Issues and Practice, 21 (1 31 41.
25. Kane, M. (2004). Certification testing as an illustration of argument-based validation. Measurement: Interdisciplinary Research and Perspectives, 2, 135 170.
26. Kane, M. (2006). Validation. In R. Brennan (Ed Educational measurement (4th ed.). Wesport, CT : American Council on Education and Praeger Publishers.
27. Kane, M., Crooks, T., & Cohen, A. (1999). Validating measures of performance. Educational Measurement: Issues and Practice, 18 (2 5 17.
28. Koretz, D. (1998). Large-scale portfolio assessments in the US: Evidence pertaining to the quality of measurement. Assessment in Education, 5 (3 309 334.
29. Koretz, D., Stecher, B., Klein, S., & McCaffrey, D. (1994). The Vermont portfolio assessment program: Findings and implications. Educational Measurement: Issues and Practice, 13, 5 16.
30. Koretz, D., Stecher, B., & Deibert, E. (1992). The reliability of scores from the 1992 Vermont Portfolio Assessment Program. CSE Technical Report 355. Los Angeles : Center for the Study of Evaluation.
31. Leung, C., & Mohan, B. (2004). Teacher formative assessment and talk in classroom contexts: Assessments as discourse and assessment of discourse. Language Testing, 21, 335 359.
32. Llosa, L. (2005). Assessing English learners' language proficiency: A qualitative investigation of teachers' interpretations of the California ELD Standards. The CATESOL Journal, 17 (1 7 18.
33. Llosa, L. (2007). Validating a standards-based classroom assessment of English proficiency: A multitrait-multimethod approach. Language Testing, 24, 4.
34. Meisels, S. J., Bickel, D. D., Nicholson, J., Xue, Y., & Atkins-Burnett, S. (2001). Trusting teacher judgments: A validity study of a curriculum-embedded performance assessment in kindergarten to grade 3 American Educational Research Journal, 38 (1 73 95.
35. Mislevy, R. J., Steinberg, L. S., & Almond, R. G. (2003). On the structure of educational assessments. Measurement: Interdisciplinary Research and Perspectives, 1 (1 3 62.
36. Perry, N. E., & Meisels, S. J. (1996). How accurate are teacher judgments of students' academic performance? Working Paper (No. 96-08), National Center for Education Statistics. Washington, DC : U.S. Department of Education, OERI.
37. Rea-Dickins, P. (2001). Mirror, mirror on the wall: Identifying processes of classroom assessment. Language Testing, 18, 429 462.
38. Rea-Dickins, P., & Gardner, S. (2000). Snares and silver bullets: Disentangling the construct of formative assessment. Language Testing, 17, 215 243.
39. Stecher, B. M. (1998). The local benefits and burdens of large-scale portfolio assessment. Assessment in Education, 5 (3 335 351.
40. Taylor, H. G., Anselmo, M., Foreman, A. L., Schatschneider, C., & Angelopoulos, J. (2000). Utility of kindergarten teacher judgments in identifying early learning problems. Journal of Learning Disabilities, 33 (2 200 210.
41. Toulmin, S. E. (2003). The uses of argument (updated ed.). Cambridge, U.K. : Cambridge University Press.
42. Widaman, K. F. (1985). Hierarchically nested covariance structure models for multitrait-multimethod data. Applied Psychological Measurement, 9 (1 1 26.
43. Zehler, A., Fleischman, H., Hopstock, P., Stephenson, T., Pendzick, M., & Sapru, S. (2003). Descriptive study of services to limited English proficient students. Washington, DC : Development Associates.