Test-based accountability: Potential benefits and pitfalls of science assessment with student diversity

Journal of Research in Science Teaching

Volumn 47, Issue 1, 2010, Pages 6-24

  Penfield, Randall D ^a   Lee, Okhee ^a  

^a UNIVERSITY OF MIAMI   (United States)

Author keywords

Accountability; Assessment; Bias; Minority; Reliability; Science; Validity

Indexed keywords

ACCOUNTABILITY; EDUCATIONAL POLICY; IN-CORE; LEARNING ENGLISH; MINORITY STUDENT; POTENTIAL BENEFITS; STUDENT POPULATIONS;

STUDENTS;

EID: 73949105210     PISSN: 00224308     EISSN: 10982736     Source Type: Journal    
DOI: 10.1002/tea.20307     Document Type: Article

References (90)

1. Abedi, J. (2004). The No Child Left Behind Act and English language learners: Assessment and accountability issues. Educational Researcher, 33, 4-14.
2. Abedi, J., Hofstetter, C.H., & Lord, C. (2004). Assessment accommodations for English language learners: Implications for policy-based empirical research. Review of Educational Research, 74, 1-28.
3. Abedi, J., & Lord, C. (2001). The language factor in mathematics tests. Applied Measurement in Education, 14, 219-234.
4. American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (1999). Standards for educational and psychological testing. Washington, DC: APA.
5. Aronson, I., & Miller, J. (2007). Competing horizons. The Science Teacher, 74(7), 64-67.
6. Bolt, S.E., & Ysseldyke, J.E. (2006). Comparing DIF across math and reading/language arts tests for students receiving a read-aloud accommodation. Applied Measurement in Education, 19, 329-355.
7. Bond, L., Moss, P.,&Carr, P. (1996). Fairness in large-scale performance assessment. InG.W. Phillips& A. Goldstein (Eds.), Technical issues in large-scale performance assessment (pp. 117-140). Washington, DC: National Center for Education Statistics.
8. Brennan R.L. (Ed.). (2006). Educational measurement (4th edn.) Westport, CT: American Council on Education and Praeger.
9. Brickhouse, N.W. (2006). Celebrating 90 years of science education: Reflections on the gold standard and ways of promoting good research. Science Education, 90, 1-7.
10. Camilli, G., & Shepard, L.A. (1994). Methods for identifying biased test items. Newbury Park, CA: Sage.
11. Champagne, A. (2006). Then and now: Science assessment 1996-2006. School Science and Mathematics, 106, 113-123.
12. Darling-Hammond, L. (1992-1993). Creating standards of practice and delivery for learner-centered schools. Stanford Law and Policy Review, 4, 37-48.
13. Darling-Hammond, L. (1994). Performance-based assessment and educational equity. Harvard Educational Review, 64, 5-30.
14. Darling-Hammond, L. (2004). Inequality and the right to learn: Access to qualified teachers in California's public schools. Teacher's College Record, 106, 1936-1966.
15. Debra P. v. Turlington, 564 F. Supp. 177, 186 (M.D. Fla. 1983).
16. Desimone, L.M., Smith, T.M.,&Phillips, K.J.R. (2007). Does policy influence mathematics and science teachers' participation in professional development? Teachers College Record, 109, 1086-1122.
17. DiBello, L.V., Roussos, L.A., & Stout, W. (2007). Review of cognitively diagnostic assessment and a summary of psychometric models. In S. Sinharay & C.R. Rao (Eds.), Handbook of statistics, Volume 26: Psychometrics (pp. 979-1030). New York: Elsevier.
18. Domestic Policy Council, Office of Science and Technology Policy. (2006). American competitiveness initiative: Leading the world in innovation. Retrieved August 11, 2008, fromwww.nist.gov/director/reports/ ACIBooklet.pdf.
19. Dorans, N.J. (2004). Using subpopulation invariance to assess test score equity. Journal of Educational Measurement, 41, 43-68.
20. Elmore, R.F., & Fuhrman, S.H. (1995). Opportunity-to-learn standards and the state role in education. Teachers College Record, 96, 432-457.
21. Ercikan, K., Gierl, M.J., McCreith, T., Puhan, G.,& Koh, K. (2004). Comparability of bilingual versions of assessments: Sources of incomparability of English and French versions of Canada's national achievement tests. Applied Measurement in Education, 17, 301-321.
22. Ferne, T., & Rupp, A.A. (2007). A synthesis of 15 years of research on DIF in language testing: Methodological advances, challenges, and recommendations. Language Assessment Quarterly, 4, 113-148.
23. Frisbie, D.A. (2005). Measurement 101: Some fundamentals revisited. Educational Measurement: Issues and Practice, 24(3), 21-28.
24. García, G.E.,&Pearson, P.D. (1994). Assessment and diversity. In L. Darling-Hammond (Ed.), Review of research in education: Vol. 20 (pp. 337-391). Washington, DC: American Educational Research Association.
25. Geier, R., Blumenfield, P.C., Marx, R.W., Krajcik, J.S., Fishman, B., Soloway, E.,& Clay-Chambers, J. (2008). Standardized test outcomes for students engaged in inquiry-based science curricula in the context of urban reform. Journal of Research in Science Teaching, 45, 922-939.
26. Hamilton, L.S. (1998). Gender differences on high school science achievement tests: Do format and content matter? Educational Evaluation and Policy Analysis, 20(3), 179-195.
27. Hess, F.M., & Petrilli, M.J. (2006). No child left behind primer. New York: Peter Lang.
28. Hewson, P.W., Kahle, J.B., Scantlebury, K., & Davies, D. (2001). Equitable science education in urban middle schools: Do reform efforts make a difference? Journal of Research in Science Teaching, 38(10), 1130-1144.
29. Holland, P.W., & Dorans, N.J. (2006). Linking and equating. In R.L. Brennan (Ed.), Educational measurement. 4th edn. (pp. 187-220). Westport, CT: American Council on Education and Praeger.
30. Joint Committee on Testing Practices. (2004). Code of fair testing practices in education.Washington, DC: Author.
31. Kahle, J.B., Meece, J., & Scantlebury, K. (2000). Urban African-American middle school science students: Does standards-based teaching make a difference? Journal of Research in Science Teaching, 37(9), 1019-1041.
32. Kane, M. (2006). Validation. In R.L. Brennan (Ed.), Educational measurement (4th edn., pp. 17-64). Westport: CT: American Council on Education and Praeger.
33. Kane, M. (2008). Terminology, emphasis, and utility in validation. Educational Researcher, 37, 76-82.
34. Kantor, H., & Lowe, R. (2006). From new deal to no deal: No Child Left Behind and the devolution of responsibility for equal opportunity. Harvard Educational Review, 76, 474-502.
35. Kieffer, M.J., Lesaux, N.K., & Snow, C.E. (2008). Promises and pitfalls: Implications of NCLB for identifying, assessing, and educating English language learners. In G.L. Sunderman (Ed.), Holding NCLB accountable: Achieving accountability, equity, and school reform (pp. 57-74). Thousand Oaks, CA: Corwin Press.
36. Knapp, M.S., & Plecki, M.L. (2001). Investing in the renewal of urban science teaching. Journal of Research in Science Teaching, 38, 1089-1100.
37. Kobrin, J.L., & Melican, G.J. (2007). Comparability of scores on the new and prior versions of the SAT Reasoning Test (College Board Research Note RN-31). New York: The College Board.
38. Kolen, M.J., & Brennan, R.L. (1995). Test equating: Methods and practices. New York: Springer-Verlag.
39. Koretz, D. (2008). The pending reauthorization of NCLB: An opportunity to rethink basic strategy. In G.L. Sunderman (Ed.), Holding NCLB accountable: Achieving accountability, equity, and school reform (pp. 9-26). Thousand Oaks, CA: Corwin Press.
40. Kornhaber, M.L. (2008). Beyond standardization in school accountability. In G.L. Sunderman (Ed.), Holding NCLB accountable: Achieving accountability, equity, and school reform (pp. 43-55). Thousand Oaks, CA: Corwin Press.
41. Lacelle-Peterson, M.W., & Rivera, C. (1994). Is it real for all kids? A framework for equitable assessment policies for English language learners. Harvard Educational Review, 64, 55-75.
42. Lee, O., & Fradd, S.H. (1998). Science for all, including students from non-English language backgrounds. Educational Researcher, 27(3), 12-21.
43. Lee, O., & Luykx, A. (2005). Dilemmas in scaling up educational innovations with nonmainstream students in elementary school science. American Educational Research Journal, 43, 411-438.
44. Lee, O., Maerten-Rivera, J., Penfield, R.D., LeRoy, K., & Secada,W.G. (2008). Science achievement of English language learners in urban elementary schools: Results of a first-year professional development intervention. Journal of Research in Science Teaching, 45, 31-52.
45. Levin, H.M. (2007). On the relationship between poverty and curriculum. North Carolina Law Review, 85, 1381-1417.
46. Linn, R.L. (2008). Toward a more effective definition of adequate yearly progress. In G.L. Sunderman (Ed.), Holding NCLB accountable: Achieving accountability, equity, and school reform (pp. 27-42). Thousand Oaks, CA: Corwin Press.
47. Lissitz, R.W., & Samuelson, K. (2007). A suggested change in terminology and emphasis regarding validity and education. Educational Researcher, 36, 437-448.
48. Liu, M., & Holland, P.W. (2008). Exploring population sensitivity of linking functions across three law school admission test administrations. Applied Psychological Measurement, 32, 27-44.
49. Lord, F.M. (1980). Applications of item response theory to practical testing problems. Hillsdale, NJ: Erlbaum.
50. Luykx, A., Lee, O., Mahotiere, M., Lester, B., Hart, J., & Deaktor, R. (2007). Cultural and home language influences on children's responses to science assessments. Teachers College Record, 109, 897-926.
51. Mislevy, R.J. (2006). Cognitive psychology and educational assessment. In R.L. Brennan (Ed.), Educational measurement (4th edn., pp. 257-305). Westport, CT: American Council on Education and Praeger.
52. National Center for Education Statistics. (2006). The nation's report card: Science 2005. Washington, DC: Government Printing Office.
53. National Research Council. (1999). High stakes: Testing for tracking, promotion, and graduation. Washington, DC: National Academy Press.
54. National Research Council. (2006). Systems for state science assessment. Washington, DC: National Academies Press.
55. National Science Foundation. (1996). Review of instructional materials for middle school science. Washington, DC: Author.
56. National Science Foundation. (1998). Infusing equity in systemic reform: An implementation scheme. Washington, DC: Author.
57. Nelson-Barber, S., Huang, C.-W., Trumbull, E., Johnson, Z., &Sexton, U. (2008, March). Elicitory test design: A novel approach to understanding the relationship between test item features and student performance on large-scale assessments. Paper presented at the annual meeting of the American Educational Researcher Association, New York.
58. No Child Left Behind Act of 2001, Public Law No. 107-110.
59. O'Neil, K.A.,&McPeek,W.M. (1993). Item and test characteristics that are associated with differential item functioning. In P.W. Holland & H. Wainer (Eds.), Differential item functioning (pp. 255-276). Hillsdale, NJ: Erlbaum.
60. PASE v. Hannon, 506 F. Supp. 831 (N.D. Ill. 1980).
61. Penfield, R.D. (2007). Assessing differential step functioning in polytomous items using a common odds ratio estimator. Journal of Educational Measurement, 44, 187-210.
62. Penfield, R.D., Alvarez, K.,&Lee, O. (2009). Using a taxonomy of differential step functioning form to improve the interpretation of DIF in polytomous items. Applied Measurement in Education, 22, 61-78.
63. Penfield, R.D.,&Camilli, G. (2007). Differential item functioning and item bias. In S. Sinharay &C.R. Rao (Eds.), Handbook of statistics, Volume 26: Psychometrics (pp. 125-167). New York: Elsevier.
64. Phillips, S.E. (1996). Legal defensibility of standards: Issues and policy perspectives. Educational Measurement: Issues and Practice, 15(2), 5-13, 19.
65. Phillips, S.E. (2001). GI Forum v. Texas Education Agency: Psychometric evidence. Applied Measurement in Education, 13, 343-385.
66. Phillips, S.E., & Camara, W.J. (2006). Legal and ethical issues. In R.L. Brennan (Ed.), Educational measurement (4th edn., pp. 733-755). Westport, CT: American Council on Education and Praeger.
67. Porter, A.C. (1995). The uses and misuses of opportunity-to-learn standards. Educational Researcher, 24(1), 21-27.
68. Porter, A.C. (2002). Measuring the content of instruction: Uses in research and practice. Educational Researcher, 31(7), 3-14.
69. Robelen, E.W. (2005). 40 years after ESEA, federal role in schools is broader than ever. Education Week, 24(31), 1, 42.
70. Rosebery, A.S., Warren, B., & Conant, F.R. (1992). Appropriating scientific discourse: Findings from language minority classrooms. Journal of the Learning Sciences, 21, 61-94.
71. Ruiz-Primo, M.A., & Shavelson, R.J. (1996). Rhetoric and reality in science performance assessments: An update. Journal of Research in Science Teaching, 33, 1045-1063.
72. Rumberger, R., & Gandara, P. (2004). Seeking equity in the education of California's English learners. Teachers College Record, 106, 2032-2056.
73. Shaw, J.M. (1997). Threats to the validity of science performance assessments for English language learners. Journal of Research in Science Teaching, 34, 721-743.
74. Shavelson, R.J., Baxter, G.P., & Pine, J. (1992). Performance assessments: Political rhetoric and measurement reality. Educational Researcher, 24(4), 22-27.
75. Siegel, M.A. (2007). Striving for equitable classroom assessments for linguistic minorities: Strategies for and effects of revising life science items. Journal of Research in Science Teaching, 44, 864-881.
76. Solano-Flores, G. (2006). Language, dialect, and register: Sociolinguistics and the estimation of measurement error in the testing of English language learners. Teachers College Record, 108, 2354-2379.
77. Solano-Flores, G., Lara, J., Sexton, U., & Navarrete, C. (2001). Testing English language learners: A sampler of student responses to science and mathematics items. Washington, DC: Council of Chief State School Officers.
78. Solano-Flores, G., & Li, M. (2006). The use of generalizability (G) theory in the testing of linguistic minorities. Educational Measurement: Issues and Practice, 25(1), 13-22.
79. Solano-Flores, G., & Nelson-Barber, S. (2001). On the cultural validity of science assessments. Journal of Research in Science Teaching, 38, 553-573.
80. Solano-Flores, G., & Trumbull, E. (2003). Examining language in context: The need for new research and practice paradigms in the testing of English-language learners. Educational Researcher, 32(2), 3-13.
81. Spillane, J.P., Diamond, J.B.,Walker, L.J., Halverson, R.,&Jita, L. (2001). Urban school leadership for elementary science instruction: Identifying and activating resources in an undervalued school subject. Journal of Research in Science Teaching, 38, 918-940.
82. Starratt, R.J. (2003). Opportunities to learn and the accountability agenda. Phi Delta Kappan, 85, 298-303.
83. Stecher, B.M., & Klein, S.P. (1997). The cost of performance assessments in large-scale testing programs. Educational Evaluation and Policy Analysis, 19, 1-4.
84. Sunderman, G.L. (2008). Introduction: Rethinking the challenge of NCLB. In G.L. Sunderman (Ed.), Holding NCLB accountable: Achieving accountability, equity, and school reform (pp. 1-8). Thousand Oaks, CA: Corwin Press.
85. Sunderman, G.L., Kim, J.S.,&Orfield, G. (2005). NCLB meets school realities: Lessons from the field. Thousand Oaks, CA: Corwin Press.
86. Traub, R.E., & Rowley, G.L. (1991). An NCME instructional module on understanding reliability. Educational Measurement: Issues and Practice, 10(1), 171-179.
87. Van Horn Helvey, S. (2002). Equation protection/Title VI and due process in high-stakes testing: An examination of the Chicago Public Schools' promotion policy. Children's Legal Rights Journal, 22, 23-34.
88. Yi, Q., & Harris, D.J. (2008). Invariance of equating functions across different subgroups of examinees taking a science achievement test. Applied Psychological Measurement, 32, 62-80.
89. Zenisky, A.L., Hambleton, R.K., & Robin, F. (2003-2004). DIF detection and interpretation in large-scale science assessments: Informing item writing practices. Educational Assessment, 9, 61-78.
90. Zwick, R. (2002). Fair game? The use of standardized admissions tests in higher education. New York: RoutledgeFalmer.