Setting Proficiency Standards for School Leadership Assessment: An Examination of Cut Score Decision Making

Educational Administration Quarterly

Volumn 49, Issue 1, 2013, Pages 124-160

  Cravens, Xiu Chen ^a   Goldring, Ellen B ^a   Porter, Andrew C ^b   Polikoff, Morgan S ^c   Murphy, Joseph ^a   Elliott, Stephen N ^d  

^a VANDERBILT UNIVERSITY   (United States)

^b UNIVERSITY OF PENNSYLVANIA   (United States)

^c UNIVERSITY OF SOUTHERN CALIFORNIA   (United States)

^d ARIZONA STATE UNIVERSITY   (United States)

Author keywords

assessment; cut score; evaluation; proficiency standard; school leadership; standard setting

Indexed keywords

EID: 84872431523     PISSN: 0013161X     EISSN: 15523519     Source Type: Journal    
DOI: 10.1177/0013161X12455330     Document Type: Article

References (42)

1. Angoff W. H.Educational measurement. 1971). Scale, norms and equivalent scores. In ThorndikeR. L. (Ed.), Educational measurement (2nd ed.). Washington, DC: American Council on Education.Thorndike R. L., ed. Washington, DC: American Council on Education; 1971:.
2. Berk R. A.Standard setting: The next generation (Where few psychometricians have gone before!).Applied Measurement in Education. 1996;9:215-235.
3. Buckendahl C. W.Qualitative inquiries of participants' experiences with standard setting.Applied Measurement in Education. 2005;18 (3): 219-221.
4. Buckendahl C. W.,Smith R. W.,Impara J. C.,Plake B. S.A comparison of Angoff and bookmark standard setting methods.Journal of Educational Measurement. 2002;39 (3): 253-263.
5. Cascio W. F.,Alexander R. A.,Barrett G. V.Setting cutoff score: Legal, psychometric, and professional issues and guidelines.Personnel Psychology. 1988;41:1-24.
6. Cavanaugh J.,Perlmutter M.Metamemory: A critical examination.Child Development. 1982;53:11-28.
7. Cizek G. J.Reconsidering standards and criteria.Journal of Educational Measurement. 1993;30 (2): 93-106.
8. Cizek G. J.Setting Performance Standards: Concepts, Methods, and Perspectives. Mahwah, NJ: Lawrence Erlbaum; 2006:.
9. Interstate school leaders licensure consortium standards for school leaders. Washington, DC: Author; 1996:.
10. Educational Leadership Policy Standards: 2008. Washington, DC: Author; 2008:.
11. Crano W. D.,Brewer M. B.Principles and methods of social research. Mahwah, NJ: Lawrence Erlbaum; 2002:.
12. Downing S. M.,Haladyna T. M.Setting performance standards: Concepts, methods, and perspectives. Mahwah, NJ: Lawrence Erlbaum; 2006:.
13. Ferdous A. A.,Plake B. S.Understanding the factors that influence decisions of panelists in a standard-setting study.Applied Measurement in Education. 2005;18 (3): 257-267.
14. Fitzpatrick A. R.Social Influences in standard setting: The effects of social interaction on group judgments.Review of Educational Research. 1989;59 (3): 315-328.
15. Ginsberg R.,Berry B.The folklore of principal evaluation.Journal of Personnel Evaluation in Education. 1990;3:205-230.
16. Giraud G.,Impara J. C.Making the cut: The cut score setting process in a public school district.Applied Measurement in Education. 2005;18 (3): 289-312.
17. GiraudG. T. (1999). Making the cut: A qualitative inquiry into the setting of cut scores on school district assessments (Unpublished doctoral dissertation). University of Nebraska, Lincoln.
18. Goldring E.,Cravens X.,Murphy J.,Porter A.,Elliott S.,Carson B.The evaluation of principals: What and how do states and urban districts assess leadership?.Elementary School Journal. 2009;110 (1): 19-39.
19. Goldring E.,Porter A.,Murphy J.,Elliott S. N.,Cravens X.Assessing learning-centered leadership: Connections to research, professional standards, and current practices.Leadership and Policy in Schools. 2009;8:1-36.
20. Green B. F. Paper presented at the MAPAC; 2000Washington, DC; 2000.
21. Haertel E. H.Standard setting as a participatory process: Implications for validation of standard-based accountability programs.Educational Measurement: Issues and Practice. 2002;22 (1): 22-32.
22. Hein S. F.,Skaggs G.Conceptualizing the classroom of target students: A qualitative investigation of panelists' experiences during standard setting.Educational Measurement: Issues and Practice. 2010;29 (2): 36-44.
23. Kane M.Validating the performance standards associated with passing scores.Review of Educational Research. 1994;64:425-461.
24. Karantonis A.,Sireci G.The bookmark standard-setting method: A literature review.Educational Measurement: Issues and Practice. 2006;25 (1): 4-12.
25. Kottkamp R. B.Introduction: Leadership preparation in education.Educational Administration Quarterly. 2011;47 (1): 3-17.
26. Kowal J.,Hassel E. A.Measuring teacher and leader performance. Chapel Hill, NC: Public Impact; 2010:.
27. Lane S.Implications of cognitive psychology for measurement and testing: Assessing students' knowledge structures.Educational Measurement: Issues and Practice. 1991;10 (1): 31-33.
28. Linn R.Demands, cautions and suggestions for setting standards.Journal of Educational Measurement. 1978;15:301-308.
29. McGinty D.Illuminating the "black box" of standard setting: An exploratory qualitative study.Applied Measurement in Education. 2005;18 (3): 269-287.
30. Mitzel H. C.,Lewis D. M.,Patz R. J.,Green D. R.Setting performance standards: Concepts, methods, and perspectives. Cizek G. J., ed. Mahwah, NJ: Lawrence Erlbaum; 2001:249-281.
31. Moustakas C.Phenomenological research methods. Thousand Oaks, CA: Sage; 1994:.
32. Nisbett R.,Wilson T.Telling more than we can know: Verbal reports on mental processes.Psychological Review. 1977;84:231-259.
33. Porter A.,Goldring E.,Murphy J.,Elliott S. N.,Cravens X.A conceptual framework for the assessment of principal and team school leadership. New York, NY: Wallace Foundation; 2006:.
34. Porter A. C.,Goldring E. B.,Elliott S. N.,Murphy J.,Polikoff M.,Cravens X.Setting performance standards for the VAL-ED assessment of principal leadership. New York, NY: Wallace Foundation; 2008:.
35. PorterA. C.MurphyJ.GoldringE. B.ElliottS. N. (2011). Framework for a principal evaluation system. Retrieved from valed.com/documents/Framework_for_Principal_Eval.pdf.
36. Porter A. C.,Polikoff M.,Goldring E. B.,Murphy J.,Elliott S. N.,May H.Developing a psychometrically sound assessment of school leadership: The VAL-ED as a case study.Educational Administration Quarterly. 2010;46 (2): 135-173.
37. Portin B. S.,Feldman S.,Knapp M. S.Purposes, uses, and practices of leadership assessment in education. New York, NY: Wallace Foundation; 2006:.
38. Shepard L.,Glaser R.,Linn R.,Bohrnstedt G.Setting performance standards for student achievement. Stanford, CA: National Academy of Education; 1993:.
39. Skorupski W. P.,Hambleton R. K.What are panelists thinking when they participate in standard-setting studies?.Applied Measurement in Education. 2005;18 (3): 233-256.
40. Schmidt F. L.,Hunter J. E.The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings.Psychological Bulletin. 1998;124 (2): 262-274.
41. Strauss A.,Corbin J.Basics of qualitative research. Thousand Oaks, CA: Sage; 1998:.
42. A blueprint for reform: The reauthorization of the Elementary and Secondary Education Act. Washington, DC: Office of Planning, Evaluation and Policy Development; 2010:.