The role of cognitive validity testing in the development of CALL, the comprehensive assessment of leadership for learning

Journal of Educational Administration

Volumn 52, Issue 3, 2014, Pages 358-378

  Blitz, Mark H ^a   Salisbury, Jason ^a   Kelley, Carolyn ^a  

^a UNIVERSITY OF WISCONSIN MADISON   (United States)

Author keywords

Assessment; Cognitive validity; Leadership; Methodology; Organizational development

Indexed keywords

EID: 84898438803     PISSN: 09578234     EISSN: None     Source Type: Journal    
DOI: 10.1108/JEA-01-2013-0008     Document Type: Article

References (46)

1. Anderson, S., Leithwood, K., Strauss, T. (2010), "Leading data use in schools: organizational conditions and practices at the school and district levels" in Leadership and Policy in Schools, Vol. 9, No. 3, pp. 292-327.
2. Basit, T.N. (2010), Conducting Research in Educational Contexts, Continuum International Publishing Group, New York, NY.
3. Bassili, J.N., Scott, B.S. (1996), "Response latency as a signal to question problems in survey research" in Public Opinion Quarterly, Vol. 60, No. 3, pp. 390-399.
4. Beatty, P.C., Willis, G.B. (2007), "Research synthesis: the practice of cognitive interviewing" in Public Opinion Quarterly, Vol. 71, No. 2, pp. 287-311.
5. Blitz, M. (2012), "A case study comparison of school leadership practice against the comprehensive assessment of leadership for learning (CALL) pilot results", WCER Working Paper No. 2012-5, Retrieved from University of Wisconsin-Madison, Wisconsin Center for Education Research web site, Madison, WI, available at: www.wcer.wisc.edu/publications/workingPapers/papers.php (accessed January 21, 2013).
6. Blitz, M., Milanowski, T., Clifford, M. (2011), "Content validity as a window to a richer understanding of leadership practice", Annual Conference of the American Education Research Association. New Orleans, LA, April.
7. Boyatzis, R.E. (1998), Transforming Qualitative Information: Thematic Analysis and Code Development, Sage Publications, Thousand Oaks, CA.
8. Brown, A.L. (1992), "Design experiments: theoretical and methodological challenges in creating complex interventions in classroom settings" in The Journal of the Learning Sciences, Vol. 2, No. 2, pp. 141-178.
9. Bryk, A.S., Driscoll, M.E. (1985), An Empirical Investigation of the School as Community, University of Chicago, Department of Education, Chicago, IL.
10. Camburn, E., Salisbury, J. (2012), "Examining the validity and reliability of the comprehensive assessment of leadership for learning (CALL) formative leadership assessment tool: Pilot study results", Annual Conference of the American Education Research Association. Vancouver, BC, April.
11. Collins, A. (1992), "Toward a design science of education" in Scanlon, E. and O'Shea, T. (Eds.), New Directions in Educational Technology, Springer-Verlag, Berlin, pp. 15-20.
12. Collins, D. (2003), "Pretesting survey instruments: an overview of cognitive methods" in Quality of Life Research, Vol. 12, No. 3, pp. 229-238.
13. Condon, C., Clifford, M. (2010), Measuring Principal Performance: How Rigorous are Commonly Used Principal Performance Assessment Instruments?, A Quality School Leadership Issue Brief, Learning Point Associates, Naperville, IL.
14. Desimone, L.M., Le Floch, K.C. (2004), "Are we asking the right questions? Using cognitive interviews to improve surveys in education research" in Educational Evaluation and Policy Analysis, Vol. 26, No. 1, pp. 1-22.
15. Elmore, R.F. (2000) in Building a New Structure for School Leadership, Albert Shanker Institute, Washington, DC, pp. 1-46.
16. Goldring, E., Porter, A., Murphy, J., Elliott, S.N., Cravens, X. (2009), "Assessing learning-centered leadership: connections to research, professional standards, and current practices" in Leadership and Policy in Schools, Vol. 8, No. 1, pp. 1-36.
17. Hallinger, P., Heck, R.H. (1996), "Reassessing the principal's role in school effectiveness: a review of empirical research, 1980-1995" in Educational Administration Quarterly, Vol. 32, No. 1, pp. 5-44.
18. Halverson, R. (2010), "School formative feedback systems" in Peabody Journal of Education, Vol. 85, No. 2, pp. 130-146.
19. Halverson, R., Prichett, R.B., Watson, J.G. (2007), "Formative feedback systems and the new instructional leadership", WCER Working Paper No. 2007-3, Wisconsin Center for Education Research, University of Wisconsin-Madison, Madison, WI, available at: www.wcer.wisc.edu/publications/workingPapers/papers.php (accessed May 15, 2007).
20. Harris, A., Spillane, J. (2008), "Distributed leadership through the looking glass" in Management in Education, Vol. 22, No. 1, p. 31.
21. Heck, R.H., Hallinger, P. (2009), "Assessing the contribution of distributed leadership to school improvement and growth in math achievement" in American Educational Research Journal, Vol. 46, No. 3, pp. 659-689.
22. Jobe, J.B., Tourangeau, R., Smith, A.F. (1993), "Contributions of survey research to the understanding of memory" in Applied Cognitive Psychology, Vol. 7, No. 7, pp. 567-584.
23. Karabenick, S., Woolley, M., Friedel, J., Ammon, B., Blazevski, J.B., Bonney, C.R., Groot, E.D., Gilbert, M.C., Musu, L., Kempler, T.M., Kelly, K.L. (2007), "Cognitive processing of self-report items in educational research: do they think what we mean" in Educational Psychologist, Vol. 42, No. 3, pp. 139-151.
24. Kelley, C., Halverson, R., Camburn, E., Blitz, M., Salisbury, J., Dikkers, S., Clifford, C. (2012), "The design and validation of the comprehensive assessment of leadership for learning (CALL) formative school leader assessment", paper prepared for the Association of Educational Finance and Policy Conference, Boston, MA, March 17.
25. Leithwood, K., Louis, K.S. (2011), Linking Leadership to Student Learning, Jossey-Bass, San Francisco, CA.
26. Leithwood, K., Riehl, C. (2005), "What we know about successful school leadership" in Firestone, W. and Riehl, C. (Eds.), A new Agenda: Directions for Research on Educational Leadership, Teachers College Press, New York, NY, pp. 22-47.
27. Leithwood, K., Jantzi, D., Steinbach, R. (1999), Changing Leadership for Changing Times. Changing Education Series, Taylor and Francis Group, Florence, KY.
28. Leithwood, K., Patten, S., Jantzi, D. (2010), "Testing a conception of how school leadership influences student learning" in Educational Administration Quarterly, Vol. 46, No. 5, pp. 671-706.
29. Leithwood, K., Sun, J. (2012), "The nature and effects of transformational school leadership a meta-analytic review of unpublished research" in Educational Administration Quarterly, Vol. 48, No. 3, pp. 387-423.
30. Mandinach, E.B., Gummer, E.S. (2013), "A systemic view of implementing data literacy in educator preparation" in Educational Researcher, Vol. 42, No. 1, pp. 30-37.
31. Mayrowetz, D. (2008), "Making sense of distributed leadership: exploring the multiple usages of the concept in the field" in Educational Administration Quarterly, Vol. 44, No. 3, pp. 424-435.
32. Mullens, J.E., Gayler, K., Goldstein, D., Hildreth, J., Rubenstein, M., Spiggle, T., Walking Eagle, K., Welsh, M. (1999), "Measuring classroom instructional processes: Using survey and case study fieldtest results to improve item construction", working paper series, Washington, DC.
33. Murphy, J., Elliott, S.N., Goldring, E., Porter, A.C. (2007), "Leadership for learning: a research-based model and taxonomy of behaviors" in School Leadership and Management, Vol. 27, No. 2, pp. 179-201.
34. Nelson, S.W., Maria, G., Boone, M.D. (2008), "Lifeworld or systemsworld: what guides novice principals?" in Journal of Educational Administration, Vol. 46, No. 6, pp. 690-701.
35. Pearce, C.L., Conger, J.A. (2003), Shared Leadership: Reframing the Hows and Whys of Leadership, Sage Publications.
36. Porter, A.C., Polikoff, M.S., Goldring, E., Murphy, J., Elliott, S.N., May, H. (2010), "Investigating the validity and reliability of the Vanderbilt Assessment of Leadership in Education" in Elementary School Journal, Vol. 111, No. 2, pp. 282-313.
37. Presser, S., Couper, M.P., Lessler, J.T., Martin, E., Martin, J., Rothgeb, J.M., Singer, E. (2004), "Methods for testing and evaluating survey questions" in Public Opinion Quarterly, Vol. 68, No. 1, pp. 109-130.
38. Robinson, M.A., Kannapel, P., Gujarati, J., Williams, H., Oettinger, A. (2008), "A formative study of the implementation of the inquiry team process in New York City public schools", Journal of Curriculum Studies, 2007-2008 findings, Consortium for Policy Research in Education.
39. Ruiz-Primo, M.A., Schultz, S.E., Li, M., Shavelson, R.J. (2001), "Comparison of the reliability and validity of scores from two concept-mapping techniques" in Journal of Research in Science Teaching, Vol. 38, No. 2, pp. 260-278.
40. Spillane, J.P. (2007), "Distributed leadership" in The Educational Forum, Vol. 69, No. 2, pp. 143-150.
41. Spillane, J.P., Diamond, J.B. (2007), Distributed Leadership in Practice, Teachers College Press, New York, NY.
42. Spillane, J.P., Halverson, R., Diamond, J.B. (2004), "Towards a theory of leadership practice: a distributed perspective" in Journal of Curriculum Studies, Vol. 36, No. 1, pp. 3-34.
43. Stoll, L., McMahon, A., Thomas, S. (2006), "Identifying and leading effective professional learning communities" in Journal of School Leadership, Vol. 16, No. 5, pp. 611-623.
44. Thomas, D.W., Holdaway, E.A., Ward, K.L. (2000), "Policies and practices involved in the evaluation of school principals" in Journal of Personnel Evaluation in Education, Vol. 14, No. 3, pp. 215-240.
45. Wildy, H., Clarke, S. (2009), "Using cognitive interviews to pilot an international survey of principal preparation: a Western Australian perspective" in Educational Assessment, Evaluation, and Accountability, Vol. 21, No. 1, pp. 105-117.
46. Firestone, W.A., Riehl, C. (2005), A New Agenda for Research in Educational Leadership, Teacher College Press, New York, NY.