The development and validation of The Inquiry Science Observation Coding Sheet

Evaluation and Program Planning

Volumn 31, Issue 3, 2008, Pages 247-258

  Brandon, P R ^a   Taum, A K H ^a   Young, D B ^a   Pottenger III, F M ^a  

^a UNIVERSITY OF HAWAII   (United States)

Author keywords

Inquiry science; Observations; Program implementation; Validity

Indexed keywords

IMPLEMENTATION PROCESS; OBSERVATIONAL METHOD; RESEARCH PROGRAM;

ARTICLE; EDUCATION; EVALUATION; HEALTH CARE QUALITY; MEDICAL RECORD; OBSERVATION; SCHOOL; SCIENCE; STANDARD; UNITED STATES; VALIDATION STUDY;

EVALUATION STUDIES AS TOPIC; FORMS AND RECORDS CONTROL; OBSERVATION; PROGRAM EVALUATION; SCHOOLS; SCIENCE; UNITED STATES;

EID: 46049108135     PISSN: 01497189     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.evalprogplan.2008.03.007     Document Type: Article

References (68)

1. Ayala, C. C. (2005). Development and validation of an inquiry science student achievement and attitudinal suite. Paper presented at the meeting of the American Evaluation Association, Toronto, October.
2. Berman P., and McLaughlin M.W. Implementation of educational innovation. Educational Forum 3 (1976) 345-371
3. Bond G.R., Evans L., Salyers M.P., Williams J., and Kim H.-W. Measurement of fidelity in psychiatric rehabilitation. Mental Health Services Research 2 (2000) 75-87
4. Brandon, P. R., Young, D. B., Taum, A. K. H., & Pottenger, F. M. (2008). The Inquiry Science Implementation Scale: Instrument development and the results of validation studies. Manuscript submitted for publication.
5. Brandon, P. R., Taum, A. K. H., Young, D. B., Pottenger, F. M., & Speitel, T. W. (in press). The complexity of measuring the quality of program implementation with observations: The case of middle-school inquiry-based science. American Journal of Evaluation.
6. Center for Science Education. (2007). The Inquiry Science Instruction Observation Protocol (ISIOP) Development Project. Retrieved October 2, 2007 from 〈http://cse.edc.org/projects/ProjectView.asp?PID=3719〉.
7. Cohen D.K., and Spillane J.P. Policy and practice: The relationship between governance and instruction. In: Fuhrman S.H. (Ed). Designing coherent education policy: Improving the system (1993), Jossey-Bass, San Francisco 35-95
8. Cronbach L.J., Ambron S.R., Dornbusch S.M., Hess R.D., Hornik R.C., Phillips D.C., et al. Toward reform of program evaluation (1980), Jossey-Bass, San Francisco
9. Curriculum Research & Development Group. Alignment of Foundational Approaches in Science Teaching (FAST) with the National Science Education Standards grades 5-8 (1996), Author, Honolulu
10. Dane A.V., and Schneider B.H. Program integrity in primary and early secondary prevention: Are implementation effects out of control?. Clinical Psychology Review 18 1 (1998) 23-45
11. de Jong T., and Ferguson-Hesser M.G.M. Types and qualities of knowledge. Educational Psychologist 31 (1996) 105-113
12. Dillman D.A. Mail and Internet surveys: The tailored design method (2000), Wiley, New York
13. Domitrovich C.E., and Greenberg M.T. The study of implementation: Current findings from effective programs that prevent mental disorders in school-aged children. Journal of Educational and Psychological Consultation 11 (2000) 193-221
14. Dumas J.E., Lynch A.M., Laughlin J.E., Smith E.P., and Prinz R.J. Promoting intervention fidelity: Conceptual issues, methods, and preliminary results from the EARLY ALLIANCE prevention trial. American Journal of Preventive Medicine 20 (2001) 38-47
15. Durlak J.A. Successful prevention programs for children and adolescents (1997), Plenum, New York
16. Duschl R.A. Assessment of inquiry. In: Atkin M., and Coffey J.E. (Eds). Everyday assessment in the science classroom (2003), National Science Teachers Association, Arlington, VA 41-59
17. Dusenbury L., Brannigan R., Falco M., and Hansen W. A review of research on fidelity of implementation: Implications for drug abuse prevention in school settings. Health Education Research 18 (2003) 237-256
18. Dusenbury L., Brannigan R., Hansen W.B., Walsh J., and Falco M. Quality of implementation: Developing measures crucial to understand the diffusion of preventive interventions. Health Education Research 20 (2005) 308-313
19. Evertson C.M., and Green J.L. Observation as inquiry and method. In: Wittrock M.C. (Ed). Handbook of research on teaching (1986), Macmillan, New York 162-213
20. Forgatch M.S., Patterson G.R., and DeGarmo D.S. Evaluating fidelity: Predictive validity for a measure of component adherence to the Oregon Model of Parent Management Training. Behavior Therapy 36 (2005) 3-13
21. Fuchs L.S., Fuchs D., and Karns K. Enhancing kindergartners' mathematical development: Effects of peer-assisted learning strategies. The Elementary School Journal 101 (2001) 495-510
22. Fullan M., and Pomfret A. Research on curriculum and instruction implementation. Review of Educational Research 47 (1977) 335-397
23. Gall M. The use of questions in teaching. Review of Educational Research 40 (1970) 707-721
24. Gall M. Synthesis of research on teachers' questioning. Educational Leadership 42 3 (1984) 40-47
25. Gresham F.M. Assessment of treatment integrity in school consultation and prereferral intervention. School Psychology Review 18 1 (1989) 37-50
26. Gresham F.M., and Gansle K.A. Treatment integrity of school-based behavioral intervention studies: 1980-1990. School Psychology Review 22 2 (1993) 1-21
27. Guilford J.P. Psychometric methods. 2nd ed. (1954), McGraw-Hill, New York
28. Hall G.E., and Loucks S.F. A developmental model for determining whether the treatment is actually implemented. American Educational Research Journal 17 (1977) 185-195
29. Hall, G. E., & Loucks, S. F. (1978). Innovation configurations: Analyzing the adaptation of innovations. Paper presented at the meeting of the American Educational Research Association, Toronto.
30. Harlen W. The national research council in preparation for a meeting on the status of evaluation of inquiry-based science education (2004), National Academy of Sciences, University of Cambridge and University of Bristol
31. Heck R.H., Brandon P.R., and Wang J. Implementing site-managed educational changes: Examining levels of implementation and effect. Educational Policy 15 (2001) 302-322
32. Hilberg R.S., Doherty R.W., Epaloose G., and Tharp R.G. The standards performance continuum: A performance-based measure of the standards for effective pedagogy. In: Waxman H.C., Tharp R.G., and Hilberg R.S. (Eds). Observational research in US classrooms: New approaches for understanding cultural and linguistic diversity (2004), Cambridge University Press, Cambridge, England 48-71
33. Hughes T.A., Fredrick L.D., and Keel M.C. Learning to effectively implement constant time delay procedures to teach spelling. Learning Disability Quarterly 25 (2002) 209-220
34. Inquiry Synthesis Project. Technical Report 2: Conceptualizing inquiry science instruction (2004), Center for Science Education, Education Development Center, Inc, Newton, MA
35. Kirby S.N., Berends M., and Naftel S. Implementation in a longitudinal sample of New American schools: Four years into scale-up (2001), RAND, Santa Monica, CA
36. Lastica, J. R., & O'Donnell, C. L. (2007, April). Considering the role of fidelity of implementation in science education research: Fidelity as teacher and student adherence to structure. In C.L. O'Donnell (Chair), Analyzing the relationship between fidelity of implementation (FOI) and student outcomes in a quasi-experiment. Symposium conducted at the meeting of the American Educational Research Association, Chicago.
37. Lee, O. (2005, April). Adaptation of an instructional intervention in linguistically, culturally, and socioeconomically diverse elementary schools. Paper presented at the meeting of the American Educational Research Association, Montreal.
38. Lee O., Hart J.E., Cuevas P., and Enders C. Professional development in inquiry-based science for elementary teachers of diverse student groups. Journal of Research in Science Teaching 41 (2004) 1021-1043
39. Li, M., & Shavelson, R. J. (2001, April). Examining the links between science achievement and assessment. Paper presented at the meeting of the American Educational Research Association, Seattle.
40. Lott G.W. The effect of inquiry teaching and advanced organizers upon student outcomes in science education. Journal of Research in Science Teaching 29 (1983) 437-451
41. Lynch, S. (2007, April). A model for fidelity of implementation in a study of a science curriculum unit: Evaluation based on program theory. In C.L. O'Donnell (Chair), Analyzing the relationship between fidelity of implementation (FOI) and student outcomes in a quasi-experiment. Symposium conducted at the meeting of the American Educational Research Association, Chicago. Retrieved May 22, 2007 from 〈http://www.gwu.edu/~scale-up/documents/Model_for_Fidelity_AERA_2007.pdf〉.
42. McHugo G.J., Drake R.E., Teague G.B., and Hie H. Fidelity to assertive community treatment and client outcomes in the New Hampshire dual disorders study. Psychiatric Services 50 (1999) 818-824
43. Moncher F.J., and Prinz R.J. Treatment fidelity in outcome studies. Clinical Psychology Review 11 (1991) 247-266
44. Mowbray C.T., Bybee D., Holter M., and Lewandowski L. Validation of a fidelity rating instrument for consumer-operated services. American Journal of Evaluation 27 (2006) 9-27
45. Mowbray C.T., Holter M.C., Teague G.B., and Bybee D. Fidelity criteria: Development, measurement, and validation. American Journal of Evaluation 24 (2003) 315-340
46. National Research Council. The national science education standards (1996), The National Academies, Washington, DC
47. National Research Council. On evaluating curricular effectiveness: Judging the quality of K-12 mathematics evaluations (2004), The National Academies, Washington, DC
48. O'Donnell C.L. Defining, conceptualizing, and measuring fidelity of implementation and its relationship to outcomes in K-12 curriculum intervention research. Review of Educational Research 78 (2008) 33-84
49. Pottenger F.M., and Young D.B. Instructional guide: FAST, Foundational approaches in science teaching. 2nd ed. (1992), University of Hawaii at Manoa, Curriculum Research & Development Group, Honolulu
50. Pottenger F.M., and Young D.B. The local environment: FAST 1, Foundational approaches in science teaching. 2nd ed. (1992), University of Hawaii at Manoa, Curriculum Research & Development Group, Honolulu
51. Pottenger F.M., and Young D.B. The local environment: FAST 1, Foundational approaches in science teaching: teacher's guide. 2nd ed. (1992), University of Hawaii at Manoa, Curriculum Research & Development Group, Honolulu
52. Redfield D.L., and Rousseau E.W. A meta-analysis of experimental research on teacher questioning behavior. Review of Educational Research 51 (1981) 236-245
53. Resnicow K., Davis M., Smith M., Lazarus-Yaroch A., Barajnowski T., Doyle C., et al. How best to measure implementation of school health curricula: A comparison of three measures. Health Education Research 13 (1998) 239-250
54. Rogg S., and Kahle J.B. Middle level standards-based inventory (1997), Miami University of Ohio, Oxford, OH
55. Rohrbach L.A., Graham J.W., and Hansen W.B. Diffusion of a school-based substance abuse prevention program: Predictors of program implementation. Preventive Medicine 22 (1993) 237-260
56. Samson G.E., Sirykowski B., Weinstein T., and Walberg H.J. The effects of teacher questioning levels on student achievement: A quantitative synthesis. Journal of Educational Research 80 (1987) 290-295
57. Scheirer M.A., and Rezmovic E.L. Measuring the degree of program implementation: A methodological review. Evaluation Review 7 (1983) 599-633
58. Shavelson, R. J., Young, D. B., Ayala, C. C., Brandon, P. R., Furtak, E. M., Ruiz-Primo, M. A., et al. (in press). On the impact of curriculum-embedded formative assessment on learning: A collaboration between curriculum and assessment developers. Applied Measurement in Education.
59. Shymansky J.A., Kyle W.C., and Alport J.M. The effects of new science curricula on student performance. Journal of Research in Science Teaching 20 (1983) 387-404
60. Stemler, S. E. (2004). A comparison of consensus, consistency, and measurement approaches to estimating interrater reliability. Practical Assessment Research & Evaluation, 9(4). Retrieved January 3, 2006 from 〈http://pareonline.net/getvn.asp?v=9&n=4/〉.
61. Sudman S., Bradburn N.M., and Schwarz N. Thinking about answers: The application of cognitive processes to survey methodology (1996), Jossey-Bass, San Francisco
62. Supovitz J.A., and May H. A study of the links between implementation and effectiveness of the America's Choice comprehensive school reform design. Journal of Education for Students Placed at Risk 9 (2004) 389-419
63. Tamir P. Inquiry and the science teacher. Science Teacher Education 67 (1983) 657-672
64. Taum, A. K. H. (2004). Foundational Approaches to Science Teaching and the five standards for effective pedagogy. Unpublished manuscript.
65. Tharp R.G., Estrada P., Dalton S., and Yamauchi L. Teaching transformed: Achieving excellence, fairness, inclusion, and harmony (2000), Westview Press, Boulder, CO
66. Von Secker C., and Lissitz R. Estimating the impact of instructional practices on student achievement in science. Journal of Research in Science Teaching 36 (1999) 1110-1126
67. Wu H.K., and Hseih C.E. Developing sixth grader's inquiry skills to construct explanations in inquiry-based learning environments. International Journal of Science Education 28 (2006) 1290-1313
68. Young D.B. Instructor's manual for FAST trainer development program (1989), University of Hawaii at Manoa, Curriculum Research & Development Group, Honolulu