Peer observations among faculty in a college of education: Investigating the summative and formative uses of the Reformed Teaching Observation Protocol (RTOP)

Educational Assessment, Evaluation and Accountability

Volumn 24, Issue 1, 2012, Pages 5-24

  Amrein Beardsley, Audrey ^a   Popp, Sharon E Osborn ^b  

^a ARIZONA STATE UNIVERSITY   (United States)

^b Western Regional Examining Board   (United States)

Author keywords

Higher education; Instructional method; Many faceted Rasch model; Peer observation; Professional development

Indexed keywords

EID: 84855988485     PISSN: 18748597     EISSN: 18748600     Source Type: Journal    
DOI: 10.1007/s11092-011-9135-1     Document Type: Article

References (44)

1. Adamson, S. L., Banks, D., Burtch, M., Cox, F., Judson, E., Turley, J. B., et al. (2003). Reformed undergraduate instruction and its subsequent impact on secondary school teaching practice and student achievement. Journal of Research in Science Teaching, 40(10), 939-957. doi: 10. 1002/tea. 10117.
2. Andrich, D. (1978). A rating formulation for ordered categories. Psychometrika, 43, 357-374.
3. Bernstein, D. J. (2008). Peer review and evaluation of the intellectual work of teaching. Change: The Magazine of Higher Learning, 40(2), 48-51. doi: 10. 3200/CHNG. 40. 2. 48-51.
4. Bingham, R., & Ottewill, R. (2001). Whatever happened to peer review? Revitalising the contribution of tutors to course evaluation. Quality Assurance in Education, 9(1), 22-39. doi: 10. 1108/09684880110381319.
5. Blackmore, J. A. (2005). A critical evaluation of peer review via teaching observation within higher education. International Journal of Educational Management, 19(3), 218-232. doi: 10. 1108/09513540510591002.
6. Boyer, E. L. (1987). College: The undergraduate years. New York: Harper and Row Publishers Inc.
7. Cole, R. E. (2003). New organizational designs for sustainable quality improvement. Keynote address delivered at the 6th International Conference on Quality Management and Organisational Development (QMOD), Paris, France.
8. Creswell, J. W., Plano Clark, V. L., Gutmann, M. L., & Hanson, W. E. (2003). Advanced mixed method research designs. In A. Tashakkori & C. Teddlie (Eds.), Handbook of mixed methods in social and behavioral research (pp. 209-240). Thousand Oaks: Sage.
9. Cross, K. P. (1986). A proposal to improve teaching - or - what taking teaching seriously should mean. AAHE Bulletin, 39(1), 9-14.
10. Dommeyer, C. J., Baum, P., Hanna, R., & Chapman, K. S. (2004). Gathering faculty teaching evaluations by in-class and online surveys: their effects on response rates and evaluations. Assessment & Evaluation in Higher Education, 29(5), 611-623. doi: 10. 1080/02602930410001689171.
11. Erickson, F. (1986). Qualitative methods in research on teaching. In M. Wittrock (Ed.), Handbook of Research on Teaching, vol 3 (pp. 119-161). New York: Macmillan.
12. Gijbels, D., van de Watering, G., Dochy, F., & van den Bossche, P. (2006). New learning environments and constructivism: The students' perspective. Instructional Science: An International Journal of Learning and Cognition, 34(3), 213-226. doi: 10. 1007/s11251-005-3347-z.
13. Gosling, D. (2002). Models of peer observation of teaching. Birmingham: Learning and Teaching Support Network Generic Centre Learning and Teaching Support Network.
14. Greene, J. C. (2007). Mixed methods social inquiry. San Francisco: Jossey-Bass.
15. Hammersley-Fletcher, L., & Orsmond, P. (2005). Reflecting on reflective practices within peer observation. Studies in Higher Education, 30(2), 213-224. doi: 10. 1080/03075070500043358.
16. Heitzmann, R. (2010). Ten suggestions for enhancing lecturing. Education Digest, 75(9), 50-54.
17. Hendry, G. D., Cumming, R. G., Lyon, P. M., & Gordon, J. (2001). Student-centered course evaluation in a four-year, problem based medical programme: Issues in collection and management of feedback. Assessment & Evaluation in Higher Education, 26(4), 327-339.
18. Hutchings, P. (1995). From idea to prototype: The peer review of teaching. Sterling: Stylus.
19. Hutchings, P. (1996). Making teaching community property: A menu for peer collaboration and peer review. Sterling: Stylus.
20. Hutchings, P. (1998). The course portfolio. Sterling: Stylus.
21. Keig, L. (2000). Formative peer review of teaching: attitudes of faculty at liberal arts colleges towards colleague assessment. Journal of Personnel Evaluation in Education, 14(1), 67-87.
22. Lawson, A. E. (2003). Using the RTOP to evaluate reformed science and mathematics instruction: Improving undergraduate instruction in science, technology, engineering, and mathematics. Committee on Undergraduate Science Education. Washington D. C.: National Academies Press, National Research Council.
23. Lawson, A. E., Benford, R., Bloom, I., Carlson, M. P., Falconer, K., Hestenes, D., et al. (2002). Evaluating college science and mathematics instruction. Journal of College Science Teaching, 36, 388-393.
24. Linacre, J. M. (2002). Optimizing rating scale category effectiveness. Journal of Applied Measurement, 3(1), 85-106.
25. Linacre, J. M. (2009). A user's guide to FACETS. Chicago: Winsteps. com.
26. Linacre, J. M., & Wright, B. D. (1994). Reasonable mean-square fit values. Rasch Measurement Transactions, 8, 370. Retrieved from http://www. rasch. org/rmt/rmt83b. htm.
27. Linacre, J. M., Engelhard, G., Jr., Tatum, D. S., & Myford, C. M. (1994). Measurement with judges: many-faceted conjoint measurement. International Journal of Educational Research, 21(6), 569-577. doi: 10. 1016/0883-0355(94)90011-6.
28. Loyens, S. M. M., Rikers, R. M. J. P., & Schmidt, H. G. (2009). Students' conceptions of constructivist learning in different programme years and different learning environments. British Journal of Educational Psychology, 79(3), 501-514. doi: 10. 1348/000709908X378117.
29. MacIsaac, D., Sawado, D., & Falconer, K. (2001). Using the Reformed Teaching Observation Protocol (RTOP) as a catalyst for self-reflective change in secondary science teaching. Paper presented at annual meeting of the American Educational Research Association, Seattle, WA.
30. Marsh, H. W., Muthen, B., Asparouhov, T., Ludtke, O., Robitzsch, A., Morin, A. J. S., et al. (2009). Exploratory structural equation modeling, integrating CFA and EFA: Application to students' evaluations of university teaching. Structural Equation Modeling: A Multidisciplinary Journal, 16(3), 439-476. doi: 10. 1037/a0019227.
31. Masters, G. N. (1982). A Rasch model for partial credit scoring. Psychometrika, 47, 149-174. doi: 10. 1007/BF02296272.
32. Miles, M. B., & Huberman, A. M. (1994). Qualitative data analysis (2nd ed.). Thousand Oaks: Sage Publications.
33. Ngware, M. W. (2005). An improvement in instructional quality: can evaluation of teaching effectiveness make a difference? Quality Assurance in Education, 13(3), 183-201. doi: 10. 1108/09684880510607936.
34. Peel, D. (2005). Peer observation as a transformatory tool? Teaching in Higher Education, 10(4), 489-504. doi: 10. 1080/13562510500239125.
35. Piburn, M., Sawada, D., Turley, J., Falconer, K., Benford, R., Bloom, I., et al. (2000). Reformed Teaching Observation Protocol (RTOP) reference manual. ACEPT Technical Report No. IN00-3. Tempe: Arizona Board of Regents.
36. Rasch, G. (1960/1980). Probabilistic models for some intelligence and attainment tests. Copenhagen: Danish Institute for Educational Research, 1960. Expanded edition, Chicago: The University of Chicago Press, 1980.
37. Roberts, T. G., Irani, T. A., Telg, R. W., & Lundy, L. K. (2005). The development of an instrument to evaluate distance education courses using student attitudes. American Journal of Distance Education, 19(1), 51-64. doi: 10. 1207/s15389286ajde1901_5.
38. Roche, L. A., & Marsh, H. W. (2000). Multiple dimensions of university teacher self-concept: construct validation and the influence of students' evaluations of teaching. Instructional Science, 28(5-6), 439-468.
39. Shulman, L. S. (1999). Taking learning seriously. Change, 31(4), 10-17. doi: 10. 1080/00091389909602695.
40. Swinglehurst, D., Russell, J., & Greenhalgh, T. (2008). Peer observation of teaching in the online environment: an action research approach. Journal of Computer Assisted Learning, 24(5), 383-393. doi: 10. 1111/j. 1365-2729. 2007. 00274. x.
41. Tinto, V. (1997). Classrooms as communities: exploring the educational character of student persistence. Journal of Higher Education, 68(6), 599-623. doi: 10. 2307/2959965.
42. Wilson, M. (2005). Constructing measures: an item response modeling approach. Mahwah: Lawrence Erlbaum Associates.
43. Wood, L. N., & Harding, A. (2007). Can you show you are a good lecturer? International Journal of Mathematical Education in Science and Technology, 38(7), 939-947.
44. Yon, M., Burnap, C., & Kohut, G. (2002). Evidence of effective teaching: Perceptions of peer reviewers. College Teaching, 50(3), 104-110. doi: 10. 1080/87567550209595887.