Addressing knowledge deficits in tutoring and the role of teaching experience: Benefits for learning and summative assessment

Journal of Educational Psychology

Volumn 106, Issue 4, 2014, Pages 934-945

  Herppich, Stephanie ^a   Wittwer, Jörg ^a   Nückles, Matthias ^a   Renkl, Alexander ^a  

^a UNIVERSITY OF FREIBURG   (Germany)

Author keywords

Formative assessment; Knowledge deficits; Learning; Summative assessment; Tutoring

Indexed keywords

EID: 84925423299     PISSN: 00220663     EISSN: 19392176     Source Type: Journal    
DOI: 10.1037/a0036076     Document Type: Article

References (61)

1. Anderson, J. R., Corbett, A. T., Koedinger, K. R., & Pelletier, R. (1995). Cognitive tutors: Lessons learned. Journal of the Learning Sciences, 4, 167-207. doi:10.1207/s15327809jls0402_2
2. Bennett, R. E. (2011). Formative assessment: A critical review. Assessment in Education: Principles, Principles, Policy & Practice, 18, 5-25. doi:10.1080/0969594x.2010.513678
3. Bentz, J., & Fuchs, L. (1996). Improving peers' helping behavior to students with learning disabilities during mathematics peer tutoring. Learning Disability Quarterly, 19, 202-215. doi:10.2307/1511207
4. Birenbaum, M., Breuer, K., Cascallar, E., Dochy, F., Dori, Y., Ridgway, J., . . . Nickmans, G. (2006). A learning integrated assessment system. Educational Research Review, 1, 61-67. doi:10.1016/j.edurev.2006.01.001
5. Black, P. (1993). Formative and summative assessment by teachers. Studies in Science Education, 21, 49-97. doi:10.1080/03057269308560014
6. Black, P., & Wiliam, D. (1998). Assessment and classroom learning. Assessment in Education: Principles, Principles, Policy & Practice, 5, 7-74. doi:10.1080/0969595980050102
7. Black, P., & Wiliam, D. (2009). Developing the theory of formative assessment. Educational Assessment, Evaluation and Accountability, 21, 5-31. doi:10.1007/s11092-008-9068-5
8. Bloom, B. S. (1984). The 2 sigma problem: The search for methods of group instruction as effective as one-to-one tutoring. Educational Researcher, 13, 4-16. doi:10.3102/0013189X013006004
9. Bromme, R., Brummernhenrich, B., Becker, B.-M., & Jucks, R. (2012). The effects of politeness-related instruction on medical tutoring. Communication Education, 61, 358 -379. doi:10.1080/03634523.2012.691979
10. Cade, W., Copeland, J., Person, N., & .D'Mello, S. (2008). Dialogue modes in expert tutoring. In B. Woolf, E. Aïmeur, R. Nkambou, & S. Lajoie (Eds.), Intelligent tutoring systems (Vol. 5091, pp. 470-479). Retrieved from http://link.springer.com/
11. Chae, H. M., Kim, J. H., & Glass, M. (2005). Effective behaviors in a comparison between novice and expert algebra tutors. In S. Hettiarachchi & R. Finkbine (Eds.), Proceedings of the Sixteenth Midwest Artificial Intelligence and Cognitive Science Conference. Retrieved from http://www.mglass.org/papers/papers.html
12. Chi, M. T. H. (2009). Active-constructive-interactive: A conceptual framework for differentiating learning activities. Topics in Cognitive Science, 1, 73-105. doi:10.1111/j.1756-8765.2008.01005.x
13. Chi, M. T. H., & Roy, M. (2010). How adaptive is an expert human tutor? In V. Aleven, J. Kay, & J. Mostow (Eds.), Intelligent tutoring systems (Vol. 6094, pp. 401-412).
14. Chi, M. T. H., Roy, M., & Hausmann, R. G. M. (2008). Observing dialogues collaboratively: Insights about human tutoring effectiveness from vicarious learning, Cognitive Science, 32, 301-341. doi:10.1080/03640210701863396
15. Chi, M. T. H., Siler, S. A., & Jeong, H. (2004). Can tutors monitor students' understanding accurately? Cognition and Instruction, 22, 363-387. doi:10.1207/s1532690xci2203_4
16. Chi, M. T. H., Siler, S. A., Jeong, H., Yamauchi, T., & Hausmann, R. (2001). Learning from human tutoring. Cognitive Science, 25, 471-533. doi:10.1207/s15516709cog2504_1
17. Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20, 37- 46. doi:10.1177/001316446002000104
18. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum.
19. Cromley, J. G., & Azevedo, R. (2005). What do reading tutors do? A naturalistic study of more and less experienced tutors in reading. Discourse Processes, 40, 83-113. doi:10.1207/s15326950dp4002_1
20. D'Mello, S., Lehman, B. A., & Person, N. K. (2010). Expert tutors feedback is immediate, direct, and discriminating. Paper presented at the Twenty-Third International Florida Artificial Intelligence Research Society Conference (FLAIRS 2010), Key West, FL. Retrieved from http://www.aaai.org
21. Fleiss, J. L., & Cohen, J. (1973). The equivalence of weighted kappa and the intraclass correlation coefficient as measures of reliability. Educational and Psychological Measurement, 33, 613-619. doi:10.1177/001316447303300309
22. Fuchs, L., Fuchs, D., Hamlett, C., Phillips, N., Karns, K., & Dutka, S. (1997). Enhancing students' helping behavior during peer-mediated instruction with conceptual mathematical explanations. Elementary School Journal, 97, 223-249. doi:10.1086/461863
23. Furtak, E. M., Ruiz-Primo, M. A., Shemwell, J. T., Ayala, C. C., Brandon, P. R., Shavelson, R. J., & Yin, Y. (2008). On the fidelity of implementing embedded formative assessments and its relation to student learning. Applied Measurement in Education, 21, 360 -389. doi:10.1080/08957340802347852
24. Graesser, A. C., D'Mello, S., & Cade, W. L. (2011). Instruction based on tutoring. In R. E. Mayer & P. A. Alexander (Eds.), Handbook of research on learning and instruction (pp. 408-426). New York, NY: Routledge.
25. Graesser, A. C., & Person, N. K. (1994). Question asking during tutoring. American Educational Research Journal, 31, 104-137. doi:10.3102/00028312031001104
26. Graesser, A. C., Person, N. K., & Magliano, J. P. (1995). Collaborative dialogue patterns in naturalistic one-on-one tutoring. Applied Cognitive Psychology, 9, 495-522. doi:10.1002/acp.2350090604
27. Hallgren, K. A. (2012). Computing inter-rater reliability for observational data: An overview and tutorial. Tutorials in Quantitative Methods for Psychology, 8, 23-34.
28. Harlen, W., & James, M. (1997). Assessment and learning: Differences and relationships between formative and summative assessment. Assessment in Education: Principles, Policy & Practice, 4, 365-379. doi:10.1080/0969594970040304
29. Hayes, A. F. (2009). Beyond Baron and Kenny: Statistical mediation analysis in the new millennium. Communication Monographs, 76, 408-420. doi:10.1080/03637750903310360
30. Hayes, A. F. (2012). PROCESS: A versatile computational tool for observed variable mediation, moderation, and conditional process modeling [White paper]. Retrieved from http://www.afhayes.com/public/process2012.pdf
31. Herppich, S., Wittwer, J., Nückles, M., & Renkl, A. (2013a). Benefits for processes cause decrements in outcomes: Training improves tutors' interactivity at the expense of assessment accuracy. In M. Knauff, M. Pauen, N. Sebanz, & I. Wachsmuth (Eds.), Proceedings of the 35th Annual Conference of the Cognitive Science Society (pp. 2530-2535). Retrieved from http://mindmodeling.org/cogsci2013/papers/0458/paper0458.pdf
32. Herppich, S., Wittwer, J., Nückles, M., & Renkl, A. (2013b). Does it make a difference? Investigating the assessment accuracy of teacher tutors and student tutors. Journal of Experimental Education, 81, 242-260. doi: 10.1080/00220973.2012.699900
33. Hoge, R., & Coladarci, T. (1989). Teacher-based judgments of academic achievement. Review of Educational Research, 59, 297-313. doi:10.3102/00346543059003297
34. Katz, S., Allbritton, D., & Connelly, J. (2003). Going beyond the problem given: How human tutors use postsolution discussions to support transfer. International Journal of Artificial Intelligence and Education, 13, 79-116.
35. Kingston, N., & Nash, B. (2011). Formative assessment: A meta-analysis and a call for research. Educational Measurement: Issues and Practice, 30, 28-37. doi:10.1111/j.1745-3992.2011.00220.x
36. McArthur, D., Stasz, C., & Zmuidzinas, M. (1990). Tutoring techniques in algebra. Cognition and Instruction, 7, 197-244. doi:10.1207/s1532690xci0703_2
37. Merrill, D. C., Reiser, B. J., Ranney, M., & Trafton, J. G. (1992). Effective tutoring techniques: A comparison of human tutors and intelligent tutoring systems. Journal of the Learning Sciences, 2, 277-305. doi:10.1207/s15327809jls0203_2
38. Pelaez, N. J., Boyd, D. D., Rojas, J. B., & Hoover, M. A. (2005). Prevalence of blood circulation misconceptions among prospective elementary teachers. Advances in Physiology Education, 29, 172-181. doi:10.1152/advan.00022.2004
39. Perie, M., Marion, S., & Gong, B. (2009). Moving toward a comprehensive assessment system: A framework for considering interim assessments. Educational Measurement: Issues and Practice, 28, 5-13. doi:10.1111/j.1745-3992.2009.00149.x
40. Preacher, K. J., & Hayes, A. F. (2008). Asymptotic and resampling strategies for assessing and comparing indirect effects in multiple mediator models. Behavior Research Methods, 40, 879-891. doi:10.3758/BRM.40.3.879
41. Preacher, K. J., & Kelley, K. (2011). Effect size measures for mediation models: Quantitative strategies for communicating indirect effects. Psychological Methods, 16, 93-115. doi:10.1037/a0022658
42. Putnam, R. T. (1987). Structuring and adjusting content for students: A study of live and simulated tutoring of addition. American Educational Research Journal, 24, 13-48. doi:10.3102/00028312024001013
43. Ready, D. D., & Wright, D. L. (2011). Accuracy and inaccuracy in teachers' perceptions of young children's cognitive abilities: The role of child background and classroom context. American Educational Research Journal, 48, 335-360. doi:10.3102/0002831210374874
44. Rimmele, R. (2013). Videograph (Version 4.2) [Computer program]. Kiel, Germany: IPN.
45. Roscoe, R. D., & Chi, M. T. H. (2007). Understanding tutor learning: Knowledge-building and knowledge-telling in peer tutors' explanations and questions. Review of Educational Research, 77, 534-574. doi: 10.3102/0034654307309920
46. Ruiz-Primo, M. A., & Furtak, E. M. (2007). Exploring teachers' informal formative assessment practices and students' understanding in the context of scientific inquiry. Journal of Research in Science Teaching, 44, 57-84. doi:10.1002/tea.20163
47. Shavelson, R. J. (2006). On the integration of formative assessment in teaching and learning with implications for teacher education. Paper prepared for the Stanford Education Assessment Laboratory and the University of Hawaii Curriculum Research and Development Group. Retrieved from http://www.stanford.edu/dept/SUSE/SEAL/Reports_Papers/Paper.htm
48. Shavelson, R. J., Young, D. B., Ayala, C. C., Brandon, P. R., Furtak, E. M., Ruiz-Primo, M. A., . . . Yin, Y. (2008). On the impact of curriculumembedded formative assessment on learning: A collaboration between curriculum and assessment developers. Applied Measurement in Education, 21, 295-314. doi:10.1080/08957340802347647
49. Shepard, L. A. (2005). Formative assessment: Caveat emptor. Paper presented at the ETS Invitational Conference 2005, New York, NY. Retrieved from http://www.aacompcenter.org/cs/aacc/view/rs/27119
50. Shepard, L. A., Hammerness, K., Darling-Hammond, L., & Rust, F. (with Baratz-Snowden, J., Gordon, E., Gutierrez, C., & Pacheco, A.). (2005). Assessment. In L. Darling-Hammond & J. Bransford (Eds.), Preparing teachers for a changing world: What teachers should learn and be able to do (pp. 275-326). San Francisco, CA: Jossey-Bass.
51. Snow, R. E., & Swanson, J. (1992). Instructional psychology: Aptitude, adaptation, and assessment. Annual Review of Psychology, 43, 583-626. doi:10.1146/annurev.ps.43.020192.003055
52. Stiggins, R. J. (2006). Assessment for learning: A key to motivation and achievement. Edge, 2, 3-19.
53. Südkamp, A., Kaiser, J., & Möller, J. (2012). Accuracy of teachers' judgments of students' academic achievement: A meta-analysis. Journal of Educational Psychology, 104, 743-762. doi:10.1037/a0027627
54. Towle, A. (1989). Modern biology. New York, NY: Holt, Rinehart and Winston.
55. van de Pol, J., Volman, M., & Beishuizen, J. (2010). Scaffolding in teacher-student interaction: A decade of research. Educational Psychology Review, 22, 271-296. doi:10.1007/s10648-010-9127-6
56. VanLehn, K. (2011). The relative effectiveness of human tutoring, intelligent tutoring systems, and other tutoring systems. Educational Psychologist, 46, 197-221. doi:10.1080/00461520.2011.611369
57. VanLehn, K., Siler, S., Murray, C., Yamauchi, T., & Baggett, W. B. (2003). Why do only some events cause learning during human tutoring? Cognition and Instruction, 21, 209-249. doi:10.1207/S1532690XCI2103_01
58. Wiliam, D., Lee, C., Harrison, C., & Black, P. J. (2004). Teachers developing assessment for learning: Impact on student achievement. Assessmentin Education: Principles, Policy & Practice, 11, 49-65. doi:10.1080/0969594042000208994
59. Wirtz, M., & Caspar, F. (2002). Beurteilerübereinstimmung und Beurteilerreliabilität [Rater agreement and rater reliability]. Göttingen, Germany: Hogrefe.
60. Wittwer, J., Nückles, M., Landmann, N., & Renkl, A. (2010). Can tutors be supported in giving effective explanations? Journal of Educational Psychology, 102, 74-89. doi:10.1037/a0016727
61. Wittwer, J., & Renkl, A. (2008). Why instructional explanations often do not work: A framework for understanding the effectiveness of instructional explanations. Educational Psychologist, 43, 49-64. doi:10.1080/00461520701756420