The nature of feedback: how peer feedback features affect students' implementation rate and quality of revisions

Journal of Educational Psychology

Volumn 108, Issue 8, 2016, Pages 1098-1120

  Patchan, Melissa M ^a   Schunn, Christian D ^a   Correnti, Richard J ^a  

^a UNIVERSITY OF PITTSBURGH   (United States)

Author keywords

Feedback implementation; Peer assessment; Peer feedback; Revision quality; Writing

Indexed keywords

EID: 84962629122     PISSN: 00220663     EISSN: 19392176     Source Type: Journal    
DOI: 10.1037/edu0000103     Document Type: Article

References (89)

1. AbuSeileek, A., & Abualsha'r, A. (2014). Using peer computer-mediated corrective feedback to support EFL learner's writing. Language Learning & Technology, 18, 76-95.
2. Agius, N. M., & Wilkinson, A. (2014). Students' and teachers' views of written feedback at undergraduate level: A literature review. Nurse Education Today, 34, 552-559. http://dx.doi.org/10.1016/j.nedt.2013.07.005
3. Ariely, D. (2000). Controlling the information flow: Effects on consumers' decision making and preferences. Journal of Consumer Research, 27, 233-248.
4. Azevedo, R., & Bernard, R. M. (1995). A meta-analysis of the effects of feedback in computer-based instruction. Journal of Educational Computing Research, 13, 111-127. http://dx.doi.org/10.2190/9LMD-3U28-3A0G-FTQT
5. Bangert-Drowns, R. L., Kulik, C. C., Kulik, J. A., & Morgan, M. T. (1991). The instructional effect of feedback in test-like events. Review of Educational Research, 61, 218-238. http://dx.doi.org/10.3102/00346543061002213
6. Bazerman, C. (2008, November). U.S. writing across the curriculum and writing in the disciplines in context. In D. Alamargot, J. Bouchand, E. Lambert, V. Millogo & C. Beaudet (Eds.), Proceedings of the International Conference From France to Quebec: Writing in All Its Forms. Poitiers, France.
7. Beaufort, A. (2004). Developmental gains of a history major: A case for building a theory of disciplinary writing expertise. Research in the Teaching of English, 39, 136-185.
8. Bienstock, J. L., Katz, N. T., Cox, S. M., Hueppchen, N., Erickson, S., & Puscheck, E. E., & the Association of Professors of Gynecology and Obstetrics Undergraduate Medical Education Committee. (2007). To the point: Medical education reviews-Providing feedback. American Journal of Obstetrics and Gynecology, 196, 508-513. http://dx.doi.org/10.1016/j.ajog.2006.08.021
9. Bitchener, J. (2012). Written corrective feedback for L2 development: Current knowledge and future research. TESOL Quarterly, 46, 855-860. http://dx.doi.org/10.1002/tesq.62
10. Bitchener, J., Young, S., & Cameron, D. (2005). The effect of different types of corrective feedback on ESL student writing. Journal of Second Language Writing, 14, 191-205. http://dx.doi.org/10.1016/j.jslw.2005.08.001
11. Bransford, J. D., & Johnson, M. K. (1972). Contextual prerequisites for understanding: Some investigations of comprehension and recall. Journal of Verbal Learning and Verbal Behavior, 11, 717-726. http://dx.doi.org/10.1016/S0022-5371(72)80006-9
12. Brown, F. J. (1932). Knowledge of results as an incentive in school room practice. Journal of Educational Psychology, 23, 532-552. http://dx.doi.org/10.1037/h0074392
13. Chi, M. T. H., Bassok, M., Lewis, M. W., Reimann, P., & Glaser, R.(1989). Self-explanations: How students study and use examples in learning to solve problems. Cognitive Science, 13, 145-182. http://dx.doi.org/10.1207/s15516709cog1302_1
14. Chi, M. T. H., & Wylie, R. (2014). The ICAP framework: Linking cognitive engagement to active learning outcomes. Educational Psychologist, 49, 219-243. http://dx.doi.org/10.1080/00461520.2014.965823
15. Cho, K., Schunn, C. D., & Charney, D. (2006). Commenting on writing: Typology and perceived helpfulness of comments from novice peer reviewers and subject matter experts. Written Communication, 23, 260-294. http://dx.doi.org/10.1177/0741088306289261
16. Corbett, A. T., & Anderson, J. R. (1991). Feedback control and learning to program with the CMU Lisp Tutor. Paper presented at the 1991 American Educational Research Association (AERA) Annual Meeting, Chicago, IL.
17. Corbett, A. T., & Anderson, J. R. (2001). Locus of feedback control in computer-based tutoring: Impact on learning rate, achievement, and attitudes. Paper presented at the ACM CHI'2001 Conference on Human Factors in Computing Systems, Seattle, WA.
18. Dohrenwend, A. (2002). Serving up the feedback sandwich. Family Practice Management, 9, 43-46.
19. Duncan, N., Prowse, S., Wakeman, C., & Harrison, R. (2007). "Feedforward": Improving students' use of tutors' comments. Assessment & Evaluation in Higher Education, 32, 271-283. http://dx.doi.org/10.1080/02602930600896498
20. Ellis, R. (2009). A typology of written corrective feedback types. ELT Journal, 63, 97-107. http://dx.doi.org/10.1093/elt/ccn023
21. Ene, E., & Upton, T. A. (2014). Learner uptake of teacher electronic feedback in ESL composition. System, 46, 80-95. http://dx.doi.org/10. 1016/j.system.2014.07.011
22. Esmaeili, F., & Behnam, B. (2014). A study of corrective feedback and learner's uptake in classroom interactions. International Journal of Applied Linguistics & English Literature, 3, 204-212. http://dx.doi.org/10.7575/aiac.ijalel.v.3n.4p.204
23. Ferris, D. R. (1995). Student reactions to teacher response in multiple-draft composition classrooms. TESOL Quarterly, 29, 33-53. http://dx.doi.org/10.2307/3587804
24. Ferris, D. R. (1997). The influence of teacher commentary on student revision. TESOL Quarterly, 31, 315-339. http://dx.doi.org/10.2307/3588049
25. Ferris, D. R., Brown, J., Liu, H., & Stine, M. E. A. (2011). Responding to L2 students in college writing classes: Teacher perspectives. TESOL Quarterly, 45, 207-234. http://dx.doi.org/10.5054/tq.2011.247706
26. Ferris, D. R., Liu, H., Sinha, A., & Senna, M. (2013). Written corrective feedback for individual L2 writers. Journal of Second Language Writing, 22, 307-329. http://dx.doi.org/10.1016/j.jslw.2012.09.009
27. Frey, M. C., & Detterman, D. K. (2004). Scholastic Assessment or g? The relationship between the Scholastic Assessment Test and general cognitive ability. Psychological Science, 15, 373-378. http://dx.doi.org/10.1111/j.0956-7976.2004.00687.x
28. Gielen, S., Peeters, E., Dochy, F., Onghena, P., & Struyven, K. (2010). Improving the effectiveness of peer feedback for learning. Learning and Instruction, 20, 304-315. http://dx.doi.org/10.1016/j.learninstruc.2009.08.007
29. Goodman, J. S., & Wood, R. E. (2004). Feedback specificity, learning opportunities, and learning. Journal of Applied Psychology, 89, 809-821. http://dx.doi.org/10.1037/0021-9010.89.5.809
30. Goodman, J. S., Wood, R. E., & Chen, Z. (2011). Feedback specificity, information processing, and transfer of training. Organizational Behavior and Human Decision Processes, 115, 253-267. http://dx.doi.org/10.1016/j.obhdp.2011.01.001
31. Goodman, J. S., Wood, R. E., & Hendrickx, M. (2004). Feedback specificity, exploration, and learning. Journal of Applied Psychology, 89, 248-262. http://dx.doi.org/10.1037/0021-9010.89.2.248
32. Graham, S., Hebert, M., & Harris, K. R. (2015). Formative assessment and writing: A meta-analysis. The Elementary School Journal, 115, 523-547. http://dx.doi.org/10.1086/681947
33. Grimm, N. (1986). Improving students' responses to their peers' essays. College Composition and Communication, 37, 91-94. http://dx.doi.org/10.2307/357386
34. Hausmann, R. G. M., Vuong, A., Towle, B., Fraundorf, S. H., Murray, R. C., & Connelly, J. (2013). An evaluation of the effectiveness of just-in-time hings. Paper presented at the 16th International Conference on Artificial Intelligence in Education (AIED 2013), Memphis, TN.
35. Hayes, J. R., Flower, L., Schriver, K. A., Stratman, J. F., & Carey, L. (1987). Cognitive processes in revision. In S. Rosenberg (Ed.), Advances in applied psycholinguistics Vol. 2 Reading, writing, and language learning (pp. 176-240). New York: Cambridge University Press.
36. Hesketh, E. A., & Laidlaw, J. M. (2002). Developing the teaching instinct, 1: Feedback. Medical Teacher, 24, 245-248. http://dx.doi.org/10.1080/014215902201409911
37. Ilgen, D. R., Fisher, C. D., & Taylor, M. S. (1979). Consequences of individual feedback on behavior in organization. Journal of Applied Psychology, 64, 349-371. http://dx.doi.org/10.1037/0021-9010.64.4.349
38. Iselin, E. R. (1988). The effects of information load and information diversity on decision quality in a structured decision task. Accounting, Organizations and Society, 13, 147-164. http://dx.doi.org/10.1016/0361-3682(88)90041-4
39. Kellogg, R. T. (1994). The psychology of writing. Oxford, UK: New York: Oxford University Press.
40. Kirkland, M. R., & Saunders, M. A. P. (1991). Maximizing student performance in summary writing: Managing cognitive load. TESOL Quarterly, 25, 105-121. http://dx.doi.org/10.2307/3587030
41. Kluger, A. N., & DeNisi, A. (1996). The effects of feedback interventions on performance: A historical review, a meta-analysis, and a preliminary feedback intervention theory. Psychological Bulletin, 119, 254-284. http://dx.doi.org/10.1037/0033-2909.119.2.254
42. Koedinger, K. R., Corbett, A. T., & Perfetti, C. (2012). The knowledge-learning-instruction framework: Bridging the science-practice chasm to enhance robust student learning. Cognitive Science, 36, 757-798. http://dx.doi.org/10.1111/j.1551-6709.2012.01245.x
43. Kumar, A. N. (2010, June). Error-flagging support for testing and its effect on adaptation. Paper presented at the 10th International Conference on the Intelligent Tutoring Systems (ITS), Pittsburgh, PA.
44. Kumar, A. N. (2012, June). Limiting the number of revisions while providing error-flagging support during tests. Paper presented at the 11th International Conference on the Intelligent Tutoring Systems (ITS), Chania, Crete, Greece.
45. LeBaron, S. W. M., & Jernick, J. (2000). Evaluation as a dynamic process. Family Medicine, 32, 13-14.
46. Li, S. (2010). The effectiveness of corrective feedback in SLA: A meta-analysis. Language Learning, 60, 309-365. http://dx.doi.org/10.1111/j.1467-9922.2010.00561.x
47. Matsumura, L. C., Patthey-Chavez, G. G., Valdes, R., & Garnier, H. (2002). Teacher feedback, writing assignment quality, and third-grade students' revision in lower- and higher-achieving urban schools. The Elementary School Journal, 103, 3-25. http://dx.doi.org/10.1086/499713
48. Molloy, E. K., & Boud, D. (2014). Feedback models for learning, teaching and performance. In J. M. Spector, M. D. Merrill, J. Elen, & M. J. Bishop (Eds.), Handbook of research on educational communications and technology, (4th ed., pp. 413-424). New York, NY: Springer Science+Business Media. http://dx.doi.org/10.1007/978-1-4614-3185-5_33
49. Mory, E. H. (2004). Feedback research revisited. In D. H. Jonassen (Ed.), Handbook ofresearch for educational communications and technology (2nd ed., pp. 745-783). Mahwah, NJ: Erlbaum.
50. Nakamaru, S. (2010). Lexical issues in writing center tutorials with international and U.S.-educated multilingual writers. Journal of Second Language Writing, 19, 95-113. http://dx.doi.org/10.1016/j.jslw.2010.01.001
51. Nelson, M. M., & Schunn, C. D. (2009). The nature of feedback: How different types of peer feedback affect writing performance. Instructional Science, 37, 375-401. http://dx.doi.org/10.1007/s11251-008-9053-x
52. Nguyen, H. V., & Litman, D. (2014, June). Improving peer feedback predictions: The sentence level is right. Paper presented at the 9th Workshop on Innovative Use of NLP for Building Educational Applications (BEA), ACL 2014 Workshop, Baltimore, MD.
53. Nguyen, H. V., Xiong, W., & Litman, D. (2014, June). Classroom evaluation of a scaffolding intervention for improving peer review localization. Paper presented at the 12th International Conference on Intelligent Tutoring Systems (ITS), Honolulu, HI.
54. Nilson, L. B. (2003). Improving student peer feedback. College Teaching, 51, 34-38. http://dx.doi.org/10.1080/87567550309596408
55. Novick, L. R., & Holyoak, K. J. (1991). Mathematical problem solving by analogy. Journal of Experimental Psychology: Learning, Memory, and Cognition, 17, 398-415. http://dx.doi.org/10.1037/0278-7393.17.3398
56. Orrell, J. (2006). Feedback on learning achievement: Rhetoric and reality. Teaching in Higher Education, 11, 441-456. http://dx.doi.org/10.1080/13562510600874235
57. Orsmond, P., & Merry, S. (2011). Feedback alignment: Effective and ineffective links between tutors' and students' understanding of course-work feedback. Assessment & Evaluation in Higher Education, 36, 125-136. http://dx.doi.org/10.1080/02602930903201651
58. Paas, F. G. W. C., Renkl, A., & Sweller, J. (2004). Cognitive load theory: Instructional implications of the interaction between information structures and cognitive architecture. Instructional Science, 32, 1-8. http://dx.doi.org/10.1023/B:TRUC.0000021806.17516.d0
59. Parkes, J., Abercrombie, S., & McCarty, T. (2013). Feedback sandwiches affect perceptions but not performance. Advances in Health Sciences Education: Theory and Practice, 18, 397-407. http://dx.doi.org/10.1007/s10459-012-9377-9
60. Patchan, M. M., Charney, D., & Schunn, C. D. (2009). A validation study of students' end comments: Comparing comments by students, a writing instructor, and a content instructor. Journal of Writing Research, 1, 124-152. http://dx.doi.org/10.17239/jowr-2009.01.02.2
61. Patchan, M. M., Hawk, B., Stevens, C. A., & Schunn, C. D. (2013). The effects of skill diversity on commenting and revisions. Instructional Science, 41, 381-405. http://dx.doi.org/10.1007/s11251-012-9236-3
62. Patchan, M. M., & Schunn, C. D. (2015). Understanding the benefits of providing peer feedback: How students respond to peers' texts of varying quality. Instructional Science, 43, 591-614. http://dx.doi.org/10. 1007/s11251-015-9353-x
63. Patchan, M. M., & Schunn, C. D. (in press).Understanding the benefits of receiving peer feedback for text revision: Relations between author and reviewer ability. Journal of Writing Research.
64. Patchan, M. M., Schunn, C. D., & Clark, R. J. (2011). Writing in natural sciences: Understanding the effects of different types of reviewers on the writing process. Journal of Writing Research, 2, 365-393. http://dx.doi.org/10.17239/jowr-2011.02.03.4
65. Pressey, S. L. (1926). A simple device which gives tests and scores: And teaches. School and Society, 23, 373-376.
66. Pressey, S. L. (1927). A machine for automatic teaching of drill material. School and Society, 25, 549-552.
67. Prins, F. J., Sluijsmans, D. M. A., & Kirschner, P. A. (2006). Feedback for general practitioners in training: Quality, styles, and preferences. Advances in Health Sciences Education: Theory and Practice, 11, 289-303. http://dx.doi.org/10.1007/s10459-005-3250-z
68. Prior, P. (2006). A sociocultural theory of writing. Handbook of writing research, 54-66.
69. Raudenbush, S. W., & Bryk, A. S. (2002). Models for cross-classified random effects. Hierarchical linear models: Applications and data analysis methods (2nd ed., pp. 373-398). Thousand Oaks, CA: Sage.
70. Raudenbush, S. W., Bryk, A. S., & Congdon, R. (2004). HLM 6 for Windows [Computer software]. Skokie, IL: Scientific Software International, Inc.
71. Richardson, B. K. (2004). Feedback. Academic Emergency Medicine, 11, e1-e5. http://dx.doi.org/10.1111/j.1553-2712.2004.tb01913.x
72. Saddler, B., & Andrade, H. (2004). The writing rubric. Educational Leadership, 62, 48-52.
73. Schlecht, K. D. (2008). Feedback. International Anesthesiology Clinics, 46, 67-84. http://dx.doi.org/10.1097/AIA.0b013e31818623f3
74. Sheen, Y. (2010). Differential effects of oral and written corrective feedback in the ESL classroom. Studies in Second Language Acquisition, 32, 203-234. http://dx.doi.org/10.1017/S0272263109990507
75. Shintani, N., Ellis, R., & Suzuki, W. (2014). Effects of written feedback and revision on learners' accuracy in using two English grammatical structures. Language Learning, 64, 103-131. http://dx.doi.org/10.1111/lang.12029
76. Shrout, P. E., & Fleiss, J. L. (1979). Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin, 86, 420-428. http://dx.doi.org/10.1037/0033-2909.86.2.420
77. Shute, V. J. (2008). Focus on formative feedback. Review of Educational Research, 78, 153-189. http://dx.doi.org/10.3102/0034654307313795
78. Sugita, Y. (2006). The impact of teachers' comments types on students' revision. ELT Journal, 60, 34-41. http://dx.doi.org/10.1093/elt/cci079
79. Symonds, P. M., & Chase, D. H. (1929). Practice vs. motivation. Journal of Educational Psychology, 20, 19-35. http://dx.doi.org/10.1037/h0069958
80. Tseng, S.-C., & Tsai, C.-C. (2007). On-line peer assessment and the role of the peer feedback: A study of high school computer course. Computers & Education, 49, 1161-1174. http://dx.doi.org/10.1016/j.compedu.2006.01.007
81. Turnitin. (2013). Closing the gap: What students say about instructor feedback. Retrieved from http://go.turnitin.com/what-students-say-about-teacher-feedback
82. Van Beuningen, C. G., De Jong, N. H., & Kuiken, F. (2012). Evidence on the effectiveness of comprehensive error correction in second language writing. Language Learning, 62, 1-41. http://dx.doi.org/10.1111/j.1467-9922.2011.00674.x
83. van Merrienboer, J. J. G., & Sweller, J. (2005). Cognitive load theory and complex learning: Recent developments and future directions. Educational Psychology Review, 17, 147-177. http://dx.doi.org/10.1007/s10648-005-3951-0
84. Voerman, L., Korthagen, F. A. J., Meijer, P. C., & Simons, R. J. (2014). Feedback revisited: Adding perspectives based on positive psychology. Implications for theory and classroom practice. Teaching and Teacher Education, 43, 91-98. http://dx.doi.org/10.1016/j.tate.2014.06.005
85. Wallace, D. L., & Hayes, J. R. (1991). Redefining revision for freshmen. Research in the Teaching of English, 25, 54-66.
86. Weaver, M. R. (2006). Do students value feedback? Student perceptions of tutors' written responses. Assessment & Evaluation in Higher Education, 31, 379-394. http://dx.doi.org/10.1080/02602930500353061
87. Xiong, W., Litman, D., & Schunn, C. (2012). Natural language processing techniques for researching and improving peer feedback. Journal of Writing Research, 4, 155-176. http://dx.doi.org/10.17239/jowr-2012.04.02.3
88. Zoghi, A., & Nikoopour, J. (2013). The interface of error types, teacher's feedback, and students' uptake. International Journal of Applied Linguistics & English Literature, 3, 54-62. http://dx.doi.org/10.7575/aiac.ijalel.v.3n.1p.54
89. Zohrabi, M., & Rezaie, P. (2012). The role of form-focused feedback on developing students' writing skill. Theory and Practice in Language Studies, 2, 1514-1519. http://dx.doi.org/10.4304/tpls.2.7.1514-1519