Introducing a scalable peer feedback system for learning teams

Assessment and Evaluation in Higher Education

Volumn 44, Issue 6, 2019, Pages 848-862

  O’Neill, Thomas ^a   Larson, Nicole ^a   Smith, Julia ^b   Donia, Magda ^c   Deng, Connie ^a   Rosehart, William ^a   Brennan, Robert ^a  

^a UNIVERSITY OF CALGARY   (Canada)

^b ESSEC BUSINESS SCHOOL   (France)

^c UNIVERSITY OF OTTAWA   (Canada)

Author keywords

Peer feedback; team based learning; teamwork

Indexed keywords

EID: 85057227964     PISSN: 02602938     EISSN: 1469297X     Source Type: Journal    
DOI: 10.1080/02602938.2018.1526256     Document Type: Article

References (57)

1. Atwater, L. E., and F. J., Yammarino. 1992. “Does Self-Other Agreement on Leadership Perceptions Moderate the Validity of Leadership and Performance Predictions?” Personnel Psychology 45 (1):141–64.
2. Atwater, L. E., and F. J., Yammarino. 1997. “Self-Other Rating Agreement: A Review and Model.” Research in Personnel and Human Resources Management 15:121–74.
3. Bailey, C., and C., Fletcher. 2002. “The Impact of Multiple Source Feedback on Management Development: Findings from a Longitudinal Study.” Journal of Organizational Behavior 23 (7):853–67.
4. Bamberger, P. A. 2007. “Competitive Appraising: A Social Dilemma Perspective on the Conditions in Which Multi-Round Peer Evaluation May Result in Counter-Productive Team Dynamics.” Human Resource Management Review 17 (1):1–18.
5. Bliese, P. D. 2000. “Within-Group Agreement, Non-Independence, and Reliability: Implications for Data Aggregation and Analysis.” In Multilevel Theory, Research, and Methods in Organizations: Foundations, Extensions, and New Directions, edited by Katherine J., Klein and Steve, Kozlowski, 349–81. San Francisco, CA: Jossey-Bass.
6. Britton, E., N., Simper, A., Leger, and J., Stephenson. 2017. “Assessing Teamwork in Undergraduate Education: A Measurement Tool to Evaluate Individual Teamwork Skills.” Assessment and Evaluation in Higher Education 42 (3):378–97.
7. Brutus, S., and M. B. L., Donia. 2010. “Improving the Effectiveness of Students in Groups with a Centralized Peer Evaluation System.” Academy of Management Learning & Education 9 (4):652–62.
8. Butler, D. L., and P. H., Winne. 1995. “Feedback and Self-Regulated Learning: A Theoretical Synthesis.” Review of Educational Research 65 (3):245–81.
9. DeNisi, A. S., and A. N., Kluger. 2000. “Feedback Effectiveness: Can 360-Degree Appraisals Be Improved?” Academy of Management Perspectives 14 (1):129–39.
10. Dierdorff, E. C., and E. A., Surface. 2007. “Assessing Training Needs: Do Work Experience and Capability Matter?” Human Performance 21 (1):28–48.
11. Donia, M. B. L., T. A., O'Neill, and S., Brutus. 2018. “The Longitudinal Effects of Peer Feedback in the Development and Transfer of Student Teamwork Skills.” Learning and Individual Differences 61:87–98.
12. Druskat, V. U., and S. B., Wolff. 1999. “Effects and Timing of Developmental Peer Appraisals in Self-Managing Work Groups.” The Journal of Applied Psychology 84 (1):58–74.
13. Dunning, D., C., Heath, and J. M., Suls. 2004. “Flawed Self-Assessment: Implications for Health, Education, and the Workplace.” Psychological Science in the Public Interest 5 (3):69–106.
14. Eddy, E. R., S. I., Tannenbaum, and J. E., Mathieu. 2013. “Helping Teams to Help Themselves: Comparing Two Team-Lead Debriefing Methods.” Personnel Psychology 66 (4):975–1008.
15. Ellis, A. P. J., B. S., Bell, R. E., Ployhart, J. R., Hollenbeck, and D. R., Ilgen. 2005. “An Evaluation of Generic Teamwork Skills Training with Action Teams: Effects on Cognitive and Skill-Based Outcomes.” Personnel Psychology 58 (3):641–72.
16. Fellenz, M. R. 2006. “Toward Fairness in Assessing Student Groupwork: A Protocol for Peer Evaluation of Individual Contributions.” Journal of Management Education 30 (4):570–91.
17. Festinger, L. 1954. “A Theory of Social Comparison Processes.” Human Relations 7 (2):117–40.
18. Fiore, S. M., A., Graesser, and S., Greiff. 2018. “ Collaborative Problem Solving Education for the 21st Century Workforce.” Nature Human Behaviour 2 (6):367.
19. Fleenor, J. W., J. W., Smither, L. E., Atwater, P. W., Braddy, and R. E., Sturm. 2010. “Self-Other Rating Agreement in Leadership: A Review.” The Leadership Quarterly 21 (6):1005–34.
20. Gabelica, C., P., Van den Bossche, M., Segers, and W., Gijselaers. 2012. “Feedback, a Powerful Lever in Teams: A Review.” Educational Research Review 7 (2):123–44.
21. Gallup Organization. 2010. “Employers’ Perceptions of Graduate Employability.” Flash Eurobarometer: 1–20 http://ec.europa.eu/public_opinion/flash/fl_304_en.pdf
22. Goffin, R. D., and D. W., Anderson. 2007. “The Self-Rater's Personality and Self-Other Disagreement in Multi-Source Performance Ratings: Is Disagreement Healthy?” Journal of Managerial Psychology 22 (3):271–89.
23. Greguras, G. J., C., Robie, and M. P., Born. 2001. “Applying the Social Relations Model to Self and Peer Evaluations.” Journal of Management Development 20 (6):508–25.
24. Greguras, G. J., C., Robie, M. P., Born, and R. J., Koenigs. 2007. “A Social Relations Analysis of Team Performance Ratings.” International Journal of Selection and Assessment 15 (4):434–48.
25. Hackman, J. R. 1987. “The Design of Work Teams.” In Handbook of Organizational Behavior, edited by Jay W., Lorsch, 315–42. Boston: Harvard Business School Press.
26. Hattie, J., and H., Timperley. 2007. “The Power of Feedback.” Review of Educational Research 77 (1):81–112.
27. Jamieson, M. V., and J. M., Shaw. 2018. July. CATME or ITP Metrics? Which One Should I Use for Design Team Development and Assessment? Paper presented at the annual meeting of the American Society for Engineering Education, Salt Lake City, UT.
28. Kemery, E. R., and L. T., Stickney. 2014. “A Multifaceted Approach to Teamwork Assessment in an Undergraduate Business Program.” Journal of Management Education 38:462–79.
29. Kozlowski, S. W., and K., Hattrup. 1992. “A Disagreement about within-Group Agreement: Disentangling Issues of Consistency versus Consensus.” Journal of Applied Psychology 77 (2):161–7.
30. LeBreton, J. M., J. R., Burgess, R. B., Kaiser, E. K., Atchley, and L. R., James. 2003. “The Restriction of Variance Hypothesis and Interrater Reliability and Agreement: Are Ratings from Multiple Sources Really Dissimilar?” Organizational Research Methods 6 (1):80–128.
31. Loignon, A. C., D. J., Woehr, J. S., Thomas, M. L., Loughry, M. W., Ohland, and D. M., Ferguson. 2017. “Facilitating Peer Evaluation in Team Contexts: The Impact of Frame-of-Reference Rater Training.” Academy of Management Learning & Education 16 (4):562–78.
32. London, M., J. W., Smither, and D. J., Adsit. 1997. “Accountability: The Achilles’ Heel of Multisource Feedback.” Group & Organization Management 22 (2):162–84.
33. Loughry, M. L., M. W., Ohland, and D. D., Moore. 2007. “Development of a Theory-Based Assessment of Team Member Effectiveness.” Educational and Psychosocial Measurement 67 (3):505–24.
34. National Association of Colleges and Employers. 2012. Job Outlook 2012. Bethlehem, PA: Author.
35. Ohland, M. W., M. L., Loughry, D. J., Woehr, L. G., Bullard, R. M., Felder, C. J., Finelli, R. A., Layton, H. R., Pomeranz, and D. G., Schmucker. 2012. “The Comprehensive Assessment of Team Member Effectiveness: Development of a Behaviorally Anchored Rating Scale for Self- and Peer Evaluation.” Academy of Management Learning & Education 11 (4):609–30.
36. Okhuysen, G. A., and M. J., Waller. 2002. “Focusing on Midpoint Transitions: An Analysis of Boundary Conditions.” Academy of Management Journal 45 (5):1056–65.
37. Olson, J. M., R. D., Goffin, and G. A., Haynes. 2007. “Relative versus Absolute Measures of Explicit Attitudes: Implications for Predicting Diverse Attitude-Relevant Criteria.” Journal of Personality and Social Psychology 93 (6):907–26.
38. O'Neill, T. A., A., Deacon, K., Gibbard, N., Larson, G., Hoffart, J., Smith, and B. L. M., Donia. 2018. “Team Dynamics Feedback for Post-Secondary Student Learning Teams.” Assessment and Evaluation in Higher Education 43 (4):571–85.
39. O'Neill, T. A., R. D., Goffin, and I. R., Gellatly. 2012. “The Use of Random Coefficient Modeling for Understanding and Predicting Job Performance Ratings: An Application with Field Data.” Organizational Research Methods 15 (3):436–62.
40. O'Neill, T. A., G. C., Hoffart, M. J. W., McLarnon, H. J., Woodley, M., Eggermont, W., Rosehart, and R., Brennan. 2017. “Constructive Controversy and Reflexivity Training Promotes Effective Conflict Profiles and Outcomes in Student Learning Teams.” Academy of Management Learning & Education 16 (2):257–76.
41. O'Neill, T. A., and E., Salas. 2018. “Creating High Performance Teamwork in Organizations.” Human Resource Management Review 28 (4):325–31.
42. O'Neill, T. A., and P., Steel. 2018. “Weighted Composites of Personality Facets: An Examination of Unit, Rational, and Mechanical Weights.” Journal of Research in Personality 73:1–11.
43. Passow, H. J. 2012. “Which ABET Competencies Do Engineering Graduates Find Most Important in Their Work?” Journal of Engineering Education 101 (1):95–118.
44. Passow, H. J., and C. H., Passow. 2017. “What Competencies Should Undergraduate Engineering Programs Emphasize? A Systematic Review.” Journal of Engineering Education 106 (3):475–526.
45. Paunonen, S. V., and T. A., O'Neill. 2010. “Self Reports, Peer-Ratings, and Construct Validity.” European Journal of Personality 24 (3):189–206.
46. Quinton, S., and T., Smallbone. 2010. “Feeding Forward: Using Feedback to Promote Student Reflection and Learning–a Teaching Model.” Innovations in Education and Teaching International 47 (1):125–35.
47. Saavedra, R., and S. K., Kwun. 1993. “Peer Evaluation in Self-Managing Work Groups.” Journal of Applied Psychology 78 (3):450–62.
48. Scullen, S. E., M. K., Mount, and M., Goff. 2000. “Understanding the Latent Structure of Job Performance Ratings.” The Journal of Applied Psychology 85 (6):956–70.
49. Shafi, A. A., J., Hatley, T., Middleton, R., Millican, and S., Templeton. 2018. “The Role of Assessment Feedback in Developing Academic Buoyancy.” Assessment & Evaluation in Higher Education 43 (3):415–27.
50. Shrout, P. E., and J. L., Fleiss. 1979. “Intraclass Correlations: Uses in Assessing Rater Reliability.” Psychological Bulletin 86 (2):420–8.
51. Shuffler, M. L., D., DiazGranados, and E., Salas. 2011. “There’s a Science for That: Team Development Interventions in Organizations.” Current Directions in Psychological Science 20 (6):365–72.
52. Steel, P., and C. J., Konig. 2006. “Integrating Theories of Motivation.” Academy of Management Review 31 (4):889–913.
53. Stevens, M. J., and M. A., Campion. 1994. “The Knowledge, Skill, and Ability Requirements for Teamwork: Implications for Human Resource Management.” Journal of Management 20 (2):503–30.
54. Taggar, S., and T. C., Brown. 2001. “Problem-Solving Team Behaviors: Development and Validation of Bos and a Hierarchical Factor Structure.” Small Group Research 32 (6):698–726.
55. Taras, V., P., Steel, T. A., O'Neill, W., Tullar, and M., McLarnon. 2016. “Free-Riding in Global Virtual Teams: An Experimental Study of Antecedents and Strategies to Minimize the Problem.” Paper presented at the annual meeting of Academy of International Business, New Orleans, June 27–30.
56. Viswesvaran, C., D. S., Ones, and F. L., Schmidt. 1996. “Comparative Analysis of the Reliability of Job Performance Ratings.” Journal of Applied Psychology 81 (5):557–74.
57. Wageman, R. 1995. “Interdependence and Group Effectiveness.” Administrative Science Quarterly 40 (1):145–80.