Designing a peer evaluation instrument that is simple, reliable, and valid

ASEE Annual Conference Proceedings

Volumn , Issue , 2004, Pages 3065-3075

  Ohland, Matthew W ^a   Loughry, Misty L ^b   Carter, Rufus L ^c   Yuhasz, Amy G ^a  

^a Clemson University   (United States)

^b CLEMSON UNIVERSITY   (United States)

^c MARYMOUNT UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ASSIGNMENTS; COMMUNICATION SKILLS; DESIGN PROJECTS; PEER EVALUATION;

ENGINEERING RESEARCH; PROJECT MANAGEMENT; RELIABILITY; STUDENTS; TEACHING;

ENGINEERING EDUCATION;

EID: 5444238596     PISSN: 01901052     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (47)

1. Criteria for Accrediting Engineering Programs. Published by The Accreditation Board for Engineering and Technology (ABET), Baltimore, Maryland. Last accessed on November 5, 2001; http://www.abet.org/images/eac_criteria_b.pdf (2000).
2. Johnson, D.W., R.T. Johnson, and K.A. Smith, Active learning: Cooperation in the college classroom, Edina, MN: Interaction Book Co., (1998).
3. Millis, B.J. and P.G. Cottell, Jr., Cooperative Learning for Higher Education Faculty, Phoenix, American Council on Education/Oryx Press, p. 194 (1998)
4. Brown, R.W., "Autorating: Getting individual marks from team marks and enhancing teamwork," Proc. Frontiers in Education Conference. IEEE/ASEE, Pittsburgh, November (1995).
5. Kaufman, D.B., R.M. Felder, and H. Fuller, "Peer ratings in cooperative learning teams," Proc. Amer. Soc. Eng. Ed., ASEE, Charlotte, June (1999).
6. Kaufman, D.B., R.M. Felder, and H. Fuller, "Accounting for individual effort in cooperative learning teams," J. Engineering Education 89(2), 133-140 (2000).
7. Layton, R. A. and M.W. Ohland, "Peer evaluations in teams of predominantly minority students," Proc. Amer. Soc. Eng. Ed., Session 2330, ASEE, Washington, DC, (2000).
8. Ohland, M.W., and R.A. Layton, "Comparing the reliability of two peer evaluation instruments," Proc. Amer. Soc. Eng. Ed., St. Louis, MO, session 3530, June (2000).
9. Layton, R.A., and M.W. Ohland, "Peer evaluations revisited: Focus on teamwork, not ability," Proc. Amer. Soc. Eng. Ed., Albuquerque, NM, June (2001).
10. Ohland, M.W., and C.J. Finelli, "Peer evaluation in a mandatory cooperative education environment," Proc. Amer. Soc. Eng. Ed., Albuquerque, NM, June (2001).
11. Finelli, C.J., "Assessing improvement in students' team skills and using a learning style inventory to increase it," Proc. Frontiers in Education Conference, IEEE/ASEE, Reno, NV, October (2001).
12. Mikic, B. and D. Grasso, "Socially-Relevant Design; The TOYtech Project at Smith College," J Engr Ed 91(3), July 2002, pp. 319-326.
13. McGourty, J. and K. De Meuse, The Team Developer: An assessment and skill building program, J. Wiley and Company, New York (2000).
14. McGourty, J., C. Sebastian, and W. Swart, "Development of a comprehensive assessment program in engineering education." J. Engineering Education 87(4), 355-361 (1998).
15. McGourty, J., "Using multisource feedback in the classroom: A computer-based approach," IEEE Trans. Ed., 43(2), 120-124, (2000).
16. Kubiszyn, T. and G. Borich, Educational testing and measurement: Classroom application and practice, 4th Ed., Harper Collins, New York, (1993).
17. Thompson, R.S., "Relative validity of peer and self-evaluations in self-directed interdependent work teams," Proc. Frontiers in Education Conference, IEEE/ASEE, Reno, NV, October (2001).
18. Van Duzer, E. and F. McMartin, "Methods to improve the validity and sensitivity of a self/peer assessment instrument," IEEE Trans. Ed., 43(2), 153-158, May (2000).
19. Version 1, received from Pat Meade at Frontiers in Education Conference, Reno, NV, October (2001)
20. Seat, E., and T.P. McAnear, "Administering, scoring and debriefing Team Developer," Proc. Frontiers in Education Conference, IEEE/ASEE, Reno, NV, October (2001).
21. Crocker, L., and J. Algina, Introduction to classical and modern test theory, Holt, Rinehart and Winston; Inc., Chicago, p. 143, 157ff, (1986).
22. McCaulley, M. H., Godleski, E. S., Yokomoto, C. F., Harrisberger, L., and Sloan, E. D., "Applications of psychological type in engineering education," Engineering Education 394 (Feb. 1983).
23. Hammond, H. P., "Report of the Committee on Engineering Education After the War," J. Engineering Ed. 34, 1944, 589-614.
24. Hammond, H. P., "Report of the Committee on Aims and Scope of Engineering Curricula," J. Engineering Ed. 30, 1940, 555-566.
25. Haddad, Jerrier A., "Engineering Education and Practice in the United States: Foundations of Our Techno-Economic Future," Report of the Committee on the Education and Utilization of the Engineer, National Research Council, Washington, D.C.: National Academy Press, 1985.
26. Grintner, L. E., "Report of the Committee on Evaluation of Engineering Education," J. Engineering Ed. 44, September 1955, 26-60.
27. Wickenden, W. E., "Report of the Investigation of Engineering Education," Urbana, IL, Society for the Promotion of Engineering Education, 1930.
28. McGrath, J. E. Social psychology: A brief introduction. New York: Holt, Rinehart and Winston (1964).
29. Alchian, A. A., and Demsetz, H. "Production, information costs and economic organization." American Economic Review, 62, 777-795 (1972).
30. Stevens, M. J., and Campion, M. A. "Staffing work teams: Development and validation of a selection test for teamwork settings." Journal of Management, 25, 207-228 (1999).
31. Howard, A. "A framework for work change." In A. Howard (Ed.), The changing nature of work (pp. 3-44). San Francisco: Jossey-Bass (1995).
32. Ilgen, D. R., and Pulakos, E. D. (Eds.). The changing nature of performance: Implications for staffing, motivation, and development. San Francisco: Jossey Bass (1999).
33. nd ed., Vol. 3, pp. 199-267). Palo Alto, CA: Consulting Psychologists Press (1992).
34. nd ed., Vol. 3, pp. 269-313). Palo Alto, CA: Consulting Psychologists Press (1992).
35. Guzzo, A., and E. Salas (Eds.) Team effectiveness and decision making in organizations. San Francisco: Jossey-Bass (1995).
36. Hedge, J. W., Bruskiewicz, K. T., Logan, K. K., Hanson, M. A., and Buck, D. Crew resource management team and individual job analysis and rating scale development for air force tanker crews (Technical Report No. 336). Minneapolis, MN: Personnel Decisions Research Institutes, Inc. (1999).
37. Barrick, M. R., and M. K. Mount, "The big five personality dimensions and job performance: A meta-analysis," Personnel Psychology 44, 1-27 (1991).
38. Salgado, J. F. "The five factor model of personality and job performance in the European community, Journal of Applied Psychology, 82(1), 30-43.
39. Motowidlo, S. J., W. C. Borman, and M. J. Schmit, "A theory of individual differences in task and contextual performance," Human Performance 10(2), 71-83.
40. Smith, C. A., D. W. Organ, and J. P. Near, "Organizational Citizenship Behavior: Its nature and antecedents," Journal of Applied Psychology 68(4), 653-663.
41. LePine, J. A., A. Erez, and D. E. Johnson, "The nature and dimensionality of organizational citizenship behavior: a critical review and meta-analysis," Journal of Applied Psychology, 87(1), 52-65.
42. Church, A. H., and Bracken, D. W. "Advancing the state of the art of 360-degree feedback: Guest editors' comments on the research and practice of multirater assessment methods." Group and Organization Management, 22, 149-161 (1997).
43. Fedor, D. B., Bettenhausen, K. L., and Davis, W. "Peer reviews: Employees' dual roles as raters and recipients." Group and Organization Management, 24, 92-120 (1999).
44. Millis, B.J., and P.G. Cottell, Jr., Cooperative learning for higher education faculty, American Council on Education Oryx Press Series on Higher Education (1997).
45. Felder, R.M., and L.K. Silverman, "Learning and teaching styles in engineering education," Engineering Education, 674-681, April (1988).
46. Wankat, P., "An analysis of the articles in the Journal of Engineering Education," J. Engr. Ed. 88(1), 37-42 (1999).
47. http://diamond.gem.valpo.edu/~harvey/teams/index.html