Establishing inter-rater agreement for TIDEE's teamwork and professional development assessments

ASEE Annual Conference and Exposition, Conference Proceedings

Volumn , Issue , 2011, Pages

  Gerlick, Robert ^a   Davis, Denny C ^b   Trevisan, Michael S ^b   Brown, Shane A ^c  

^a PITTSBURG STATE UNIVERSITY   (United States)

^b WASHINGTON STATE UNIVERSITY   (United States)

^c Washington State University   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

CURRICULA; EDUCATION; ENGINEERING EDUCATION; PERSONNEL TRAINING; PROFESSIONAL ASPECTS; RELIABILITY; STUDENTS;

INTER-RATER AGREEMENTS; INTER-RATER RELIABILITIES; PERFORMANCE CRITERION; PROFESSIONAL DEVELOPMENT; SELF-DIRECTED LEARNING; SUMMATIVE ASSESSMENTS; TEACHING AND LEARNING; TECHNICAL PERFORMANCE;

TEACHING;

EID: 85029094074     PISSN: None     EISSN: 21535965     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (14)

1. ABET. (2009). Criteria for accrediting engineering programs. Retrieved November 1, 2009, from http://www.abet.org/forms.shtml
2. Davis, D., Beyerlein, S., Thompson, P., Harrison, O., & Trevisan, M. (2009). Assessments for Capstone Engineering Design [Electronic Version]. Retrieved July 1, 2010 from www.tidee.org .
3. Davis, D., Trevisan, M., Davis, H., Gerlick, R., McCormack, J., Beyerlein, S., Thompson, P., Howe, S., Leiffer, P., Brackin, P., & Khan, J. (2010). Assessing professional skill development in capstone design courses. Proceedings of the Capstone Conference, Boulder, CO.
4. Davis, D., Trevisan, M., Gerlick, R., Davis, H., McCormack, J., Beyerlein, S., Thompson, P., Howe, S., Leiffer, P., & Brackin, P. (2010). Assessing team member citizenship in capstone engineering design courses. International Journal of Engineering Education, 26(4), 1-13.
5. American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (1999). Standards for Educational and Psychological Testing. Washington, DC: American Educational Research Association.
6. Drummond, R., & Jones, D. D. (2006). Assessment Procedures for Counselors and Helping Professionals (6ed.). Upper Saddle River, NJ: Pearson Education.
7. Kubiszyn, T., & Borich, G. (2003). Education Testing and Measurement: Classroom Application and Practice (7 ed.): John Wiley and Sons, Inc.
8. Abedi, j., Baker, E. L., & Herl, H. (1995). Comparing Reliability Indices Obtained by Different Approaches for Performance Assessments. National Center for Research on Evaluation, Standards, and Student Testing (CRESST).
9. Woehr, D. J., & Huffcutt, A. I. (1994). Rater training for performance appraisal: A quantitative review. Journal of Occupational and Organizational Psychology, 67, 189-205.
10. Bernardin, H. J., & Buckley, M. R. (1981). Strategies in rater training. Academy of Management Review, 6(2), 205-212.
11. Lievens, F. (2001). Assessor training strategies and their effects on accuracy, interrater reliability, and discriminant validity. Journal of Applied Psychology, 86(2), 255-264.
12. Stamoulis, D. T., & Hauenstein, N. M. A. (1993). Rater training and rating accuracy: Training for dimensional accuracy versus training for ratee differentiation. Journal of Applied Psychology, 78(6), 994-1003.
13. Borman, W. C. (1979). Format and training effects on rating accuracy and rater errors. Journal of Applied Psychology, 64(4), 410-421.
14. McIntyre, R. M., Smith, D. E., & Hassett, C. E. (1984). Accuracy of performance ratings as affected by rater training and perceived purpose of rating. Journal of Applied Psychology, 69(1), 147-156.