The impact of training on the accuracy of teacher-completed direct behavior ratings (DBRs)

Journal of Positive Behavior Interventions

Volumn 12, Issue 1, 2010, Pages 55-63

  LeBel, Teresa J ^a   Kilgus, Stephen P ^a   Briesch, Amy M ^a   Chafouleas, Sandra ^a  

^a UNIVERSITY OF CONNECTICUT   (United States)

Author keywords

Accuracy; Assessment; Behavior; DBRs; Direct behavior ratings; Training

Indexed keywords

EID: 72549090713     PISSN: 10983007     EISSN: None     Source Type: Journal    
DOI: 10.1177/1098300708325265     Document Type: Article

References (39)

1. Angkaw, A.C., Tran, G.Q., & Haaga, D.A.F. (2006). Effects of training intensity on observers' ratings of anxiety, social skills, and alcohol-specific coping skills. Behaviour Research and Therapy, 44, 533-544.
2. Athey, T.R., & McIntyre, R.M. (1987). The effect of rater training on rater accuracy: Levels-of-processing theory and social facilitation theory perspectives. Journal of Applied Psychology, 72, 567-572.
3. Bernardin, H.J. (1978). Effects of rater training on leniency and halo errors in student ratings of instructors. Journal of Applied Psychology, 63, 301-308.
4. Bernardin, H.J., & Buckley, M.R. (1981). Strategies in rater training. Academy of Management Review, 6(2), 205-212.
5. Bernardin, H.J., & Pence, E.C. (1980). Effects of rater training: Creating new response sets and decreasing accuracy. Journal of Applied Psychology, 65, 60-66.
6. Borman, W.C. (1979). Format and training effects on rating accuracy and rater errors. Journal of Applied Psychology, 64, 410-421.
7. Chafouleas, S.M., Christ, T.J., & Riley-Tillman, T. C. (In press). Generalizability and dependability of scaling gradients on direct behavior ratings. Educational and Psychological Measurement.
8. Chafouleas, S.M., Christ, T.J., Riley-Tillman, T.C., Briesch, A.M., & Chanese, J.M. (2007). Generalizability and dependability of daily behavior report cards to measure social behavior of preschoolers. School Psychology Review, 36, 63-80.
9. Chafouleas, S.M., McDougal, J.L., Riley-Tillman, T.C., Panahon, C.J., & Hilt, A.M. (2005). What do daily behavior report cards really measure? An initial comparison of DBRCS with direct observation for off task behavior. Psychology in the Schools, 42, 669-676.
10. Chafouleas, S.M., Riley-Tillman, T.C., & McDougal, J.L. (2002). Good, bad, or in-between: How does the daily behavior report card rate? Psychology in the Schools, 39, 157-169.
11. Chafouleas, S.M., Riley-Tillman, T.C., & Sassu, K.A. (2006). Acceptability and reported use of daily behavior report cards among teachers. Journal of Positive Behavior Interventions, 8, 174-182.
12. Chafouleas, S.M., Riley-Tillman, T.C., & Sugai, G. (2007). School-based behavioral assessment: Informing instruction and intervention. New York: Guilford.
13. Christ, T.J., Chafouleas, S.M., & Riley-Tillman, T.C. (2008). Direct behavior rating (DBRs): How many points on a scale do we need? Unpublished manuscript.
14. Crone, D.A., Horner, R.H., & Hawken, L.S. (2004). Responding to problem behavior in schools: The Behavior Education Program. New York: Guilford.
15. Eckert, T.L., Hintze, J.M., & Shapiro, E.S. (1997). School psychologists' acceptability of behavioral and traditional assessment procedures for externalizing problem behaviors. School Psychology Quarterly, 12, 150-169.
16. Eckert, T.L., & Shapiro, E.S. (1999). Methodological issues in analog acceptability Research: Are teachers' acceptability ratings of assessment methods influenced by experimental design? School Psychology Review, 28, 5-16.
17. Gresham, F.M. (1989). Assessment of treatment integrity in school consultation and prereferral intervention. School Psychology Review, 18, 37-50.
18. Keown-Gerrard, J.L., & Sulsky, L.M. (2001). The effects of task information training and frame-of-reference training with situational constraints on rating accuracy. Human Performance, 14, 305-320.
19. Kratochwill, T.R., Elliott, S.N., & Busse, R.T. (1995). Behavior consultation: A five-year evaluation of consultant and client outcomes. School Psychology Quarterly, 10, 87-117.
20. Lievens, F., & Sanchez, J.I. (2007). Can training improve the quality of inferences made by raters in competency modeling? A quasi-experiment. Journal of Applied Psychology, 92, 812-819.
21. Madle R.A., Neisworth, J.T., & Kurtz P.D. (1980). Biasing of hyper-kinetic behavior ratings by diagnostic reports: Effects of observer training and assessment method. Journal of Learning Disabilities, 13, 35-38.
22. McIntyre, R.M., Smith, D.E., & Hassett, C.E. (1984). Accuracy of performance ratings as affected by rater training and perceived purpose of rating. Journal of Applied Psychology, 69, 147-156.
23. Pelham, W.E., Fabiano, G.A., & Massetti, G.M. (2005). Evidence-based assessment of attention deficit hyperactivity disorder in children and adolescents. Journal of Clinical Child and Adolescent Psychology, 34, 449-476.
24. Richman, G.S., Harrison, K.A., & Summers, J.A. (1995). Assessing and modifying parent responses to their children's noncompliance. Education & Treatment of Children, 18, 105-116.
25. Riley-Tillman, T.C., Chafouleas, S.M., Sassu, K.A., Chanese, J.M., & Glazer, A.D. (2008). Examining agreement between direct behavior ratings (DBRs) and systematic direct observation data for on-task and disruptive behavior. Journal of Positive Behavior Interventions, 10, 136-143.
26. Saudargas, R.A., & Lentz, F.E. (1986). Estimating percent of time and rate via direct observation: A suggested observational procedure and format. School Psychology Review, 15, 36-48.
27. Spool, M.D. (1978). Training programs for observers of behavior: A review. Personnel Psychology, 31, 853-888.
28. Stamoulis, D.T., & Hauenstein, N.M.A. (1993). Rater training and rater accuracy: Training for dimensional accuracy versus training for rating differentiation. Journal of Applied Psychology, 78, 994-1003.
29. Stark, L.J., Powers, S.W., Jelalian, E., Rape, R.N., & Miller, D.L. (1994). Modifying problematic mealtime interactions of children with cystic fibrosis and their parents via behavioral parent training. Journal of Pediatric Psychology, 19, 751-768.
30. Steege, M.W., Davin, T., & Hathaway, M. (2001). Reliability and accuracy of a performance based behavioral recording procedure. School Psychology Review, 30, 252-261.
31. Sterling-Turner, H.E., Watson, T.S., & Moore, J.W. (2002). The effects of direct training and treatment integrity on treatment outcomes in school consultation. School Psychology Quarterly, 17, 47-77.
32. Sterling-Turner, H.E., Watson, T.S., Wildmon, M., Watkins, C., & Little, E. (2001). Investigating the relationship between training type and treatment integrity. School Psychology Quarterly, 16, 56-67.
33. Stevens, J., Quittner, A.L., & Abikoff, H. (1998). Factors influencing elementary school teachers' ratings of ADHD and ODD behaviors. Journal of Clinical Child Psychology, 27, 406-414.
34. Tapp, J. (2004). MOOSES (Multi-Option Observation System for Experimental Studies). Retrieved fromhttp://kc.vanderbilt.edu/mooses/mooses.html.
35. Thornton, G.C., III, & Zorich, S. (1980). Training to improve observer accuracy. Journal of Applied Psychology, 65, 351-354.
36. Uggerslev, K.L., & Sulsky, L.M. (2008). Using frame-of-reference training to understand the implications of rater idiosyncrasy for rating accuracy. Journal of Applied Psychology, 93, 711-719.
37. Volpe, R.J., DiPerna J.C., Hintze J. M., & Shapiro, E.S. (2005). Observing students in classroom settings: A review of seven coding schemes. School Psychology Review, 34, 454-474.
38. Walker, H.M., Ramsey, E., & Gresham, F.M. (2004). Heading off disruptive behavior: How early intervention can reduce defiant behavior-and win back teaching time. American Educator. Retrieved fromhttp://www.aft.org/pubsreports/american_educator/winter03-04/early_intervention.html.
39. Woehr, D.J. (1994). Understanding frame-of-reference training: The impact of training on the recall of performance information. Journal of Applied Psychology, 79, 525-534.