The efficiency of behavior rating scales to assess inattentive-overactive and oppositional-defiant behaviors: Applying generalizability theory to streamline assessment

Journal of School Psychology

Volumn 49, Issue 1, 2011, Pages 131-155

  Volpe, Robert J ^a   Briesch, Amy M ^a   Gadow, Kenneth D ^b  

^a NORTHEASTERN UNIVERSITY   (United States)

^b STONY BROOK UNIVERSITY   (United States)

Author keywords

Behavioral assessment; Brief rating scales; Dependability; Progress monitoring; Rating scales

Indexed keywords

CENTRAL STIMULANT AGENT; METHYLPHENIDATE;

ADOLESCENT; ARTICLE; ATTENTION DEFICIT DISORDER; BEHAVIOR DISORDER; CHILD; FEMALE; HUMAN; MALE; PSYCHOLOGICAL ASPECT; PSYCHOLOGICAL RATING SCALE; REPRODUCIBILITY; STANDARD;

ADOLESCENT; ATTENTION DEFICIT AND DISRUPTIVE BEHAVIOR DISORDERS; ATTENTION DEFICIT DISORDER WITH HYPERACTIVITY; CENTRAL NERVOUS SYSTEM STIMULANTS; CHILD; CHILD BEHAVIOR DISORDERS; FEMALE; HUMANS; MALE; METHYLPHENIDATE; PSYCHIATRIC STATUS RATING SCALES; REPRODUCIBILITY OF RESULTS;

EID: 78650840559     PISSN: 00224405     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.jsp.2010.09.005     Document Type: Article

References (75)

1. Achenbach T.M., McConaughy S.H., Howell C.T. Child/adolescent behavioral and emotional problems: Implications of cross-informant correlations for situational specificity. Psychological Bulletin 1987, 101:213-232.
2. American Psychiatric Association Diagnostic and statistical manual of mental disorders 1987, Author, Washington, DC. 3rd ed.
3. American Psychiatric Association Diagnostic and statistical manual of mental disorders 2004, Author, Washington, DC. 4th ed.
4. Atkins M.S., Pelham W.E., Licht M.H. The differential validity of teacher ratings of inattention/overactivity and aggression. Journal of Abnormal Child Psychology 1989, 17:423-435.
5. Barkley R.A. Attention deficit hyperactivity disorder: A handbook for diagnosis and treatment 2006, Guilford Press, New York. 3rd ed.
6. Bergeron R., Floyd R.G., McCormack A.C., Farmer W.L. The generalizability of externalizing behavior composites and subscale scores across time, rater, and instrument. School Psychology Review 2008, 37:91-108.
7. Brennan R.L. Generalizability theory 2001, Springer, New York.
8. Briesch A.M., Volpe R.J. Selecting progress monitoring tools for evaluating social behavior. School Psychology Forum 2007, 1:59-74.
9. Cates G.L., Skinner C.H., Watson S.T., Meadows T.J., Weaver A., Jackson B. Instructional effectiveness and instructional efficiency as considerations for data-based decision making: An evaluation of interspersing procedures. School Psychology Review 2003, 32:601-616.
10. Chafouleas S.M., Briesch A.M., Riley-Tillman T.C., Christ T.J., Kilgus S.J. An investigation of the generalizability and dependability of Direct Behavior Rating Single-Item Scales (DBR-SIS) to measure academic engagement and disruptive behavior of middle school students. Journal of School Psychology 2010, 48:219-246.
11. Chafouleas S.M., Christ T.J., Riley-Tillman T.C., Briesch A.M., Chanese J.A.M. Generalizability and dependability of direct behavior ratings to assess social behavior of preschoolers. School Psychology Review 2007, 36:63-79.
12. Chafouleas, S. M., Volpe, R. J., Gresham, F. M., & Cook, C. R. (in press). School-based behavioral assessment within problem-solving models: Current status and future directions. School Psychology Review.
13. Christ T.C., Johnson-Gros K.N., Hintze J.M. An examination of alternate assessment durations when assessing multiple-skill computational fluency: The generalizability and dependability of curriculum-based outcomes within the context of educational decisions. Psychology in the Schools 2003, 42:615-622.
14. Cone J.D. The behavioral assessment grid (BAG): A conceptual framework and taxonomy. Behavior Therapy 1978, 9:882-888.
15. Cortina J.M. What is coefficient alpha? An examination of theory and applications. Journal of Applied Psychology 1993, 78:98-104.
16. Crick J.E., Brennan R.L. Manual for GENOVA: A generalized analysis of variance system 1983, The American College Testing Program, Iowa City, IA.
17. Cronbach L.J. My current thoughts on coefficient alpha and successor procedures. Educational and Psychological Measurement 2004, 64:391-418.
18. Cronbach L.J., Gleser C.G., Rajaratnam N., Nanda H. The dependability of behavioral measurements 1972, Wiley, New York.
19. Dumenci L., McConaughy S.H., Achenbach T.M. A hierarchical 3-factor model of inattention-hyperactivity-impulsivity derived from the Attention Problems Syndrome of the Teacher's Report Form. School Psychology Review 2004, 33:287-301.
20. DuPaul G.J., Stoner G. ADHD in the schools: Assessment and intervention strategies 2003, Guilford Press, New York. 2nd ed.
21. Elliott S.N., Witt J.C., Galvin G., Peterson R. Acceptability of positive and reductive interventions: Factors that influence teachers' decisions. Journal of School Psychology 1984, 22:353-360.
22. Enders C.K. A primer on maximum likelihood algorithms available for use with missing data. Structural Equation Modeling 2001, 8:128-141.
23. Fabiano G.A., Pelham W.E., Gnagy E.M., Burrows-MacLean L., Coles E.K., Chacko A., et al. The single and combined effects of multiple intensities of behavior modification and multiple intensities of methylphenidate in a classroom setting. School Psychology Review 2007, 36:195-216.
24. Fabiano G.A., Vujnovic R., Pelham W.E., Waschbusch D.A., Massetti G.M., Yu J., Volker M. Enhancing the effectiveness of special education programming for children with ADHD using a daily report card. School Psychology Review 2010, 39:219-239.
25. Gadow K.D., Nolan E.E., Paolicelli L.M., Sprafkin J. A procedure for assessing the effects of methylphenidate on hyperactive children in public school settings. Journal of Clinical Child Psychology 1991, 20:268-276.
26. Gadow K.D., Nolan E.E., Sprafkin J., Sverd J. School observations of children with attention-deficit hyperactivity disorder and comorbid tic disorder: Effects of methylphenidate treatment. Journal of Developmental and Behavioral Pediatrics 1995, 16:167-176.
27. Gadow K.D., Nolan E.E., Sverd J., Sprafkin J., Schneider J. Methylphenidate in children with oppositional defiant disorder and both co-morbid chronic multiple tic disorder and ADHD. Journal of Child Neurology 2008, 23:981-990.
28. Gadow K.D., Sprafkin J. ADHD symptom checklist - 4 2008 manual 2008, Checkmate Plus, Stony Brook, NY.
29. Gadow K.D., Sverd J., Nolan E.E., Sprafkin J., Schneider J. Immediate-release methylphenidate for ADHD in children with comorbid chronic multiple tic disorder. Journal of the American Academy of Child and Adolescent Psychiatry 2007, 46:840-848.
30. Gresham, F., Cook, C., Collins, T., Dart, E., Rasetshwane, K., Truelson, E., Grant, S. (in press). Developing a change sensitive brief behavior rating scale as a progress monitoring tool for social behavior: An example using the Social Skills Rating System-Teacher form. School Psychology Review.
31. Griffiths A., VanDerHeyden A.M., Skokut M., Lilles E. Progress monitoring in oral reading fluency within the context of RTI. School Psychology Quarterly 2009, 24:13-23.
32. Hintze J.M., Christ T.J., Keller L.A. The generalizability of CBM survey-level mathematics assessments: Just how many samples do we need?. School Psychology Review 2002, 31:514-528.
33. Hintze J.M., Matthews W.J. The generalizability of systematic direct observations across time and setting: A preliminary investigation of the psychometrics of behavioral assessment. School Psychology Review 2004, 33:258-270.
34. Hyman I.A., Wojtowicz A., Duke-Lee K., Haffner M., Fiorello C.A., Storlazzzi J.J., et al. School-based methylphenidate placebo protocols: Methodological and practical issues. Journal of Learning Disabilities 1998, 31:581-594.
35. Kurlan R. Tourette's syndrome: Current concepts. Neurology 1989, 39:1625-1630.
36. Landis J.R., Koch G.G. The measurement of observer agreement for categorical data. Biometrics 1977, 33:159-174.
37. Lei P., Smith M., Suen H.K. The use of generalizability theory to estimate data reliability in single-subject observational research. Psychology in the Schools 2007, 44:433-439.
38. Loney J., Milich R. Hyperactivity, inattention, and aggression in clinical practice. Advances in developmental and behavioral pediatrics 1982, Vol. 3:113-147. JAI, Greenwich, CT.
39. McGraw K.O., Wong S.P. Forming inferences about some intraclass correlation coefficients. Psychological Methods 1996, 1:30-46.
40. McIntosh, K., Spaulding, S. A., & Frank, J. (in press). Establishing research-based trajectories of office discipline referrals for individual students. School Psychology Review.
41. Meier S.T., McDougal J.L., Bardos A. Development of a change-sensitive outcome measure for children receiving counseling. Canadian Journal of School Psychology 2008, 23:148-160.
42. Merrell K.W. Behavioral, social, and emotional assessment of children and adolescents 2008, Erlbaum, Mahwah, NJ. 3rd ed.
43. Milich R., Balentine A., Lynam D.R. ADHD combined type and inattentive type are distinct and unrelated disorders. Clinical Psychology: Science and Practice 2001, 8:463-488.
44. Milich R., Landau S. Teacher ratings of inattention/overactivity and aggression: Cross-validation with classroom observations. Journal of Clinical Child Psychology 1988, 17:92-97.
45. Nist L., Joseph L.M. Effectiveness and efficiency of flashcard drill instructional methods on urban first-graders' word recognition, acquisition, maintenance, and generalization. School Psychology Review 2008, 37:294-308.
46. Nolan E.E., Gadow K.D. Relation between ratings and observations of stimulant drug response in hyperactive children. Journal of Clinical Child Psychology 1994, 23:78-90.
47. Parkes J. The relationship between the reliability and cost of performance assessments. Education Policy Analysis Archives 2000, March, 8(16). Retrieved March 8, 2008 from http://epaa.asu.edu/epaa/v8n16/.
48. Pelham W.E. Pharmacotherapy for children with attention-deficit hyperactivity disorder. School Psychology Review 1993, 21:199-223.
49. Pelham W.E., Fabiano G.A., Massetti G.M. Evidence-based assessment of attention deficit hyperactivity disorder in children and adolescents. Journal of Clinical Child and Adolescent Psychology 2005, 3:449-476.
50. Pelham W.E., Milich R., Murphy D.A., Murphy H.A. Normative data on the IOWA Conners Teacher Rating Scale. Journal of Clinical Child Psychology 1989, 18:259-262.
51. Pillow D.R., Pelham W.E., Hoza B., Molina B.S.G., Stultz C.H. Confirmatory factor analyses examining attention deficit hyperactivity disorder symptoms and other childhood disruptive behaviors. Journal of Abnormal Child Psychology 1998, 26:293-309.
52. Poncy B.C., Skinner C.H., Axtell P.A. An investigation of the reliability and standard error of measurement of words read correctly per minute using curriculum-based measurement. Journal of Psychoeducational Assessment 2005, 23:326-338.
53. Reich W. Diagnostic Interview for Children and Adolescents (DICA). Journal of the American Academy of Child and Adolescent Psychiatry 2000, 39:59-66.
54. Reynolds C.R., Kamphaus R.W. Behavior Assessment System for Children-2 (BASC-2) 2004, American Guidance Service, Circle Pines, MN.
55. Salvia J., Ysselydke J.E., Bolt S. Assessment in special and inclusive education 2010, Houghton Mifflin, Boston. 11th ed.
56. Schafer J.L., Olsen M.K. Multiple imputation for multivariate missing-data problems: A data analyst's perspective. Multivariate Behavioral Research 1998, 33:545-571.
57. Shapiro E.S., Heick P. School psychologist assessment practices in the evaluation of students referred for social/behavioral/emotional problems. Psychology in the Schools 2004, 41:551-561.
58. Shavelson R.J., Webb N.M. Generalizability theory: A primer 1991, Sage, Newbury Park, CA.
59. Shinn M. Best practices in curriculum-based measurement in a problem-solving model. Best practices in school psychology V 2008, Vol. 2:243-261. National Association of School Psychologists, Bethesda, MD. A. Thomas, J. Grimes (Eds.).
60. Shrout P.E., Fleiss J.L. Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin 1979, 86:420-428.
61. Silva F. Psychometric foundations of behavioral assessment 1993, SAGE, Newbury Park, CA.
62. Smith P.L. Gaining accuracy in generalizability theory: Using multiple designs. Journal of Educational Measurement 1981, 18:147-154.
63. Sprafkin, J., Mattison, R. E., Gadow, K. D., Schneider, J., & Lavigne, J. (in press). A DSM-IV-referenced teacher rating scale for monitoring behavioral improvement. Journal of Attention Disorders.
64. Suen H.K., Ary D. Analyzing quantitative behavioral observation data 1989, Erlbaum, Hillsdale, N.J.
65. Suen H.K., Rzasa S.E. Psychometric foundations of behavioral assessment. Comprehensive handbook of psychological assessment 2004, Vol. 3:37-56. Wiley, Hoboken, NJ.
66. Sugai G., Horner R.H. The evolution of discipline practices: School-wide positive behavior supports. Child & Family Behavior Therapy 2002, 24(1/2):23-50.
67. Thorndike R., Hagen E. Measurement and evaluation in psychology and education 1977, Wiley, New York. 4th ed.
68. Volpe, R. J., & Gadow, K. D. (in press). Creating abbreviated rating scales to monitor classroom inattention-overactivity, aggression, and peer conflict: Reliability, validity, and treatment sensitivity. School Psychology Review.
69. Volpe R.J., Gadow K.D., Blom-Hoffman J., Feinberg A.B. Factor-analytic and individualized approaches to constructing brief measures of ADHD behavior. Journal of Emotional and Behavioral Disorders 2009, 17:118-128.
70. Volpe R.J., Heick P., Gureasko-Moore D. An agile model for monitoring the effects of stimulant medication in schools. Psychology in the Schools 2005, 42:509-523.
71. Volpe R.J., McConaughy S.H., Hintze J.M. Generalizability of classroom behavior problem and on-task scores from the Direct Observation Form. School Psychology Review 2009, 38:382-401.
72. Walker B., Cheney D., Stage S., Blum C. Schoolwide screening and positive behavior supports: Identifying and supporting students at risk for school failure. Journal of Positive Behavior Interventions 2005, 7:194-204.
73. Walker H.M., Severson H.H. Systematic Screening for Behavior Disorders: Users guide and administration manual 1990, Sopris West, Longmont, CO.
74. Waschbusch D.A., Willoughby M.T. Parent and teacher ratings on the IOWA Conners Rating Scale. Journal of Psychopathology and Behavioral Assessment 2008, 30:180-192.
75. Witt J.C., Elliott S.N. Acceptability of classroom intervention strategies. Advances in school psychology 1985, 251-288. Erlbaum, Hillsdale, NJ. T.R. Kratochwill (Ed.).