Inter-rater reliability of clinician-rated outcome measures in child and adolescent mental health services

Administration and Policy in Mental Health and Mental Health Services Research

Volumn 34, Issue 6, 2007, Pages 504-512

  Hanssen Bauer, Ketil ^a   Aalen, Odd O ^b   Ruud, Torleif ^c   Heyerdahl, Sonja ^a  

^a ULLEVÅL UNIVERSITY HOSPITAL   (Norway)

^b UNIVERSITY OF OSLO   (Norway)

^c AKERSHUS UNIVERSITY HOSPITAL   (Norway)

Author keywords

Adolescent; Children; Mental health services; Outcome; Reliability

Indexed keywords

ARTICLE; CHILDREN GLOBAL ASSESSMENT SCALE; CONTROLLED STUDY; CORRELATION COEFFICIENT; EXPERIENCE; FEMALE; GLOBAL ASSESSMENT OF PSYCHOSOCIAL DISABILITY; HEALTH OF THE NATION OUTCOME SCALES FOR CHILDREN AND ADOLESCENT; HUMAN; INTERRATER RELIABILITY; MALE; MENTAL HEALTH SERVICE; NORWAY; OUTCOME ASSESSMENT; PEDIATRICS; PHYSICIAN; PRIORITY JOURNAL; RATING SCALE; SCORING SYSTEM; VIGNETTE;

ADOLESCENT; CHILD; FEMALE; HEALTH PERSONNEL; HUMANS; MALE; MENTAL HEALTH SERVICES; NORWAY; OBSERVER VARIATION; OUTCOME ASSESSMENT (HEALTH CARE);

EID: 35748986146     PISSN: 0894587X     EISSN: 15733289     Source Type: Journal    
DOI: 10.1007/s10488-007-0134-y     Document Type: Article

References (47)

1. American Psychiatric Association (2000). Diagnostic and statistical manual of mental disorders (4th ed., Text Revision ed.). Washington, DC: American Psychiatric Association.
2. Anastasi, A., & Urbina, S. (1997). Psychological testing (7th ed.). Prentice Hall International.
3. Australian Mental Health Outcomes and Classification Network (2005). Child & Adolescent National Outcomes & Casemix Collection Standard Reports, Version 1.1. Brisbane, Queensland.
4. Bates, M. P. (2001). The child and adolescent functional assessment scale (CAFAS): Review and current status. Clinical Child & Family Psychology Review, 4(1), 63-84.
5. Bickman, L., Guthrie, P. R., Foster, E. M., Lambert, E. W., Summerfelt, W., Breda, C. S., et al. (1995). Evaluating managed mental health services: The Fort Bragg experiment. New York, NY, US: Plenum Press.
6. Bilenberg, N. (2003). Health of the nation outcome scales for children and adolescents (HoNOSCA). Results of a Danish field trial. European Child & Adolescent Psychiatry, 12(6), 298-302.
7. Brann, P., Coleman, G., & Luk, E. (2001). Routine outcome measurement in a child and adolescent mental health service: An evaluation of HoNOSCA. The Australian and New Zealand Journal of Psychiatry, 35(3), 370-376.
8. CAFAS in Ontario (2005). Child and adolescent functional assessment scale. http://www.cafasinontario.ca/html/home.asp, access date 12.4.2007.
9. CAMHS Outcome Research Consortium (2007). CORC homepage. http://www.corc.uk.net/, access date 19.3.2007.
10. Dunn, G. (1989). Design and analysis of reliability studies: The statistical evaluation of measurement errors. New York: Oxford University Press.
11. Dyrborg, J., Larsen, F. W., Nielsen, S., Byman, J., Nielsen, B. B., & Gautre-Delay, F. (2000). The children's global assessment scale (CGAS) and global assessment of psychosocial disability (GAPD) in clinical practice-substance and reliability as judged by intraclass correlations. European Child & Adolescent Psychiatry, 9(3), 195-201.
12. Efron, B., & Tibshirani, R. J. (1993). An introduction to the bootstrap. Chapman & Hall.
13. Epstein, M. H., Kutash, K., & Duchnowski, A. J. (2005). Outcomes for children and youth with emotional and behavioral disorders and their families: Programs and evaluation best practices (2nd ed.). Austin, TX: Pro-Ed.
14. Fonagy, P. (2002). Outcome measurement in children and adolescents. In W. W. IsHak, T. Burt, & L. I. Sederer (Eds.), Outcome measurement in psychiatry: a critical review (pp. 59-75). Washington, DC: American Psychiatric Publishing, Inc.
15. Ford, T., Tingay, K., Wolpert, M., & the CORC Steering Group. (2006). CORC's survey of routine outcome monitoring and national CAMHS dataset developments: A response to Johnston and Gowers. Child and Adolescent Mental Health, 11(1), 50-52.
16. Garralda, M. E., Yates, P., & Higginson, I. (2000). Child and adolescent mental health service use. HoNOSCA as an outcome measure. British Journal of Psychiatry, 177, 52-58.
17. Gowers, S. G., Harrington, R. C., Whitton, A., Beevor, A., Lelliott, P., Jezzard, R., et al. (1999a). Health of the nation outcome scales for children and adolescents (HoNOSCA). Glossary for HoNOSCA score sheet. British Journal of Psychiatry, 174, 428-431.
18. Gowers, S. G., Harrington, R. C., Whitton, A., Lelliott, P., Beevor, A., Wing, J., et al. (1999b). Brief scale for measuring the outcomes of emotional and behavioural disorders in children. Health of the nation outcome scales for children and adolescents (HoNOSCA). British Journal of Psychiatry, 174, 413-416.
19. Gowers, S., Bailey-Rogers, S. J., Shore, A., & Levine, W. (2000). The health of the nation outcome scales for child & adolescent mental health (HoNOSCA). Child Psychology & Psychiatry Review, 5(2), 50-56.
20. Green, B., Shirk, S., Hanze, D., & Wanstrath, J. (1994). The children's global assessment scale in clinical practice: An empirical evaluation. Journal of the American Academy of Child and Adolescent Psychiatry, 33(8), 1158-1164.
21. Hoagwood, K., Jensen, P. S., Petti, T., & Burns, B. J. (1996). Outcomes of mental health care for children and adolescents: I. A comprehensive conceptual model. Journal of the American Academy of Child and Adolescent Psychiatry, 35(8), 1055-1063.
22. Hodges, K., & Gust, J. (1995). Measures of impairment for children and adolescents. Journal of Mental Health Administration, 22(4), 403-413.
23. Hodges, K., Wong, M. M., & Latessa, M. (1998) Use of the child and adolescent functional assessment scale (CAFAS) as an outcome measure in clinical settings. Journal of Behavioral Health Services & Research, 25(3), 325-336.
24. Hodges, K., & Wotring, J. (2004) The role of monitoring outcomes in initiating implementation of evidence-based treatments at the state level. Psychiatric Services, 55(4), 396-400.
25. HoNOSCA Project (2003). Health of the nation outcome scales for children and adolescents. http://www.liv.ac.uk/honosca/Home.htm, access date 16.4.2007.
26. Jensen, P. S., Hoagwood, K., & Petti, T. (1996). Outcomes of mental health care for children and adolescents: II. Literature review and application of a comprehensive model. Journal of the American Academy of Child and Adolescent Psychiatry, 35(8), 1064-1077.
27. Johnston, C., & Gowers, S. (2005). Routine outcome measurement: A survey of UK child and adolescent mental health services. Child and Adolescent Mental Health, 10(3), 133-139.
28. Kazdin, A. E. (2003). Psychotherapy for children and adolescents. Annual Review of Psychology, 54, 253-276.
29. MH-SMART (2007). The mental health standard measures of assessment and recovery summary. http://www.tepou.co.nz/page/tepou_11.php, access date 19.3.2007.
30. National Mental Health Strategy. (2004). Australian mental health outcomes and classification network. http://www.mhnocc.org/amhocn/, access date 14.3.2007.
31. Norcini, J. (2004) Back to the future: clinical vignettes and the measurement of physician performance. Annals of Internal Meicine, 141(10), 813-814.
32. Peabody, J. W., Luck, J., Glassman, P., Jain, S., Hansen, J., Spell, M., et al. (2004). Measuring the quality of physician practice by using clinical vignettes: A prospective validation study. Annals of Internal Medicine, 141(10), 771-780.
33. Pirkis, J. E., Burgess, P. M., Kirk, P. K., Dodson, S., Coombs, T. J., & Williamson, M. K. (2005). A review of the psychometric properties of the health of the nation outcome scales (HoNOS) family of measures. Health and Quality of Life Outcomes, 3, 76.
34. Rey, J. M., Starling, J., Wever, C., Dossetor, D. R., & Plapp, J. M. (1995). Inter-rater reliability of global assessment of functioning in a clinical setting. Journal of Child Psychology and Psychiatry, 36(5), 787-792.
35. Schorre, B. E., & Vandvik, I. H. (2004). Global assessment of psychosocial functioning in child and adolescent psychiatry. A review of three unidimensional scales (CGAS, GAF, GAPD). European Child & Adolescent Psychiatry, 13(5), 273-286.
36. Shaffer, D., Gould, M. S., Brasic, J., Ambrosini, P., Fisher, P., Bird, H., et al. (1983). A children's global assessment scale (CGAS). Archives of General Psychiatry, 40(11), 1228-1231.
37. Shrout, P. E. (1998). Measurement reliability and agreement in psychiatry. Statistical Methods in Medical Research, 7(3), 301-317.
38. Shrout, P. E., & Fleiss, J. L. (1979). Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin, 86(2), 420-428.
39. Sitter, M., & Hagen, H. (2004). Opptrappingsplanens mål: Status psykisk helsevern for barn og unge. In V. Halsteinli (Ed.), Samdata psykisk helsevern. Sektorrapport 2003. Sammenligningsdata for psykisk helsevern. Sintef Helse.
40. Vatnaland, T., Vatnaland, J., Friis, S., & Opjordsmoen, S. (2007). Are GAF scores reliable in routine clinical use? Acta Psychiatrica Scandinavica, 115(4), 326-330.
41. Weiss, B. (1998). Annotation: Routine monitoring of the effectiveness of child psychotherapy. Journal of Child Psychology and Psychiatry, 39(7), 943-950.
42. Williams, R., & Kerfoot, M. (2005). Child and adolescent mental health services. Strategy, planning, delivery and evaluation. Oxford University Press.
43. Winters, N. C., Collett, B. R., & Myers, K. M. (2005). Ten-year review of rating scales, VII: Scales assessing functional impairment. Journal of the American Academy of Child and Adolescent Psychiatry, 44(4), 309-338.
44. Wolpert, M., Thompson, M., & Tingay, K. (2005). Data collection, clinical audit, and measuring outcomes. In R. Williams & M. Kerfoot (Eds.), Child and adolescent mental health services. Strategy, planning, delivery and evaluation (pp. 519-533). Oxford University Press.
45. World Health Organization (1994). International statistical classification of diseases and related health problems 10th revision (ICD-10).
46. World Health Organization (1996). Multiaxial classification of child and adolescent psychiatric disorders. Cambridge University Press.
47. Yates, P., Garralda, M. E., & Higginson, I. (1999). Paddington complexity scale and health of the nation outcome scales for children and adolescents. British Journal of Psychiatry, 174, 417-423.