Assessing psychiatric competencies: What does the literature tell us about mehtods of workplace-based assessment?

Advances in Psychiatric Treatment

Volumn 14, Issue 2, 2008, Pages 122-130

  Fitch, Chris ^a   Malik, Amit ^a,b   Lelliott, Paul ^a,c   Bhugra, Dinesh ^a,d   Andiappan, Manoharan ^d  

^a ROYAL COLLEGE OF PSYCHIATRISTS   (United Kingdom)

^b SOUTHERN HEALTH NHS FOUNDATION TRUST   (United Kingdom)

^c OXLEAS NHS FOUNDATION TRUST   (United Kingdom)

^d KING'S COLLEGE LONDON   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

ARTICLE; CONSTRUCT VALIDITY; CONTENT VALIDITY; EVALUATION; HUMAN; INTERNAL CONSISTENCY; INTERRATER RELIABILITY; MEDICAL EDUCATION; MEDICAL LITERATURE; METHODOLOGY; PROFESSIONAL COMPETENCE; PSYCHIATRIST; RELIABILITY; REPRODUCIBILITY; SKILL; VALIDITY; WORKPLACE;

EID: 58049086226     PISSN: 13555146     EISSN: None     Source Type: Journal    
DOI: 10.1192/apt.bp.107.003871     Document Type: Article

References (51)

1. Alguire, P. C. (1998) A review of journal clubs in postgraduate medical education. Journal of General Internal Medicine, 13, 347-353.
2. Archer, J., Norcini, J., Southgate, L., et al (2006) mini-PAT (Peer Assessment Tool): a valid component of a national assessment programme in the UK. Advances in Health Sciences Education, online content (http://springerlink.metapress.com/content/v8hth3h06507ph56/ ?p=a3e3005a289e47e386a81bb976abef58&pi=2).
3. Bozeman, D. (1997) Interrater agreement in multisource performance appraisal: a commentary. Journal of Organizational Behavior, 18, 313-316.
4. Brown, N. & Doshi, M. (2006) Assessing professional and clinical competence: the way forward. Advances in Psychiatric Treatment, 12, 81-89.
5. Datta V., Chang A., Mackay S., et al (2002) The relationship between motion analysis and surgical technical assessments. American Journal of Surgery, 184, 70-73.
6. Dugdale, A. (1996) Long-case clinical examinations. Lancet, 347, 1335.
7. Durning, S. J., Cation, L. J., Markert, R. J., et al (2002) Assessing the reliability and validity of the mini-clinical evaluation exercise for internal medicine residency training. Academic Medicine, 77, 900-904.
8. Ebbert, J. O., Montori, V. M. & Schultz, H. J. (2001) The journal club in postgraduate medical education: a systematic review. Medical Teacher, 23, 455-461.
9. Goetz, A. A., Peters, M. J., Folse, R., et al (1979) Chart review skills: a dimension of clinical competence. Journal of Medical Education, 54, 788-796.
10. Green, M. L. (1999) Graduate medical education training in clinical epidemiology, critical appraisal, and evidence-based medicine: a critical review of curricula. Academic Medicine, 74, 686-694.
11. Greenhalgh, T. (1997) How to read a paper: assessing the methodological quality of published papers. BMJ, 315, 305-308.
12. Hall, W., Violato, C., Lewkonia, R., et al (1999) Assessment of physician performance in Alberta: the Physician Achievement Review. Canadian Medical Association Journal, 161, 52-57.
13. Hamdy, H., Prasad, K., Williams, R., et al (2003) Reliability and validity of the direct observation clinical encounter examination (DOCEE). Medical Education, 37, 205-212.
14. Higgins, R. S. D., Bridges, J., Burke, J. M., et al (2004) Implementing the ACGME general competencies in a cardiothoracic surgery residency program using a 360-degree feedback. Annals of Thoracic Surgery, 77, 12-17.
15. Holmboe, E. S. (2004) Faculty and the observation of trainees' clinical skills: problems and opportunities. Academic Medicine, 79, 16-22.
16. Holmboe, E. S., Huot, S., Chung, J., et al (2003) Construct validity of the Mini Clinical Evaluation Exercise (MiniCEX). Academic Medicine, 78, 826-830.
17. Holmboe, E. S., Yepes, M., Williams, F., et al (2004) Feedback and the mini clinical evaluation exercise. Journal of General Internal Medicine, 5, 558-561.
18. Jennett, P. & Affleck, L. (1998) Chart audit and chart stimulated recall as methods of needs assessment in continuing professional health education. Journal of Continuing Education in the Health Professions, 18, 163-171.
19. Kogan, J. R., Bellini, L. M. & Shea, J. A. (2003) Feasibility, reliability, and validity of the mini-clinical evaluation exercise (mCEX) in a medicine core clerkship. Academic Medicine, 78, S33-S35.
20. Kroboth, F. J., Hanusa, B. H., Parker, S., et al (1992) The interrater reliability and internal consistency of a clinical evaluation exercise. Journal of General Internal Medicine, 7, 174-179.
21. Lee, A. G., Boldt, C., Golnik, K. C., et al (2005) Using the journal club to teach and assess competence in practice-based learning and improvement: a literature review and recommendation for implementation. Survey of Ophthalmology, 50, 542-548.
22. Lipner, R. S., Blank, L. L., Leas, B. F., et al (2002) The value of patient and peer ratings in recertification. Academic Medicine, 77, S64-S66.
23. Lockyer, J. M. & Violato, C. (2004) An examination of the appropriateness of using a common peer assessment instrument to assess physician skills across specialties. Academic Medicine, 79, S5-S8.
24. Lockyer, J., Blackmore, D., Fidler, H., et al (2006) A study of a multisource feedback system for international medical graduates holding defined licenses. Medical Education, 40, 340-347.
25. Maatsch, J. L., Huang, R. R., Downing, S., et al (1984) The predictive validity of test formats and a psychometric theory of clinical competence. Research in Medical Education, 23, 76-82.
26. Martin, J. A., Regehr, G., Reznick, R., et al (1997) Objective structured assessment of technical skill (OSATS) for surgical residents. British Journal of Surgery, 84, 273-278.
27. Mason, R., Choudhry, N., Hartley, E., et al (2003) Developing an effective system of 360-degree appraisal for consultants: results of a pilot study. Clinical Governance Bulletin, 4, 11-12.
28. McKinley, R. K., Fraser, R. C., van der Vleuten, C., et al (2000) Formative assessment of the consultation performance of medical students in the setting of general practice using a modified version of the Leicester Assessment Package. Medical Education, 34, 573-579.
29. Moorthy, K., Vincent, C. & Darzi, A. (2005) Simulation based training. BMJ, 330, 493-495.
30. Morris, A., Hewitt, J. & Roberts, C. M. (2006) Practical experience of using directly observed procedures, mini clinical evaluation examinations, and peer observation in pre registration house officer (FY1) trainees. Postgraduate Medical Journal, 82, 285-288.
31. Noel, G. L., Herbers, J. E., Capow, M. P., et al (1992) How well do internal medicine faculty members evaluate the clinical skills of residents? Annals of Internal Medicine, 1, 757-765.
32. Norcini, J. J. (2002) The death of the long case? BMJ, 324, 408-409.
33. Norcini, J. J., Blank, L. L., Arnold, G. K., et al (1995) The mini-CEX (clinical evaluation exercise): a preliminary investigation. Annals of Internal Medicine, 123, 795-799.
34. Norcini, J. J., Blank, L. L., Duffy, D., et al (2003) The mini-CEX: a method for assessing clinical skills. Annals of Internal Medicine, 138, 476-481.
35. Norman, G. R. & Shannon, S. I. (1998) Effectiveness of instruction in critical appraisal (evidence-based medicine) skills: a critical appraisal. Canadian Medical Association Journal, 158, 177-181.
36. Norman, G. R., Davis, D. A., Lamb, S., et al (1993) Competency assessment of primary care physicians as part of a peer review 270 program. JAMA, 9, 1046-1051.
37. Olson, L. G., Coughlan, J., Rolfe, I., et al (2000) The effect of a Structured Question Grid on the validity and perceived fairness of a medical long case assessment. Medical Education, 34, 46-52.
38. Price, J. & Byrne, G. J. A. (1994) The direct clinical examination: an alternative method for the assessment of clinical psychiatry skills in undergraduate medical students. Medical Education, 28, 120-125.
39. Ramsey, P. G., Wenrich, M. D., Carline, J. D., et al (1993) Use of peer ratings to evaluate physician performance. JAMA, 13, 1655-1660.
40. Ramsey, P. G., Carline, J. D., Blank, L. L., et al (1996) Feasibility of hospital-based use of peer ratings to evaluate the performances of practising physicians. Academic Medicine, 71, 364-370.
41. Regehr, G., MacRae, H., Reznick, R. K., et al (1998) Comparing the psychometric properties of checklists and global rating scales for assessing performance on an OSCE-format examination. Academic Medicine, 73, 993-997.
42. Schurwirth, L. W. T. & van der Vleuten, C. P. M. (2006) A plea for new psychometric models in educational assessment. Medical Education, 40, 296-300.
43. Solomon, D. J., Reinhart, M. A., Bridgham, R. G., et al (1990) An assessment of an oral examination format for evaluating clinical competence in emergency medicine. Academic Medicine, 65, S43-S44.
44. ten Cate, O. (2006) Trust, competence, and the supervisor's role in postgraduate training. BMJ, 333, 748-751.
45. Thomas, P. A., Gebo, K. A. & Hellmann, D. B. (1999) A pilot study of peer review in residency training. Journal of General Internal Medicine, 14, 551-554.
46. Violato, C., Marini, A., Toews, J., et al (1997) Feasibility and psychometric properties of using peers, consulting physicians, co-workers, and patients to assess physicians. Academic Medicine, 72, S82-S84.
47. Wass, V. & Jolly, B. (2001) Does observation add to the validity of the long case? Medical Education, 35, 729-734.
48. Wass, V. & van der Vleuten, C. P. M. (2004) The long case. Medical Education, 38, 1176-1180.
49. Weinrich, M. D., Carline, I. D., Giles, L. M., et al (1993) Ratings of the performances of practising internists by hospital-based registered nurses. Academic Medicine, 68, 680-687.
50. Weisse, A. B. (2002) The oral examination. Awful or awesome? Perspectives in Biology and Medicine, 45, 569-578.
51. Wilkinson, J., Benjamin, A. & Wade, W. (2003) Assessing the performance of doctors in training. BMJ, 327, S91-S92.