Reliability and validity of OSCE checklists used to assess the communication skills of undergraduate medical students: A systematic review

Patient Education and Counseling

Volumn 98, Issue 12, 2015, Pages 1482-1491

  Setyonugroho, Winny ^a   Kennedy, Kieran M ^b   Kropmans, Thomas J B ^b  

^a UNIVERSITAS MUHAMMADIYAH YOGYAKARTA   (Indonesia)

^b NATIONAL UNIVERSITY OF IRELAND   (Ireland)

Author keywords

Assessment; Communication skills; OSCE; Systematic review; Undergraduate medical education

Indexed keywords

ANAMNESIS; CHECKLIST; CLINICAL COMPETENCE; COMMUNICATION SKILL; DOCTOR PATIENT RELATION; EMPATHY; HEALTH BELIEF; HUMAN; MEDICAL STUDENT; NONVERBAL COMMUNICATION; OBJECTIVE STRUCTURED CLINICAL EXAMINATION; PATIENT COUNSELING; PRIORITY JOURNAL; RELIABILITY; REVIEW; SYSTEMATIC REVIEW; VALIDITY;

EID: 84947291661     PISSN: 07383991     EISSN: 18735134     Source Type: Journal    
DOI: 10.1016/j.pec.2015.06.004     Document Type: Review

References (77)

1. Stewart M.A. Effective physician-patient communication and health outcomes: a review. Can. Med. Assoc. J. 1995, 152(9):1423-1433.
2. Ong L.M., de Haes J.C., Hoos A.M., Lammes F.B. Doctor-patient communication: a review of the literature. Soc. Sci. Med. 1995, 40(7):903-918.
3. Phillips C. Communication: the first tool in risk management for long-term care. J. Am. Med. Dir. Assoc. 2004, 5(2):123-126.
4. Beck R.S., Daughtridge R., Sloane P.D. Physician-patient communication in the primary care office: a systematic review. J. Am. Board Fam. Pract. 2002, 15(1):25-38.
5. Levinson W. Physician-patient communication. A key to malpractice prevention. J. Am. Med. Assoc. 1994, 272(20):1619-1620.
6. Maguire P., Pitceathly C. Key communication skills and how to acquire them. Br. Med. J. 2002, 325(7366):697-700.
7. Maguire P., Pitceathly C. Managing the difficult consultation. Clin. Med. 2003, 3(6):532-537.
8. Noble L.M., Kubacki A., Martin J., Lloyd M. The effect of professional skills training on patient-centredness and confidence in communicating with patients. Med. Educ. 2007, 41(5):432-440.
9. Bowleg L., Valera P., Teti M., Tschann J.M. Silences, gestures, and words: nonverbal and verbal communication about HIV/AIDS and condom use in black heterosexual relationships. Health Commun. 2010, 25(1):80-90.
10. Yedidia M.J., Gillespie C.C., Kachur E., Schwartz M.D., Ockene J., Chepaitis A.E., et al. Effect of communications training on medical student performance. J. Am. Med. Assoc. 2003, 290(9):1157-1165.
11. Humphris G.M. Communication skills knowledge, understanding and OSCE performance in medical trainees: a multivariate prospective study using structural equation modelling. Med. Educ. 2002, 36(9):842-852.
12. Harden R.M., Stevenson M., Downie W.W., Wilson G.M. Assessment of clinical competence using objective structured examination. Br. Med. J. 1975, 1(5955):447-451.
13. Turner J.L., Dankoski M.E. Objective structured clinical exams: a critical review. Fam. Med. 2008, 40(8):574-578.
14. Hodges B., Turnbull J., Cohen R., Bienenstock A., Norman G. Evaluating communication skills in the OSCE format: reliability and generalizability. Med. Educ. 1996, 30(1):38-43.
15. Hodges B., McIlroy J.H. Analytic global OSCE ratings are sensitive to level of training. Med. Educ. 2003, 37(11):1012-1016.
16. Kogan J.R., Bellini L.M., Shea J.A. Feasibility, reliability, and validity of the mini-clinical evaluation exercise (mCEX) in a medicine core clerkship. Acad. Med. 2003, 78(10 Suppl):S33-S35.
17. Khattab A.D., Rawlings B. Assessing nurse practitioner students using a modified objective structured clinical examination (OSCE). Nurse Educ. Today 2001, 21(7):541-550.
18. Khattab A.D., Rawlings B. Use of a modified OSCE to assess nurse practitioner students. Br. J. Nurs. 2008, 17(12):754-759.
19. Boon H., Stewart M. Patient-physician communication assessment instruments: 1986 to 1996 in review. Patient Educ. Couns. 1998, 35(3):161-176.
20. Barry M., Bradshaw C., Noonan M. Improving the content and face validity of OSCE assessment marking criteria on an undergraduate midwifery programme: a quality initiative. Nurse Educ. Pract. 2013, 13(5):477-480.
21. Moineau G., Power B., Pion A.M., Wood T.J., Humphrey-Murto S. Comparison of student examiner to faculty examiner scoring and feedback in an OSCE. Med. Educ. 2011, 45(2):183-191.
22. Chang A., Boscardin C., Chou C.L., Loeser H., Hauer K.E. Predicting failing performance on a standardized patient clinical performance examination: the importance of communication and professionalism skills deficits. Acad. Med. 2009, 84(10 Suppl.):S101-S104.
23. Schirmer J.M., Mauksch L., Lang F., Marvel M.K., Zoppi K., Epstein R.M., et al. Assessing communication competence: a review of current tools. Fam. Med. 2005, 37(3):184-192.
24. Amin Z., Eng K.H. Basics in Medical Education, World Scientific. Appendix A: Calgary-Cambridge Observation Guide 2002, Available from: http://www.worldscientific.com/doi/pdf/10.1142/9789812795472_bmatter.
25. van Es J.M., Schrijver C.J., Oberink R.H., Visser M.R. Two-dimensional structure of the MAAS-Global rating list for consultation skills of doctors. Med. Teach. 2012, 34(12):e794-e799.
26. Patricio M.F., Juliao M., Fareleira F., Carneiro A.V. Is the OSCE a feasible tool to assess competencies in undergraduate medical education?. Med. Teach. 2013, 35(6):503-514.
27. Cegala D.J., Gade C., Lenzmeier Broz S., McClure L. Physicians' and patients' perceptions of patients' communication competence in a primary care medical interview. Health Commun. 2004, 16(3):289-304.
28. Cegala D.J., Lenzmeier Broz S. Physician communication skills training: a review of theoretical backgrounds, objectives and skills. Med. Educ. 2002, 36(11):1004-1016.
29. Beckman T.J., Cook D.A. Educational epidemiology. J. Am. Med. Assoc. 2004, 292(24):2969. (author reply 70-1).
30. Beckman T.J., Cook D.A., Mandrekar J.N. What is the validity evidence for assessments of clinical teaching?. J. Gen. Intern. Med. 2005, 20(12):1159-1164.
31. Beckman T.J., Ghosh A.K., Cook D.A., Erwin P.J., Mandrekar J.N. How reliable are assessments of clinical teaching? A review of the published instruments. J. Gen. Intern. Med. 2004, 19(9):971-977.
32. Fallowfield L., Jenkins V. Effective communication skills are the key to good cancer care. Eur. J. Cancer 1999, 35(11):1592-1597.
33. Fallowfield L., Jenkins V. Acronymic trials: the good, the bad, and the coercive. Lancet 2002, 360(9346):1622.
34. Fallowfield L., Jenkins V. Communicating sad, bad, and difficult news in medicine. Lancet 2004, 363(9405):312-319.
35. Fallowfield L., Jenkins V. Current concepts of communication skills training in oncology. Recent Results Cancer Res. 2006, 168:105-112.
36. Vazirani S., Hays R.D., Shapiro M.F., Cowan M. Effect of a multidisciplinary intervention on communication and collaboration among physicians and nurses. Am. J. Crit. Care 2005, 14(1):71-77.
37. Beuscart-Zephir M.C., Pelayo S., Anceaux F., Meaux J.J., Degroisse M., Degoulet P. Impact of CPOE on doctor-nurse cooperation for the medication ordering and administration process. Int. J. Med. Inform. 2005, 74(7-8):629-641.
38. Cook D.A., Beckman T.J. Current concepts in validity and reliability for psychometric instruments: theory and application. Am. J. Med. 2006, 119(2):166.e7-166.e16.
39. Kraan H.F., Crijnen A.A., de Vries M.W., Zuidweg J., Imbos T., Van der Vleuten C.P. To what extent are medical interviewing skills teachable?. Med. Teach. 1990, 12(3-4):315-328.
40. Essers G., Kramer A., Andriesse B., van Weel C., van der Vleuten C., van Dulmen S. Context factors in general practitioner-patient encounters and their impact on assessing communication skills-an exploratory study. BMC Fam. Pract. 2013, 14:65.
41. Hobma S., Ram P., Muijtjens A., van der Vleuten C., Grol R. Effective improvement of doctor-patient communication: a randomised controlled trial. Br. J. Gen. Pract. 2006, 56(529):580-586.
42. Reinders M.E., Blankenstein A.H., van Marwijk H.W., Knol D.L., Ram P., van der Horst H.E., et al. Reliability of consultation skills assessments using standardised versus real patients. Med. Educ. 2011, 45(6):578-584.
43. Bloch R., Norman G. Generalizability theory for the perplexed: a practical introduction and guide: AMEE Guide No. 68. Med. Teach. 2012, 34(11):960-992.
44. step2cs.net Copyright© 2009-2015 Clinical Skills Review, LLC. All Rights Reserved. Legal@Step2CS.net.
45. Al-Naami M.Y. Reliability, validity, and feasibility of the Objective Structured Clinical Examination in assessing clinical skills of final year surgical clerkship. Saudi Med. J. 2008, 29(12):1802-1807.
46. Bergus G.R., Woodhead J.C., Kreiter C.D. Trained lay observers can reliably assess medical students' communication skills. Med. Educ. 2009, 43(7):688-694.
47. Blue A.V., Chessman A.W., Gilbert G.E., Mainous A.G. Responding to patients' emotions: important for standardized patient satisfaction. Fam. Med. 2000, 32(5):326-330.
48. Blue A.V., Chessman A.W., Gilbert G.E., Schuman S.H., Mainous A.G. Medical students' abilities to take an occupational history: use of the WHACS mnemonic. J. Occup. Environ. Med. 2000, 42(11):1050-1053.
49. Boehlecke B., Sperber A.D., Kowlowitz V., Becker M., Contreras A., McGaghie W.C. Smoking history-taking skills: a simple guide to teach medical students. Med. Educ. 1996, 30(4):283-289.
50. Cave J., Washer P., Sampson P., Griffin M., Noble L. Explicitly linking teaching and assessment of communication skills. Med. Teach. 2007, 29(4):317-322.
51. Chesser A.M., Laing M.R., Miedzybrodzka Z.H., Brittenden J., Heys S.D. Factor analysis can be a useful standard setting tool in a high stakes OSCE assessment. Med. Educ. 2004, 38(8):825-831.
52. Chessman A.W., Blue A.V., Gilbert G.E., Carey M., Mainous A.G. Assessing students' communication and interpersonal skills across evaluation settings. Fam. Med. 2003, 35(9):643-648.
53. Huang C.C., Chan C.Y., Wu C.L., Chen Y.L., Yang H.W., Chen C.H., et al. Assessment of clinical competence of medical students using the objective structured clinical examination: first 2 years' experience in Taipei Veterans General Hospital. J. Chin. Med. Assoc. 2010, 73(11):589-595.
54. Jacobs J.C., Denessen E., Postma C.T. The structure of medical competence and results of an OSCE. Neth. J. Med. 2004, 62(10):397-403.
55. Park R.S., Chibnall J.T., Blaskiewicz R.J., Furman G.E., Powell J.K., Mohr C.J. Construct validity of an objective structured clinical examination (OSCE) in psychiatry: associations with the clinical skills examination and other indicators. Acad. Psychiatry 2004, 28(2):122-128.
56. Robins L.S., White C.B., Alexander G.L., Gruppen L.D., Grum C.M. Assessing medical students' awareness of and sensitivity to diverse health beliefs using a standardized patient station. Acad. Med. 2001, 76(1):76-80.
57. Rosebraugh C.J., Speer A.J., Solomon D.J., Szauter K.E., Ainsworth M.A., Holden M.D., et al. Setting standards and defining quality of performance in the validation of a standardized-patient examination format. Acad. Med. 1997, 72(11):1012-1014.
58. Scheffer S., Muehlinghaus I., Froehmel A., Ortwein H. Assessing students' communication skills: validation of a global rating. Adv. Health Sci. Educ. Theory Pract. 2008, 13(5):583-592.
59. Thistlethwaite J.E. Developing an OSCE station to assess the ability of medical students to share information and decisions with patients: issues relating to interrater reliability and the use of simulated patients. Educ. Health (Abingdon) 2002, 15(2):170-179.
60. Troncon L.E. Significance of experts' overall ratings for medical student competence in relation to history-taking. Sao Paulo Med. J. 2006, 124(2):101-104.
61. van Dalen J., Kerkhofs E., van Knippenberg-Van Den Berg B.W., van Den Hout H.A., Scherpbier A.J., van der Vleuten C.P. Longitudinal and concentrated communication skills programmes: two dutch medical schools compared. Adv. Health Sci. Educ. Theory Pract. 2002, 7(1):29-40.
62. Volkan K., Simon S.R., Baker H., Todres I.D. Psychometric structure of a comprehensive objective structured clinical examination: a factor analytic approach. Adv. Health Sci. Educ. Theory Pract. 2004, 9(2):83-92.
63. Walters K., Osborn D., Raven P. The development, validity and reliability of a multimodality objective structured clinical examination in psychiatry. Med. Educ. 2005, 39(3):292-298.
64. Wass V., Jolly B. Does observation add to the validity of the long case?. Med. Educ. 2001, 35(8):729-734.
65. Wass V., McGibbon D., Van der Vleuten C. Composite undergraduate clinical examinations: how should the components be combined to maximize reliability?. Med. Educ. 2001, 35(4):326-330.
66. Wilkerson L., Fung C.C., May W., Elliott D. Assessing patient-centered care: one approach to health disparities education. J. Gen. Intern. Med. 2010, 25(Suppl. 2):S86-S90.
67. Wong M.L., Fones C.S., Aw M., Tan C.H., Low P.S., Amin Z., et al. Should non-expert clinician examiners be used in objective structured assessment of communication skills among final year medical undergraduates?. Med. Teach. 2007, 29(9):927-932.
68. Blue A.V., Stratton T.D., Plymale M., DeGnore L.T., Schwartz R.W., Sloan D.A. The effectiveness of the structured clinical instruction module. Am. J. Surg. 1998, 176(1):67-70.
69. Bosse H.M., Schultz J.H., Nickel M., Lutz T., Moltner A., Junger J., et al. The effect of using standardized patients or peer role play on ratings of undergraduate communication training: a randomized controlled trial. Patient Educ. Couns. 2012, 87(3):300-306.
70. Harasym P.H., Woloschuk W., Cunning L. Undesired variance due to examiner stringency/leniency effect in communication skill scores assessed in OSCEs. Adv. Health Sci. Educ. Theory Pract. 2008, 13(5):617-632.
71. Ho M.J., Yao G., Lee K.L., Hwang T.J., Beach M.C. Long-term effectiveness of patient-centered training in cultural competence: what is retained? What is lost?. Acad. Med. 2010, 85(4):660-664.
72. Kaul P., Barley G., Guiton G. Medical student performance on an adolescent medicine examination. J. Adolesc. Health 2012, 51(3):299-301.
73. Mazor K.M., Ockene J.K., Rogers H.J., Carlin M.M., Quirk M.E. The relationship between checklist scores on a communication OSCE and analogue patients' perceptions of communication. Adv. Health Sci. Educ. Theory Pract. 2005, 10(1):37-51.
74. McLay R.N., Rodenhauser P., Anderson D.S., Stanton M.L., Markert R.J. Simulating a full-length psychiatric interview with a complex patient: an OSCE for medical students. Acad. Psychiatry 2002, 26(3):162-167.
75. Regehr G., Freeman R., Hodges B., Russell L. Assessing the generalizability of OSCE measures across content domains. Acad. Med. 1999, 74(12):1320-1322.
76. Verma M., Singh T. Communication skills in clinical practice fad or necessity?. Indian Pediatr. 1994, 31(2):237-238.
77. Schuman S.H., Simpson W.M. WHACS your patients. J. Occup. Environ. Med. 1999, 41(10):829.