How to construct and implement script concordance tests: Insights from a systematic review

Medical Education

Volumn 46, Issue 6, 2012, Pages 552-563

  Dory, Valérie ^a,b   Gagnon, Robert ^c   Vanpee, Dominique ^b,d   Charlin, Bernard ^c  

^a Fonds de la Recherche Scientifique FNRS   (Belgium)

^b UNIVERSITÉ CATHOLIQUE DE LOUVAIN   (Belgium)

^c UNIVERSITÉ DE MONTRÉAL   (Canada)

^d UNIVERSITY HOSPITAL OF MONT GODINNE   (Belgium)

Author keywords

[No Author keywords available]

Indexed keywords

CLINICAL COMPETENCE; CURRICULUM; DECISION MAKING; EDUCATION; HUMAN; MEDICAL EDUCATION; METHODOLOGY; REPRODUCIBILITY; REVIEW; STANDARD;

CLINICAL COMPETENCE; CURRICULUM; DECISION MAKING; EDUCATION, MEDICAL; EDUCATIONAL MEASUREMENT; HUMANS; REPRODUCIBILITY OF RESULTS;

EID: 84861468716     PISSN: 03080110     EISSN: 13652923     Source Type: Journal    
DOI: 10.1111/j.1365-2923.2011.04211.x     Document Type: Article

References (73)

1. Norman G. Research in clinical reasoning: past history and current trends. Med Educ 2005;39 (4):418-27.
2. Croskerry P. A universal model of diagnostic reasoning. Acad Med 2009;84 (8):1022-8.
3. Pelaccia T, Tardif J, Triby E, Charlin B. An analysis of clinical reasoning through a recent and comprehensive approach: the dual-process theory. Med Educ Online 2011;16:5890.
4. De Neys W, Glumicic T. Conflict monitoring in dual process theories of thinking. Cognition 2008;106 (3):1248-99.
5. Eva KW. What every teacher needs to know about clinical reasoning. Med Educ 2005;39 (1):98-106.
6. Custers EJFM, Regehr G, Norman GR. Mental representations of medical diagnostic knowledge: a review. Acad Med 1996;71 (10) (Suppl):55-61.
7. Charlin B, Tardif J, Boshuizen HP. Scripts and medical diagnostic knowledge: theory and applications for clinical reasoning instruction and research. Acad Med 2000;75 (2):182-90.
8. Charlin B, Boshuizen HP, Custers EJ, Feltovich PJ. Scripts and clinical reasoning. Med Educ 2007;41 (12):1178-84.
9. van der Vleuten CP, Newble DI. How can we test clinical reasoning? Lancet 1995;345 (8956):1032-4.
10. Kreiter CD, Bergus G. The validity of performance-based measures of clinical reasoning and alternative approaches. Med Educ 2009;43 (4):320-5.
11. Schuwirth L. Is assessment of clinical reasoning still the Holy Grail? Med Educ 2009;43 (4):298-300.
12. Elstein AS. Beyond multiple-choice questions and essays: the need for a new way to assess clinical competence. Acad Med 1993;68 (4):244-9.
13. Case SM, Swanson DB. Extended-matching items: a practical alternative to free-response questions. Teach Learn Med 1993;5:107-15.
14. Miller GE. The assessment of clinical skills/competence/performance. Acad Med 1990;9 (Suppl):63-7.
15. Wass V, van der Vleuten C, Shatzer J, Jones R. Assessment of clinical competence. Lancet 2001;357 (9260):945-9.
16. van der Vleuten CP, Schuwirth LW. Assessing professional competence: from methods to programmes. Med Educ 2005;39 (3):309-17.
17. Norcini J, Anderson B, Bollela V etal. Criteria for good assessment: consensus statement and recommendations from the Ottawa 2010 Conference. Med Teach 2011;33 (3):206-14.
18. Charlin B, Brailovsky C, Leduc C, Blouin D. The diagnosis script questionnaire: a new tool to assess a specific dimension of clinical competence. Adv Health Sci Educ Theory Pract 1998;3 (1):51-8.
19. Lubarsky S, Charlin B, Cook DA, Chalk C, van der Vleuten CP. Script concordance testing: a review of published validity evidence. Med Educ 2011;45 (4):329-38.
20. Haig A, Dozier M. BEME Guide No. 3: systematic searching for evidence in medical education - Part 1: Sources of information. Med Teach 2003;25 (4):352-63.
21. Gantelet M. Impact du panel de référence sur les résultats d'un test de concordance de script (TCS) développé en formation initiale des sages-femmes. Dissertation for the completion of a diploma of the Ecole des cadres sages-femmes. Dijon: Ecole de Cadres Sages-Femmes 2008.
22. Lamia B, Sitbon O. Evaluation du raisonnement et de la compétence clinique par le test de concordance de script Universités Paris V, VI, XI, XII. Inter-university diploma dissertation. Paris: Universités Paris V, VI, XI, XII 2006.
23. Deschênes MF, Charlin B, Gagnon R, Goudreau J. Use of a script concordance test to assess development of clinical reasoning in nursing students. J Nurs Educ 2011;50 (7):381-7.
24. Cohen LJ, Fitzgerald SG, Lane S, Boninger ML, Minkel J, McCue M. Validation of the seating and mobility script concordance test. Assist Technol 2009;21 (1):47-56.
25. Carriere B, Gagnon R, Charlin B, Downing S, Bordage G. Assessing clinical reasoning in pediatric emergency medicine: validity evidence for a script concordance test. Ann Emerg Med 2009;53 (5):647-52.
26. Ramaekers S, Kremer W, Pilot A, van Beukelen P, van Keulen H. Assessment of competence in clinical reasoning and decision making under uncertainty: the script concordance test method. Assess Eval High Educ 2010;35 (6):661-73.
27. Goulet F, Jacques A, Gagnon R, Charlin B, Shabah A. Poorly performing physicians: does the script concordance test detect bad clinical reasoning? J Contin Educ Health Prof 2010;30 (3):161-6.
28. Meterissian S, Zabolotny B, Gagnon R, Charlin B. Is the script concordance test a valid instrument for assessment of intraoperative decision-making skills? Am J Surg 2007;193 (2):248-51.
29. Monnier P, Bédard M, Gagnon R, Charlin B. The relationship between script concordance test scores in an obstetrics-gynecology rotation and global performance assessments in the curriculum. Int J Med Educ 2011;2:3-6.
30. Boulouffe C, Charlin B, Vanpee D. Evaluation of clinical reasoning in basic emergencies using a script concordance test. Am J Pharm Educ 2010;74 (10):194.
31. Ruiz JG, Tunuguntla R, Charlin B, Ouslander JG, Symes SN, Gagnon R, Phancao F, Roos BA. The script concordance test as a measure of clinical reasoning skills in geriatric urinary incontinence. J Am Geriatr Soc 2010;58 (11):2178-84.
32. Lubarsky S, Chalk C, Kazitani D, Gagnon R, Charlin B. The script concordance test: a new tool assessing clinical judgement in neurology. Can J Neurol Sci 2009;36 (3):326-31.
33. Lambert C, Gagnon R, Nguyen D, Charlin B. The script concordance test in radiation oncology: validation study of a new tool to assess clinical reasoning. Radiat Oncol 2009;4:7.
34. Tunuguntla R, Ouslander JG, Symes S, Phancao F, Charlin B, Gagnon R, Roos BA, Ruiz JG. A script concordance test (SCT) to measure clinical reasoning for managing geriatric urinary incontinence (UI). J Gen Intern Med 2009;24(Suppl 1):10.
35. Gagnon R, Lubarsky S, Lambert C, Charlin B. Optimization of answer keys for script concordance testing: should we exclude deviant panellists, deviant responses, or neither? Adv Health Sci Educ Theory Pract 2011;16 (5):601-8.
36. Charlin B, Gagnon R, Pelletier J, Coletti M, Abi-Rizk G, Nasr C, Sauve E, van der Vleuten C. Assessment of clinical reasoning in the context of uncertainty: the effect of variability within the reference panel. Med Educ 2006;40 (9):848-54.
37. Marie I, Sibert L, Roussel F, Hellot MF, Lechevallier J, Weber J. The script concordance test: a new evaluation method of both clinical reasoning and skills in internal medicine. Rev Med Interne 2005;26 (6):501-7.
38. Sibert L, Charlin B, Corcos J, Gagnon R, Lechevallier J, Grise P. Assessment of clinical reasoning competence in urology with the script concordance test: an exploratory study across two sites from different countries. Eur Urol 2002;41 (3):227-33.
39. Gibot S, Bollaert PE. Le test de concordance de script comme outil d'évaluation formative en réanimation médicale. Pédagogie Médicale 2008;9 (1):7-18.
40. Fournier JP, Thiercelin D, Pulcini C, Alunni-Perret V, Gilbert E, Minguet JM, Bertrand F. Évaluation du raisonnement clinique en médecine d'urgence: les tests de concordance des scripts décèlent mieux l'expérience clinique que les questions à choix multiples à contexte riche. Pédagogie Médicale 2006;7 (1):20-30.
41. Charlin B, Brailovsky C, Brazeau-Lamontagne L, Samson L, Leduc C, van der Vleuten C. Script questionnaires: their use for assessment of diagnostic knowledge in radiology. Med Teach 1998;20 (6):567-71.
42. Sibert L, Giorgi R, Dahamna B, Doucet J, Charlin B, Darmoni SJ. Is a web-based concordance test feasible to assess therapeutic decision-making skills in a French context? Med Teach 2009;31 (4):162-8.
43. Lemay JF, Donnon T, Charlin B. The reliability and validity of a paediatric script concordance test with medical students, paediatric residents and experienced paediatricians. Can Med Educ J 2010;1 (2):89-95.
44. Charlin B, Desaulniers M, Gagnon R, Blouin D, van der Vleuten C. Comparison of an aggregate scoring method with a consensus scoring method in a measure of clinical reasoning capacity. Teach Learn Med 2002;14 (3):150-6.
45. Fournier JP, Staccini P, Ladner J, Roussel F, Gagnon R, Charlin B. Utilisation du test de concordance des scripts (TCS) pour la correction de l'épreuve de lecture critique d'article (LCA). Pédagogie Médicale 2009;10 (Suppl1):40.
46. Collard A, Gelaes S, Vanbelle S, Bredart S, Defraigne JO, Boniver J, Bourguignon JP. Reasoning versus knowledge retention and ascertainment throughout a problem-based learning curriculum. Med Educ 2009;43 (9):854-65.
47. Sibert L, Darmoni SJ, Dahamna B, Hellot MF, Weber J, Charlin B. Online clinical reasoning assessment with script concordance test in urology: results of a French pilot study. BMC Med Educ 2006;6:45.
48. Gagnon R, Charlin B, Roy L, St-Martin M, Sauve E, Boshuizen HP, van der Vleuten C. The cognitive validity of the script concordance test: a processing time study. Teach Learn Med 2006;18 (1):22-7.
49. Bland AC, Kreiter CD, Gordon JA. The psychometric properties of five scoring methods applied to the script concordance test. Acad Med 2005;80 (4):395-9.
50. Sibert L, Charlin B, Gagnon R, Corcos J, Khalaf A, Grise P. Evaluation of clinical reasoning in urology: contribution of the script concordance test. Prog Urol 2001;11 (6):1213-9.
51. Williams RG, Klamen DL, Hoffman RM. Medical student acquisition of clinical working knowledge. Teach Learn Med 2008;20(1):5-10.
52. Joly L, Braun M, Fournier JP, Benetos A. Test de concordance de script et apprentissage du raisonnement clinique en gériatrie: intérêt dans la formation en stage des étudiants hospitaliers. Pédagogie Médicale 2009;10 (Suppl 1):S39.
53. Joly L, Braun M, Benetos A. Apprentissage du raisonnement clinique et test de concordance de script en gériatrie: intérêt dans la formation en stage des étudiants hospitaliers. Rev Med Interne 2009;30 (Suppl2):135-6.
54. Khonputsa P, Besinque K, Fisher D, Gong WC. Use of script concordance test to assess pharmaceutical diabetic care: a pilot study in Thailand. Med Teach 2006;28 (6):570-3.
55. Demeester A. Evaluation du raisonnement clinique des étudiants sages-femmes par le test de concordance de script. Master's dissertation. Paris: Université Paris 13 2004.
56. Gagnon R, Charlin B, Lambert C, Carriere B, van der Vleuten C. Script concordance testing: more cases or more questions? Adv Health Sci Educ Theory Pract 2009;14 (3):367-75.
57. Bertrand C, Jabre P, Lecarpentier E, Margenet A, Combes X, Carriere B, Gagnon R, Charlin B, Claudepierre P, Farcet JP. Étude exploratoire du test de concordance de script pour l'évaluation certificative des médecins en télémédecine. Pédagogie Médicale 2009;10 (Suppl1):42.
58. Bursztejn AC, Cuny JF, Adam JL, Sido L, Schmutz JL, de Korwin JD, Latarche C, Braun M, Barbaud A. Test de concordance de script en dermatologie: évaluation du choix du panel d'experts. Pédagogie Médicale 2010;11 (Suppl1):69.
59. Charlin B, Gagnon R, Sauve E, Coletti M. Composition of the panel of reference for concordance tests: do teaching functions have an impact on examinees' ranks and absolute scores? Med Teach 2007;29 (1):49-53.
60. Sibert L, Charlin B, Corcos J, Gagnon R, Grise P, van der Vleuten C. Stability of clinical reasoning assessment results with the script concordance test across two different linguistic, cultural and learning environments. Med Teach 2002;24 (5):522-7.
61. Charlin B, Gagnon R, Carriere B, Lambert C. The Script Concordance Test as a Tool for Assessment in Context of Uncertainty: a Scoring Process Study. Dundee: Association for Medical Education in Europe 2006.
62. Kreiter CD, Bland AC, Gordon JA. Comparing Two Methods of Scoring a Script Concordance Test. Central Group on Educational Affairs, Association of American Medical Colleges, Spring Conference, 7-10th April 2005, Madison, WI.
63. Vanbelle S, Massart V, Giet D, Albert A. Test de concordance de script: un nouveau mode d'établissement des scores limitant l'effet du hasard. Pédagogie Médicale 2007;8 (2):71-81.
64. Gagnon R, Charlin B, Coletti M, Sauve E, van der Vleuten C. Assessment in the context of uncertainty: how many members are needed on the panel of reference of a script concordance test? Med Educ 2005;39 (3):284-91.
65. Holloway R, Nesbit K, Bordley D, Noyes K. Teaching and evaluating first and second year medical students' practice of evidence-based medicine. Med Educ 2004;38 (8):868-78.
66. Park AJ, Barber MD, Bent AE, Dooley YT, Dancz C, Sutkin G, Jelovsek JE. Assessment of intraoperative judgement during gynecologic surgery using the script concordance test. Am J Obstet Gynecol 2010;203 (3):240-6.
67. Schuwirth L, Colliver J, Gruppen L etal. Research in assessment: consensus statement and recommendations from the Ottawa 2010 Conference. Med Teach 2011;33 (3):224-33.
68. Fournier JP, Demeester A, Charlin B. Script concordance tests: guidelines for construction. BMC Med Inform Decis Mak 2008;8:18.
69. Downing SM. Validity: on meaningful interpretation of assessment data. Med Educ 2003;37 (9):830-7.
70. Charlin B, Gagnon R, Lubarsky S, Lambert C, Meterissian S, Chalk C, Goudreau J, Van der Vleuten C. Assessment in the context of uncertainty using the script concordance test: more meaning for scores. Teach Learn Med 2010;22 (3):180-6.
71. Duggan P, Charlin B. Summative assessment of 5th year medical students' clinical reasoning by script concordance test: requirements and challenges. BMC Med Educ (accepted).
72. Humbert AJ, Johnson MT, Miech E, Friedberg F, Grackin JA, Seidman PA. Assessment of clinical reasoning: a script concordance test designed for pre-clinical medical students. Med Teach 2011;6:472-7.
73. Hammick M, Dornan T, Steinert Y. Conducting a best evidence systematic review. Part 1: From idea to data coding. BEME Guide No. 13. Med Teach 2010;32 (1):3-15.