Quantitative Research Methods in Medical Education

Understanding Medical Education: Evidence, Theory and Practice: Second Edition

Volumn , Issue , 2013, Pages 349-369

  Norman, Geoff ^a   Eva, Kevin W ^b  

^a MCMASTER UNIVERSITY   (Canada)

^b UNIVERSITY OF BRITISH COLUMBIA   (Canada)

Author keywords

Experimental tradition; Medical education; Psychometric tradition; Quantitative research methods; Research designs

Indexed keywords

EID: 84988876299     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.1002/9781118472361.ch25     Document Type: Chapter

References (100)

1. Dolmans D (2003) The effectiveness of PBL: the debate continues. Some concerns about the BEME movement. Medical Education. 37: 1129-30.
2. Norman G (2002) Research in medical education: three decades of progress. British Medical Journal. 324: 1560-2.
3. Bordage G (2007) Moving the field forward: going beyond quantitative-qualitative. Academic Medicine. 82: S126-8.
4. Meltzoff J (1998) Research questions and hypotheses. In: Critical Thinking about Research, pp. 13-30. American Psychological Associa-tion, Washington, DC.
5. Bordage G (2009) Conceptual frameworks to illuminate and magnify. Medical Education. 43: 312-9.
6. Eva KW and Lingard L (2008) What's next? A guiding question for educators engaged in educational research. Medical Education. 42(8): 752-4.
7. Albert M, Hodges B and Regehr G (2007) Research in medical edu-cation: balancing service and science. Advances in Health Sciences Education: Theory and Practice. 12: 103-15.
8. Cook D, Beckman TJ and Bordage GE (2007) Quality of reporting of experimental studies in medical education: a systematic review. Medical Education. 41: 737-45.
9. Popper K (1959) The Logic of Scientific Discovery. Hutchinson and Co., London.
10. Norman GR (1999) The adult learner: a mythical species. Academic Medicine. 74: 886-9.
11. Regehr G (2010) It's NOT rocket science: rethinking our metaphors for research in health professions education. Medical Education. 44(1): 31-9.
12. Stokes DE (1997) Pasteur's Quadrant: Basic Science and Technological Innovation. Brookings Institution Press, Washington, DC.
13. Cook TD and Campbell DT (1979) Quasi-Experimentation: Design and Analysis for Field Settings. Rand McNally, Chicago, IL.
14. Papadakis MA, Teherani A, Banach MA et al. (2005) Disciplinary action by medical boards and prior behavior in medical school. New England Journal of Medicine. 353: 2673-82.
15. Hojat M, Paskin DL, Callahan CA et al. (2007) Components of post-graduate competence: analyses of thirty years of longitudinal data. Medical Education. 41: 982-9.
16. Rose G (2001) Sick individuals and sick populations. International Journal of Epidemiology. 30: 427-32.
17. Norman GR (2003) RCT = results confounded and trivial. Medical Education. 37: 582-4.
18. Schuwirth LW and van der Vleuten CP (2006) Challenges for edu-cationalists. British Medical Journal. 333: 544-6.
19. Norman GR, Eva KW and Schmidt HG (2005) Implications of psychology-type theories for full curriculum interventions. Medical Education. 39: 247-9.
20. Campbell DT and Stanley JC (1963) Experimental and Quasi-Experimental Designs for Research on Teaching. Rand McNally, Chicago, IL.
21. Goldacre B (2007) A kind of magic? (http://www.guardian.co.uk/science/2007/nov/16/sciencenews.g2; accessed 16 July 2013).
22. Norman GR and Shannon SI (1998) Effectiveness of instruction in critical appraisal (evidence-based medicine) skills: a critical appraisal. Canadian Medical Association Journal. 158: 177-81.
23. Schmidt HG and Moust JHC (1995) What makes a tutor effective? A structural equations modeling approach to learning in problem-based curricula. Academic Medicine. 70: 708-14.
24. Lipsey MW and Wilson DB (1993) The efficacy of psychological, educational, and behavioral treatment. Confirmation from meta-analysis. American Psychologist. 48: 1181-209.
25. Meyer GJ, Finn SE, Eyde LD et al. (2001) Psychological testing and psychological assessment: a review of evidence and issues. Ameri-can Psychologist. 56: 128-65.
26. Woodward CA, Cohen M, Ferrier BM et al. (1989) Correlates of certification in family medicine in the billing patterns of Ontario general practitioners. Canadian Medical Association Journal. 141: 897-904.
27. Koh GC-H, Khoo HE, Wong ML and Koh D (2008) The effects of problem-based learning during medical school on physician com-petency: a systematic review. Canadian Medical Association Journal. 178: 34-41.
28. Colliver JA and Markwell SJ (2007) Research on problem-based learning: the need for critical analysis of methods and findings. Medical Education. 41: 533-5.
29. Schmidt HG, Rotgans JI and Yew EH (2011) The process of problem-based learning: what works and why. Medical Education. 45: 792-806.
30. Cook DA (2005) The research we still are not doing: an agenda for the study of computer-based learning. Academic Medicine. 80: 541-8.
31. Ehrenberg RG, Brewer DJ, Gamoran A and Willms JD (2001) Does class size matter?. Scientific American. 285: 79-84.
32. Norman GR and Streiner DL (2007) Biostatistics: The Bare Essentials (3e). BC Decker, Hamilton, ON.
33. Larsen DP, Butler AC and Roediger HL (2008) Test-enhanced learn-ing in medical education. Medical Education. 42: 959-66.
34. Lurie SJ, Mooney CJ and Lyness JM (2009) Measurement of the general competencies of the accreditation council for graduate medical education: a systematic review. Academic Medicine. 84: 301-9.
35. Whitcomb ME (2002) Research in medical education: what do we know about the link between what doctors are taught and what they do?. Academic Medicine. 77: 1067-8.
36. Bligh J (2003) Editorial. Medical Education. 37: 184-5.
37. Colliver JA (2003) The research enterprise in medical education. Teaching and Learning in Medicine. 15: 154-5.
38. Norman GR (2006) Outcomes, objectives and the seductive appeal of simple solutions. Advances in Health Sciences Education. 11: 217-20.
39. Gruppen L (2007) Improving medical education research. Teaching and Learning in Medicine. 19: 331-5.
40. Eva KW, Reiter HI, Trinh K, Wasi P, Rosenfeld J and Norman GR (2009) Predictive validity of the multiple mini-interview for select-ing medical trainees. Medical Education. 43(8): 767-75.
41. Tamblyn R, Abrahamowicz M, Dauphinee D et al. (2007) Physician scores on a national clinical skills examination as predictors of com-plaints to medical regulatory authorities. Journal of the American Medical Association. 298: 993-1001.
42. Werner LS and Bull BS (2003) The effect of three commercial coach-ing courses on Step One USMLE performance. Medical Education. 37: 527-31.
43. Cohen PA (1981) Student rating of instruction and student achieve-ment: a meta-analysis of multisection validity studies. Review of Educational Research. 51: 281-309.
44. Eva KW and Regehr G (2005) Self-assessment in the health profes-sions: a reformulation and research agenda. Academic Medicine. 80: S46-54.
45. Gordon MJ (1991) A review of the validity and accuracy of self-assessments in health professions training. Academic Medicine. 66: 762-9.
46. Davis DA, Mazmanian PE, Fordis M et al. (2006) Accuracy of physi-cian self-assessment compared with observed measures of compe-tence: a systematic review. Journal of the American Medical Association. 296: 1094-102.
47. Peterson LN, Eva KW, Rusticus SA and Lovato CY (2012) The readi-ness for clerkship survey: can self-assessment data be used to evalu-ate program effectiveness?. Academic Medicine. 87: 1355-60.
48. Levinson AJ, Weaver B, Garside S et al. (2007) Virtual reality and brain anatomy: a randomised trial of e-learning instructional designs. Medical Education. 41: 495-501.
49. Cohen JJ (1988) Statistical Power Analysis for the Behavioral Sciences. Erlbaum, Hillsdale, NJ.
50. Colliver JA, Markwell SJ, Verhulst SJ and Robbs RS (2007) The prognostic value of documented unprofessional behavior in medical school records for predicting and preventing subsequent medical board disciplinary action: the Papadakis studies revisited. Teaching and Learning in Medicine. 19: 213-5.
51. Swanson DB, Norman GR and Linn RL (1995) Performance based assessment: lessons from the professions. Educational Research. 24: 5-11.
52. Harden RM and Gleeson FA (1979) Assessment of clinical compe-tence using an Objective Structured Clinical Examination (OSCE). Medical Education. 13: 41-54.
53. Streiner DL and Norman GR (2003) Health Measurement Scales: A Practical Guide to Their Development and Use (3e). Oxford University Press, Oxford.
54. Eva KW, Rosenfeld J, Reiter HI and Norman GR (2004) An admissions OSCE: the multiple mini-interview. Medical Education. 38: 314-26.
55. Hodgson CS, Teherani A, Gough HG et al. (2007) The relationship between measures of unprofessional behavior during medical school and indices on the California Psychological Inventory. Aca-demic Medicine. 82: S4-7.
56. Mamede S and Schmidt HG (2004) The structure of refl ective prac-tice in medicine. Medical Education. 38: 1302-8.
57. Kane MT (2006) Validation. In: Brennan RL (ed.) Educational Meas-urement. Praeger Publishers, Westport, CT.
58. Messick S (1989) Validity. In: Linn RL (ed.) Educational Measurement, pp. 103-4. Macmillan, New York.
59. Downing SM (2003) Validity on the meaningful interpretation of assessment data. Medical Education. 37: 830-7.
60. Newble DI and Jaeger K (1983) The effect of assessments and exami-nations on the learning of medical students. Medical Education. 17: 165-71.
61. Harasym PH, Woloschuk W, Mandin H and Brundin-Mather R (1996) Reliability and validity of interviewers' judgements of medical school candidates. Academic Medicine. 71: S40-2.
62. Paul A (2004) The Cult of Personality: How Personality Tests are Leading Us to Miseducate Our Children, Mismanage Our Companies, and Mis-understand Ourselves. Free Press, New York.
63. Hodges B, Turnbull J, Cohen R et al. (1996) Evaluating communica-tion skills in the OSCE format: reliability and generalizability. Medical Education. 30: 38-43.
64. Hodges B, Regehr G, McNaughton N et al. (1999) OSCE checklists do not capture increasing levels of expertise. Academic Medicine. 74: 1129-34.
65. Eva KW and Hodges BD (2012) Scylla or Charybdis? Can we navi-gate between objectification and judgement in assessment? Medical Education. 46: 914-9.
66. Ramsey PG, Carline JD, Inui TS et al. (1989) Predictive validity of certification by the American Board of Internal Medicine. Annals of Internal Medicine. 110: 719-26.
67. Norman GR (2004) The morality of medical school admissions. Advances in Health Sciences Education. 9: 79-82.
68. Bargh JA and Chartrand TL (1999) The unbearable automaticity of being. American Psychologist. 54: 462-79.
69. Wainer H (1976) Estimating coefficients in linear models: it don't make no nevermind. Psychological Bulletin. 83: 213-7.
70. Jamieson S (2004) Likert scales: how to (ab)use them. Medical Educa-tion. 38: 1217-8.
71. Glass GV and Stanley JC (1970) Statistical Methods in Education and Psychology. Prentice Hall, Englewood Cliffs, NJ.
72. Carifio J and Perla R (2008) Resolving the 50-year debate around using and misusing Likert scales. Medical Education. 42: 1150-2.
73. Norman G (2010) Likert scales, levels of measurement, and the "laws" of statistics. Advances in Health Sciences Education. 15: 625-32.
74. Wikipedia (2010) Bonferroni correction. (http://en.wikipedia.org/wiki/Bonferroni_correction; accessed 16 July 2013).
75. Cronbach LJ (1957) The two disciplines of scientific psychology. American Psychologist. 12: 671-84.
76. Harden R, Grant J, Buckley G and Hart I (2000) Best evidence medical education. Advances in Health Sciences Education. 5: 71-90.
77. Smith ML and Glass GV (1977) Meta-analysis of psychotherapy outcome studies. American Psychologist. 32: 752-60.
78. Vernon DT and Blake RL (1993) Does problem-based learning work? A meta-analysis of evaluative research. Academic Medicine. 68: 550-63.
79. Hamdy H, Prasad K, Anderson MB et al. (2006) BEME guide no. 5: predictive values of assessment measurements obtained in medical schools and future performance in practice. Medical Teacher. 28: 103-16.
80. Eva KW (2009) Broadening the debate about quality in medical education research. Medical Education. 43: 294-6.
81. Eva KW (2012) The state of science 2012: building blocks for the future. Medical Education. 46: 1-2.
82. Bordage G (2001) Reasons reviewers reject and accept manuscripts: the strengths and weaknesses in medical education reports. Aca-demic Medicine. 76: 889-96.
83. Driessen E, van Tartwijk J, van der Vleuten C and Wass V (2007) Portfolios in medical education: why do they meet with mixed success? A systematic review. Medical Education. 41: 1224-33.
84. Issenberg SB, McGaghie WC, Petrusa ER et al. (2005) Features and uses of high-fi delity simulation that lead to effective learning: a BEME systematic review. Medical Teacher. 27: 10-28.
85. Dornan T, Littlewood S, Margolis S et al. (2006) How can experience in clinical and community settings contribute to early medical edu-cation? A BEME systematic review. Medical Teacher. 28: 3-18.
86. Hammick M, Freeth D, Koppel I et al. (2007) BEME guide no. 9: a best evidence systematic review of interprofessional education. Medical Teacher. 29: 735-51.
87. Albanese MA and Mitchell S (1993) Problem-based learning: a review of literature on its outcomes and implementation issues. Academic Medicine. 68: 52-81.
88. Norman GR and Schmidt HG (1992) The psychological basis of problem-based learning. Academic Medicine. 67: 557-65.
89. Eva KW (2008) On the limits of systematicity. Medical Education. 42: 852-3.
90. Cook DA, Hatala R, Brydges R et al. (2011) Technology -enhanced simulation for health professions education: a systematic review and meta-analysis. JAMA: The Journal of the American Medical Asso-ciation. 306: 978-88.
91. Cook DA and West CP (2012) Conducting systematic reviews in medical education: a stepwise approach. Medical Education. 46: 943-52.
92. Garg A, Norman GR, Spero L and Maheshwari P (1999) Do virtual computer models hinder anatomy learning?. Academic Medicine. 74: S87-9.
93. Cronbach LJ (1975) Beyond the two disciplines of scientific psychol-ogy. American Psychologist. 30: 116-27.
94. Schmidt HG (2005) Influence of research on practices in medical educa-tion: the case of problem-based learning. Paper presented at the Associa-tion for Medical Education in Europe, Amsterdam, The Netherlands, September.
95. Cook DA, Bordage G and Schmidt HG (2008) Description, justifica-tion and clarification: a framework for classifying the purposes of research in medical education. Medical Education. 42: 128-33.
96. Bowling A (1997) Research Methods in Health: Investigating Health and Health Services. Open University Press, Philadelphia.
97. Chalmers A (1999) What is this Thing Called Science? (3e). Open Univer-sity Press, Buckingham.
98. Norman GR, van der Vleuten CPM and Newble DI (eds) (2002) Inter-national Handbook of Research in Medical Education (vol. 7). Springer International Handbooks of Education, New York.
99. Norman GR and Streiner DL (2007) Biostatistics: The Bare Essentials (3e). BC Decker, Hamilton, ON.
100. Streiner D and Norman GR (2003) Health Measurement Scales: A Practical Guide to Their Development and Use (3e). Oxford University Press, Oxford.