Ensuring reliability in UK written tests of general practice: The MRCGP examination 1998-2003

Medical Teacher

Volumn 27, Issue 1, 2005, Pages 37-45

  Munro, Neil ^a   Denney, Mei Ling ^a   Rughani, Amar ^a   Foulkes, John ^a   Wilson, Andrew ^a   Tate, Peter ^a  

^a NONE

Author keywords

[No Author keywords available]

Indexed keywords

ACCREDITATION; ARTICLE; COMPETENCE; CORONER; CORRELATION COEFFICIENT; DECISION MAKING; EDUCATION PROGRAM; EDUCATIONAL TECHNOLOGY; GENERAL PRACTICE; HEALTH CARE QUALITY; INTELLIGENCE TEST; MEDICAL EDUCATION; MEDICAL PRACTICE; PROBLEM SOLVING; QUALITY CONTROL; QUESTIONNAIRE; RATING SCALE; RELIABILITY; REPRODUCIBILITY; SCORING SYSTEM; UNITED KINGDOM; VALIDATION PROCESS;

EDUCATION, MEDICAL, GRADUATE; EDUCATIONAL MEASUREMENT; FAMILY PRACTICE; GREAT BRITAIN; HUMANS; QUALITY CONTROL; WRITING;

EID: 17144380371     PISSN: 0142159X     EISSN: None     Source Type: Journal    
DOI: 10.1080/01421590400013461     Document Type: Article

References (42)

1. Barnett, R. (1989) How Shall We Assess Them?, Information Services, Discussion Paper 1 (London, Council for National Academic Awards)
2. Belton, A. (1986) How to increase your trainee pass rate in the MRCGP examination, Journal of the Association of Course Organizers 2, pp. 56-64.
3. Berkowitz, D., Wolkowitz, B., Fitch, R. & Kopriva, R. (2000) The Use of Tests as Part of high-Stakes Decision for Students: A Resources Guide for Educators and Policy-Makers (Washington, DC, US Department of Education).
4. Biggs, J. (1999) Teaching for Quality at University (Buckingham, SRHE/Open University Press).
5. Bloom, B.S. (1956) Taxonomy of Educational Objectives: Handbook 1, Cognitive Domain (New York, D. Mckay).
6. Braddock, R., Lloyd-Jones, R. & Schoer, L. (1963) Research in Written Composition (Urbana, IL, National Council of Teachers of English).
7. Brennan, R.L. (1983) Elements of Generalisability Theory (Iowa, ACT Publications).
8. Chase, C.I. (1986) The impact of some obvious variables on essay test scores, Journal of Educational Measurement, 23, pp. 33-42.
9. Cronbach, L.J. (1951) Co-efficient alpha and the internal structure of tests, Psychometrika, 16, pp. 292-334.
10. Croosley, J., Davies, H., Humphris, G. & Jolly, B. (2002) Generalisability: a key to unlock professional assessment, Medical Education, 36, pp. 972-978.
11. Feldt, L.S. & Brennan, R.L. (1989) Reliability, in: R.L. Linn (Ed.) Educational Measurement, 3rd edn (Washington, DC, American Council on Education/Macmillan Series on Higher Education).
12. Forster, M. & Masters, G. (1996) Portfolios (Victoria, Australian Council for Educational Research).
13. French, J.W. (1966) Schools of thought in judging excellence of English themes, in: A. Anastasi (Ed.) Testing Problems in Perspective (Washington, DC, American Council on Education).
14. Frijns, P.H.A.M., Van Der Vleuten, C.P.M., Verwijnen, G.M. & Van Leeuwen, T.D. (1990) The effect of structure in scoring methods on the reproducibility of tests using open ended questions, in: W. Bender, R.J. Hiemstra, A.J.J.A. Scherbier & R.P. Zwierstra (Eds) Teaching and Assessing Clinical Competence (Amsterdam, Boekwerk), pp. 466-471.
15. Fry, J., Fawley, J.H., Pinsent, R.J.F.H. (Eds) (1983) A History of the Royal College of General Practitioners: the First 25 Years (Lancaster, MTP Press).
16. Gorman, T.P., White, J., Orchard, L. et al. (1983) Language Performance in School: Secondary Survey Report No 2 (London, HMSO).
17. Green, J.A. (1975) Teacher Made Tests, 2nd edn (London, Harper & Row).
18. Hartog, P. & Rhodes, E.C. (1936) The Marks of Examiners (London, Macmillan).
19. Hyland, T. (1994) Competence, Education and NVQs: Dissenting Perspectives (London, Cassell Education).
20. Jolly, B. (2000) "The Professor has submitted her questions and thus we may proceed to set the examination paper," Medical Education, 34, pp. 7-8.
21. Kuisma, R. (1999) Criteria referenced marking of written assignments, Assessment & Evaluation in Higher Education, 24, p. 27.
22. Laduca, A. (1994) Validation of professional licensure examinations: professions theory, test design, and construct validation, Evaluation and the Health Professions, 17, pp. 178-197.
23. Macnamara, J. & Madaus, G.F. (1969) Marker reliability in the Irish leaving certificate, Irish Journal of Education, 3, pp. 15-21.
24. Marsh, H.W. & Ireland, R. (1987) The assessment of writing effectiveness: a multidimensional perspective, Australian Journal of Psychologv, 39, pp. 353-367.
25. Mitchelmore, M.C. (1981) Reporting student achievement: how many grades?, British Journal of Educational Psychology, 51, pp. 218-227.
26. Mulholland, H. & McAleer, S. (1988) Report to the Examination Board of the Royal College of General Practitioners. Dundee, Centre for Medical Education, unpublished.
27. Munro, N., Rughani, A., Foulkes, J., Wilson, A. & Neighbour, R. (2000) Assessing validity in written tests of general practice: exploration by factor analysis of candidate response patterns to paper I of the MRCGP examination, Medical Education, 34, pp. 35-41.
28. Murphy, R.J.L. (1978) Reliability of marking in eight GCE examinations, British Journal of Education and Psychology, 48, pp. 196-200.
29. Murphy, R.J.L. (1982) A further report into the reliability of marking of GCE examinations, British Journal of Educational Psychology, 52, pp. 58-63.
30. Newble, D.L. & Entwistle, N.J. (1986) Learning styles and approaches: implications for medical education, Medical Education, 20, pp. 162-175.
31. Norcini, J.J., Swanson, D.B., Grosso, L.J. et al. (1985) Reliability, validity and efficiency of various item formats in assessment of physician competence, Medical Education, 19, pp. 238-247.
32. Nuttall, D.L. & Willmott, A.S. (1972) British Examinations: Techniques of analysis P12 (Slough, NFER Publishing).
33. Pilliner, A.E.G. (1969) Estimation of number of grades to be awarded in one examination by consideration of its reliability coefficient, internal research paper, Godfrey Thompson Unit, Edinburgh.
34. Rafoth, B.A. & Rubin, D.L. (1984) The impact of content and mechanics on judgments of writing quality, Written communication, 1, pp. 446-458.
35. Southgate, L. & Grant, J. (2003) Principles and standards for an assessment system for postgraduate medical training, Joint Statement by Postgraduate Medical Education and Training Board and Academy of the Medical Royal Colleges.
36. Swanson, D.B. (1987) Review of membership examination procedures of the Royal College of General Practitioners, unpublished.
37. The General and Specialist Medical Practice (Education, Training and Qualifications) Order 2003, Statutory Instrument 2003 No. 1250.
38. Van Der Vleuten, C.P.M. (1996) The assessment of professional competence: developments, research and practical implications, Advances in Health Science Education, 1, pp. 41-67.
39. Verma, M., Chantwal, J. & Singh, T. (1997) Reliability of essay type questions - effect of structuring, Assessment in Education 4, pp. 265-270.
40. Wass, V., McGibbon, D. & Van Der Vleuten, C. (2001) Composite undergraduate examinations: how should components be combined to maximize reliability?, Medical Education, 35, pp. 326-330.
41. Willmott, A.S. & Nuttall, D.L. (1975) The reliability of examinations at 16+ (London, Macmillan Education Ltd).
42. Wood, R. (1991) Assessment & Testing: A Survey of research (Cambridge, Cambridge University Press), pp. 144-145.