Evaluating construct equivalence and criterion-related validity for repeat examinees on a standardized patient examination

Academic Medicine

Volumn 86, Issue 10, 2011, Pages 1253-1259

  Raymond, Mark R ^a   Kahraman, Nilufer ^a   Swygert, Kimberly A ^a   Balog, Kevin P ^a  

^a NATIONAL BOARD OF MEDICAL EXAMINERS   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 80053471973     PISSN: 10402446     EISSN: None     Source Type: Journal    
DOI: 10.1097/ACM.0b013e31822bc0a4     Document Type: Article

References (18)

1. Messick S. Standards of validity and the validity of standards in performance assessment. Educ Meas Issues Pract. 1995;14: 5-8.
2. De Champlain A, Swygert KA, Swanson DB, Boulet JR. Assessing the underlying structure of the USMLE Step 2 test of clinical skills using confirmatory factor analysis. Acad Med. 2006;81(10 suppl):S17-S20. http://journals.lww.com/academicmedicine/Fulltext/2006/10001/ Assessing-the-Underlying-Structure-of-the-United.6.aspx. Accessed June 10, 2011.
3. Harik P, Clauser BE, Grabovsky I, Margolis MJ, Dillon GF, Boulet JR. Relationships among subcomponents of the USMLE Step 2 clinical skills examination, the Step 1, and the Step 2 clinical knowledge examinations. Acad Med. 2006;81(10 suppl):S21-S24. http://journals.lww.com/academicmedicine/ Fulltext/2006/10001/Relationships-among-Subcomponents-of-the-USMLE.7.aspx. Accessed June 10, 2011.
4. Anastasi A. Coaching, test sophistication, and developed abilities. Am Psychol. 1981;36: 1086-1093.
5. Rowland-Morin PA, Burchard KW, Garb JL, Coe NPW. Influence of effective communication by surgery students on their oral examination scores. Acad Med. 1991;66:169 -171. http://journals.lww.com/academicmedicine/Abstract/1991/03000/ Influence-of-effective-communication-by-surgery.11.aspx. Accessed June 10, 2011.
6. Raymond MR, Luciw-Dubas UA. The second time around: Accounting for retest effects on oral examinations. Eval Health Prof. 2010;33: 386-403.
7. Swygert KA, Balog KP, Jobe A. The impact of repeat information on examinee performance for a large-scale standardized-patient examination. Acad Med. 2010;85:1506 -1510. http://journals.lww.com/academicmedicine/Fulltext/2010/ 09000/The-Impact-of-Repeat-Information-on-Examinee.25.aspx. Accessed June 10, 2011.
8. Hausknecht JP, Halpert JA, Di Paolo NT, Gerrard MO. Retesting in selection: A metaanalysis of practice effects for tests of cognitive ability. J Appl Psychol. 2007;92: 373-385.
9. Lievens F, Reeve CL, Heggestad ED. An examination of psychometric bias due to retesting on cognitive ability tests in selection settings. J Appl Psychol. 2007;92:1672-1682.
10. Raymond MR, Neustel S, Anderson D. Same-form retest effects on credentialing examinations. Educ Meas Issues Pract. 2009; 28:19-27.
11. Boulet JR, McKinley DW, Whelan GP, Hambleton RK. The effect of task exposure on repeat candidate scores in high-stakes standardized patient assessment. Teach Learn Med. 2003;15:227-232. (Pubitemid 37414471)
12. Niehaus AH, DaRosa DA, Markwell SJ, Folse R. Is test security a concern when OSCE stations are repeated across clerkship rotations? Acad Med. 1996;71:287-289. http://journals.lww.com/academicmedicine/Abstract/1996/03000/ Is-test-security-a-concern-when-OSCE-stations-are.23.aspx. Accessed June 10, 2011. (Pubitemid 26089609)
13. Swartz MH, Colliver JA, Cohen DS, Barrows HS. The effect of deliberate, excessive violations of test security on performance on a standardized patient examination. Acad Med. 1993;68(10 suppl):S76 -S78. http://journals.lww.com/ academicmedicine/Abstract/1993/10000/The-effect-of-deliberate, -excessive-violations-of.52.aspx. Accessed June 10, 2011. (Pubitemid 23317466)
14. Muthén LK, Muthén BO. Mplus User's Guide. 4th ed. Los Angeles, Calif: Muthen & Muthen; 2006.
15. Bentler PM. Comparative fit indexes in structural models. Psychol Bull. 1990;107: 238-246.
16. Clauser BE, Nungester RJ. Classification accuracy for tests that allow retakes. Acad Med. 2001;76(10 suppl):S108-S110. http://journals.lww.com/ academicmedicine/Fulltext/2001/10001/Classification-Accuracy-for-Tests-That- Allow.36.aspx. Accessed June 10, 2011. (Pubitemid 32954281)
17. Barzansky B, Etzel SI. Educational programs in the U.S. medical schools, 2003-2004. JAMA. 2004;292:1025-1031. (Pubitemid 39129188)
18. Ramineni C, Harik P, Margolis MJ, Clauser BE, Swanson DB, Dillon GF. Sequence effects in the United States Medical Licensing Examination (USMLE) Step 2 clinical skills (CS) examination. Acad Med. 2007;82(10 suppl):S101-S104. http://journals.lww.com/academicmedicine/Fulltext/2007/10001/ Sequence-Effects-in-the-United-States-Medical.27.aspx. Accessed June 10, 2011. (Pubitemid 47482563)