Unproctored internet test verification: Using adaptive confirmation testing

Organizational Research Methods

Volumn 14, Issue 4, 2011, Pages 608-630

  Makransky, Guido ^a,b   Glas, Cees A W ^b  

^a Master Management International A/S   (Denmark)

^b UNIVERSITY OF TWENTE   (Netherlands)

Author keywords

computer adaptive testing; item response theory; unproctored Internet testing

Indexed keywords

EID: 80052817506     PISSN: 10944281     EISSN: 15527425     Source Type: Journal    
DOI: 10.1177/1094428110370715     Document Type: Article

References (45)

1. Anderson C. D.,Warner J. L.,Spector C. E.Inflation bias in self-assessment examination: Implications for valid employee selection.Journal of Applied Psychology. 1984;69:574-580.
2. Angoff W. H.The development of statistical indices for detecting cheaters.Journal of the American Statistical Association. 1974;69:44-49.
3. Arthur W.,Glaze R. M.,Villado A. J.,Taylor J. E.Unproctored Internet-based tests of cognitive ability and personality: Magnitude of cheating and response distortion.Industrial and Organizational Psychology: Perspectives on Science and Practice. 2009;2:39-45.
4. Automatic Data Processing Inc. (2008). 2008 ADP screening index. Retrieved December 21, 2009, fromwww.adp.com/media/press-releases/2008-news-releases/adp-annual-pre-employment-screening-index.aspx.
5. Beaty J. C.,Dawson C. R.,Fallaw S. S.,Kantrowitz T. M.Recovering the scientist-practitioner model: How I-Os should respond to unproctored Internet testing.Industrial and Organizational Psychology: Perspectives on Science and Practice. 2009;2:58-63.
6. Bellezza F. S.,Bellezza S. F.Detection of cheating on multiple-choice tests by using error-similarity analysis.Teaching of Psychology. 1989;16:151-155.
7. Birnbaum A.Statistical theories of mental test scores. Lord F. M.Novick M. R., ed. Reading, MA: Addison-Wesley; 1968:395-479.
8. Burke E.,van Someren G.,Tatham NVerify range of ability tests: Technical manual. Thames Dittion, Surrey: SHL Group; 2006:.
9. Chapman D. S.,Webster J.The use of technologies in the recruiting, screening, and selection processes for job candidates.International Journal of Selection and Assessment. 2003;11:113-120.
10. Cizek G. J.Cheating on tests: How to do it, detect it, and prevent it. Mahwah, NJ: Lawrence Erlbaum; 1999:.
11. Drasgow F.,Levine M. V.,Zickar M. J.Optimal detection of mismeasured individuals.Applied Measurement in Education. 1996;9:47-64.
12. Eggen T. J. H. M.Item selection in adaptive testing with the sequential probability ratio test.Applied Psychological Measurement. 1999;23:249-261.
13. Eggen T. J. H. M,Straetmans G. J. J. M.Computerized adaptive testing for classifying examinees into three categories.Educational and Psychological Measurement. 2000;60:713-734.
14. Fallaw S. S.,Solomonson A. L.,McClelland L.(2009). Current trends in assessment use: A multiorganizational survey.Poster submitted for the 24th Annual Conference of the Society for Industrial and Organizational Psychology; 2009New Orleans, LA; 2009.
15. Finkelman MAn Adaptation of Stochastic Curtailment to Truncate Wald's SPRT in Computerized Adaptive Testing. Los Angeles, CA: National Center for Research on Evaluation, Standards, and Student Testing (CRESST); 2003:.
16. Finkelman M.On using stochastic curtailment to shorten the SPRT in sequential mastery testing.Journal of Educational and Behavioral Statistics. 2008;33:442-463.
17. Foster D.Secure, online, high-stakes testing: Science fiction or business reality?.Industrial and Organizational Psychology: Perspectives on Science and Practice. 2009;2:31-34.
18. Glas C. A. W.,van der Linden W. J.Computerized adaptive testing with item cloning.Applied Psychological Measurement. 2003;27:249-263.
19. Glas C. A. W.,van der Linden W. J.,Geerlings H.Elements of adaptive testing. van der Linden W. J.Glas C. A. W., ed. New York, NY: Springer; 2010:303-331.
20. International Guidelines on Computer-Based and Internet Delivered Testing (2005). Retrieved January 5, 2010, fromwww.psychtesting.org.uk/downloadfile.cfm?file_uuid=9BD783C8-1143-DFD0-7E98-798CD61E4F00&ext=pdf.
21. Jennison C.,Turnbull B. W.Group sequential methods with applications to clinical trials. Boca Raton, FL: Chapman & Hall/CRC; 2000:.
22. Jiao, H., Wang, S., & Lau, C. A. (2004). An investigation of two combination procedures of SPRT for threecategory classification decisions in computerized classification test. Paper presented at the annual meeting of the American Educational Research Association, San Antonio, April 2004
23. Kaminski K. A.,Hemingway M. A.To proctor or not to proctor? Balancing business needs with validity in online assessment.Industrial and Organizational Psychology: Perspectives on Science and Practice. 2009;2:24-26.
24. Kingsbury, G. G. (2009). Adaptive item calibration: A simple process for estimating item parameters within a computerized adaptive test. Paper presented at the 2009 GMAC conference on computerized adaptive testing, Minneapolis, Minnesota
25. LaHuis D. M.,Copeland D.Investigating faking using a multilevel logistic regression approach to measuring person fit.Organizational Research Methods. 2009;12:296-319.
26. Luecht R. M.Handbook of test development. Downing S. M.Haladyna T. M., ed. Mahwah, NJ: Lawrence Erlbaum; 2006:575-596.
27. Makransky G.,Glas C. A. W.An automatic online calibration design in adaptive testing.Journal of Applied Testing Technology. 2010;11:.
28. Muraki E.,Bock R. DChicago, IL: Scientific Software; 1996:.
29. Nye C. D.,Do B.,Drasgow F.,Fine S.Two-step testing in employee selection: Is score inflation a problem?.International Journal of Selection and Assessment. 2008;16:112-120.
30. Ones D. S.,Viswesvaran C.,Reiss A. D.Role of social desirability in personality testing for personnel selection: The red herring.Journal of Applied Psychology. 1996;81:660-679.
31. Partchev I.3PL: A useful model with a mild estimation problem.Measurement. 2009;7:94-96.
32. Pearlman K.Unproctored Internet testing: Practical, legal, and ethical concerns.Industrial and Organizational Psychology: Perspectives on Science and Practice. 2009;2:14-19.
33. Rasch, G. (1960). Probabilistic models for some intelligence and attainment tests. Copenhagen, Denmark: Danish Institute for Educational Research.
34. Reckase, M. D. (1983). A procedure for decision making using tailored testing. In D. J. Weiss (Ed.), New horizons in testing: Latent trait theory and computerized adaptive testing (pp. 237-254). New York, NY: Academic Press.
35. Rudner L. M.(2002).An examination of decision-theory adaptive testing procedures.Paper presented at the annual meeting of the American Educational Research Association, April 1-5, 2002; 2002New Orleans, LA; 2002.
36. Schmidt F. L.,Hunter J. E.The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings.Psychological Bulletin. 1998;124:262-274.
37. Spray, J. A., & Reckase, M. D. (1994). The selection of test items for decision making with a computerized adaptive test. Paper presented at the Annual Meeting of the National Council for Measurement in Education, April 5-7, 1994, New Orleans, LA.
38. Spray J. A.,Reckase M. D.Comparison of SPRT and sequential Bayes procedures for classifying examinees into two categories using a computerized test.Journal of Educational & Behavioral Statistics. 1996;21:405-414.
39. Thissen D.,Mislevy R. J.(2000). Test algorithms. In H. Wainer (Ed.),Computerized adaptive testing: A primer. Wainer H., ed. Hillsdale, NJ: Lawrence Erlbaum; 2000:101-134.
40. Tippins N. T.Internet alternatives to traditional proctored testing: Where are we now?.Industrial and Organizational Psychology: Perspectives on Science and Practice. 2009;2:2-10.
41. van der Linden W. J.,(2002).A statistical test for detecting answer copying on multiplechoice tests.Sotaridona L.University of Twente Research Report 02-04; 2002Enschede, the Netherlands; 2002.
42. van Krimpen-Stoop, E. M. L. A., & Meijer, R. R. (2000). Detecting person misfit in adaptive testing using statistical process control techniques. In W. J. van der Linden & C. A. W. Glas (Eds.), Computerized adaptive testing: Theory and practice (pp. 201-219). Boston, MA: Kluwer-Nijhoff
43. Warm T. A.Weighted likelihood estimation of ability in item response theory.Psychometrika. 1989;54:427-450.
44. Whitley B. E.Factors associated with cheating among college students: A review.Research in Higher Education. 1998;39:235-274.
45. Wu M. L.,Adams R. J.,Wilson M. R.,Haldane SCamberwell, Australia: ACER; 2007:.