Proctored Versus Unproctored Internet Tests: Are unproctored noncognitive tests as predictive of job performance?

International Journal of Selection and Assessment

Volumn 19, Issue 1, 2011, Pages 1-10

  Beaty, James C ^a   Nye, Christopher D ^b   Borneman, Matthew J ^a,c   Kantrowitz, Tracy M ^a   Drasgow, Fritz ^b   Grauer, Eyal ^d  

^a PreVisor   (United States)

^b UNIVERSITY OF ILLINOIS AT URBANA CHAMPAIGN   (United States)

^c SOUTHERN ILLINOIS UNIVERSITY   (United States)

^d Applied Psychological Techniques   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 79951564010     PISSN: 0965075X     EISSN: 14682389     Source Type: Journal    
DOI: 10.1111/j.1468-2389.2011.00529.x     Document Type: Article

References (41)

1. AERA, APA, & NCME. (1999). Standards for educational and psychological testing. Washington, DC: American Educational Research Association, American Psychological Association, National Council on Measurement in Education.
2. Arthur, W., Glaze, R. M., Villado, A. J., & Taylor, J. E. (2009). Unproctored internet-based tests of cognitive ability and personality: Magnitude of cheating and response distortion. Industrial and Organizational Psychology: Perspectives on Science and Practice, 2, 39-45.
3. Barrick, M. R., & Mount, M. K. (1991). The big five personality dimensions and job performance: A meta-analysis. Personnel Psychology, 44, 1-26.
4. Bartram, D., & Brown, A. (2004). Online testing: Mode of administration and the stability of OPQ 32i scores. International Journal of Selection and Assessment, 12, 278-284.
5. Bauer, T. N., Truxillo, D. M., Tucker, J. S., Weathers, V., Bertolino, M., & Erdogan, B., et al. (2006). Selection in the information age: The impact of privacy concerns and computer experience on applicant reactions. Journal of Management, 32, 601-621.
6. Beaty, J. C., Dawson, C. R., Fallaw, S. S., & Kantrowitz, T. M. (2009). Recovering the scientist practitioner model: How IOs should respond to UIT. Industrial and Organizational Psychology: Perspectives on Science and Practice, 2, 58-63.
7. Beaty, J. C., Fallon, J. D., Shepherd, W. J., & Barrett, C. (2002). Proctored versus unproctored web-based administration of a cognitive ability test. Paper presented at the 17th Annual conference of the Society for Industrial and Organizational Psychology, Toronto.
8. Berman, J. A. (1993). Validation of the P.D.I. Employment Inventory in a retail chain. Journal of Business and Psychology, 7, 413-419.
9. Carless, S. A., Fewings-Hall, S., Hall, M., Hay, M., Hemsworth, P. H., & Coleman, G. J. (2007). Selecting unskilled and semi-skilled blue-collar workers: The criterion-related validity of the PDI-Employment Inventory. International Journal of Selection and Assessment, 15, 335-340.
10. Carr, J. Z., Oswald, F. L., & Schmidt, A. M. (2001). The medium and the message: Dual effects of supervision and web-based testing on construct equivalence for ability and personality measures. Paper presented at the 16th Annual Conference of the Society for Industrial and Organizational Psychology, San Diego, CA.
11. Cochran, W. G. (1977). Sampling techniques (3rd ed.). New York: Wiley.
12. Conway, J. M., & Huffcutt, A. I. (1997). Psychometric properties of multisource performance ratings: A meta-analysis of subordinate, supervisor, peer, and self-ratings. Human Performance, 10, 331-360.
13. Coyne, I., & Bartram, D. (2006). Introduction to the special issue on the ITC guidelines on computer-based and Internet-delivered testing. International Journal of Testing, 6, 115-119.
14. Do, B. -R., Shepherd, W. J., & Drasgow, F. (2005). Measurement equivalence across proctored and unproctored administration modes of web-based measures. Paper presented at the 20th annual conference of the Society for Industrial and Organizational Psychology, Los Angeles, CA.
15. Drasgow, F., Nye, C. D., Guo, J., & Tay, L. (2009). Cheating on proctored tests: The other side of the unproctored debate. Industrial and Organizational Psychology: Perspectives on Science and Practice, 2, 46-48.
16. Erwin, F. (2001). Customer Service and Clerical Potential Index (CSCPI) technical manual. Minneapolis, MN: Developed by Richardson, Bellows, Henry & Company (RBH), PreVisor Incorporated.
17. Fallaw, S. S., Solomonson, A. L., & McClelland, L. (2009). Current trends in assessment use: A multi-organizational survey. Poster presented at the 24th annual conference of the Society for Industrial and Organizational Psychology, New Orleans, LA.
18. Gupta, D. (2008). Proctored versus unproctored online testing using a personality measure: Are there any differences? Unpublished doctoral dissertation, University of North Texas, Texas.
19. Hough, L. M., Eaton, N. K., Dunnette, M. D., Kamp, J. D., & McCloy, R. A. (1990). Criterion-related validities of personality constructs and the effect of response distortion on those validities. Journal of Applied Psychology, 75, 581-595.
20. Hunter, J. E., & Schmidt, F. L. (2004). Methods of meta-analysis: Correcting error and bias in research findings (2nd ed.). Thousand Oaks, CA: Sage Publications.
21. Kaminski, K. A., & Hemingway, M. A. (2009). To proctor or not to proctor? Balancing needs with validity in online assessment. Industrial and Organizational Psychology: Perspectives on Science and Practice, 2, 24-26.
22. Komar, S., Brown, D. J., Komar, J. A., & Robie, C. (2008). Faking and the validity of conscientiousness: A Monte Carlo investigation. Journal of Applied Psychology, 93, 140-154.
23. Lahti, K., & DeKoekkoek, P. (2006). ROI for proctored versus unproctored assessment programs: Estimates from multiple utility models and identification of moderators. Paper presented at the 21st annual conference of the Society for Industrial/Organizational Psychology, Dallas, TX.
24. Mesmer-Magnus, J., & Viswesvaran, C. (2006). Assessing response distortion in personality tests. In R. L. Griffith, & M. H. Peterson (Eds.), A closer examination of applicant faking behavior (pp 85-113). Greenwich, CT: Information Age Publishing.
25. Naglieri, J. A., Drasgow, F., Schmit, M., Handler, L., Prifitera, A., Margolis, A., & Velazquez, R. (2004). Psychological testing on the Internet: New problems, old issues. American Psychologist, 59, 150-162.
26. Nye, C. D., Do, B. -R., Drasgow, F., & Fine, S. (2008). Two-step testing in employee selection: Is score inflation a problem? International Journal of Selection and Assessment, 16, 112-120.
27. Ones, D. S., Viswesvaran, C., & Reiss, A. D. (1996). Role of social desirability in personality testing for personnel selection: The red herring. Journal of Applied Psychology, 81, 660-679.
28. Paajanen, G. E., Hansen, T. L., & McLellan, R. A. (1993). PDI Employment Inventory and PDI Customer Service Inventory manual. Minneapolis, MN: Personnel Decisions International.
29. Pearlman, K. (2009). Unproctored Internet testing: Practical, legal, and ethical concerns. Industrial and Organizational Psychology: Perspectives on Science and Practice, 2, 14-19.
30. Raju, N. S., & Brand, P. A. (2003). Determining the significance of correlations corrected for unreliability and range restriction. Applied Psychological Measurement, 27, 52-71.
31. Richman, W. L., Kiesler, S., Weisband, S., & Drasgow, F. (1999). A meta-analytic study of social desirability distortion in computer-administered questionnaires, traditional questionnaires, and interviews. Journal of Applied Psychology, 84, 754-775.
32. Rosse, J. G., Stecher, M. D., Miller, J. L., & Levin, R. A. (1998). The impact of response distortion on preemployment personality testing and hiring decisions. Journal of Applied Psychology, 83, 634-644.
33. Salgado, J. F. (1997). The five factor model of personality and job performance in the European community. Journal of Applied Psychology, 82, 30-43.
34. Schmit, M. J., Kihm, J. A., & Robie, C. (2000). Development of a global measure of personality. Personnel Psychology, 53, 153-193.
35. Shah, B. V., Barnwell, B. G., & Bieler, G. S. (1995). SUDAAN user's manual: Software for analysis of correlated data. Research Triangle Park, NC: Research Triangle Institute.
36. Shen, W., Beatty, A., & Sackett, P. R. (2009). Further moderators of the interrater reliabilities of supervisory job performance ratings. Poster presented at the 24th annual conference of the Society for Industrial and Organizational Psychology, New Orleans, LA.
37. Smith, D., & Ellingson, J. E. (2002). Substance versus style: A new look at social desirability in motivating contexts. Journal of Applied Psychology, 87, 211-219.
38. Sturman, M. C., Cheramie, R. A., & Cashen, L. H. (2005). The impact of job complexity and performance measurement on the temporal consistency, stability, and test-retest reliability of employee job performance ratings. Journal of Applied Psychology, 90, 269-283.
39. Tippins, N. T. (2009). Internet alternatives to traditional proctored testing: Where are we now? Industrial and Organizational Psychology: Perspectives on Science and Practice, 2, 2-10.
40. Tippins, N. T., Beaty, J., Drasgow, F., Gibson, W. M., Pearlman, K., Segall, D. O., & Shepherd, W. J. (2006). Unproctored Internet testing in employment settings. Personnel Psychology, 59, 189-225.
41. Younger, J. (2008). Online job recruitment - Trends, benefits, outcomes and implications. Retrieved from ?Online-Job-Recruitment--Trends, -Benefits, -Outcomes-And-Implications&id=750692