Test administration and the use of test scores

Handbook of Employee Selection

Volumn , Issue , 2010, Pages 151-169

  Johnson, Jeff W ^a   Oswald, Frederick L ^b  

^a Personnel Decisions Research Institutes   (United States)

^b RICE UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84960237384     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.4324/9780203809808     Document Type: Chapter

References (79)

1. Angoff, W. H. (1971). Scales, norms, and equivalent scores. In R. L. Thorndike (Ed.), Educational measurement (2nd ed., pp. 508-600). Washington, DC: American Council on Education.
2. Arthur, W., Bell, S. T., Villado, A. J., & Doverspike, D. (2006). The use of person-organization fit in employment decision making: An assessment of its criterion-related validity. Journal of Applied Psychology, 91, 786-801.
3. Barrett, G. V., & Lueke, S B. (2004). Legal and practical implications of banding for personnel selection. In H. Aguinis (Ed.), Test-score banding in human resource selection: Technical, legal, and societal issues (pp. 71-111). Westport, CT: Praeger.
4. Bobko, P., Roth, P. L., & Buster, M. A. (2007). The usefulness of unit weights in creating composite scores: A literature review, application to content validity, and meta-analysis. Organizational Research Methods, 10, 689-709.
5. Brannick, M. T., & Darling R. W. (1991). Specifying importance weights consistent with a covariance structure. Organizational Behavior and Human Decision Processes, 50, 395-410.
6. Burke, E. F. (1997). A short note on the persistence of retest effects on aptitude scores. Journal of Occupational and Organizational Psychology, 70, 295-301.
7. Campbell, J. P. (1996). Group differences and personnel decisions: Validity, fairness, and affirmative action. Journal of Vocational Behavior, 49, 122-158.
8. Campbell, W. J., & Reilly, M. E. (2000). Accommodations for persons with disabilities. In J. F. Kehoe (Ed.), Managing selection in changing organizations (pp. 319-370). San Francisco, CA: Jossey-Bass.
9. Campion, M. A., Outtz, J. L., Zedeck, S., Schmidt, F. L., Kehoe, J. F., Murphy, K. R., & Guion, R. M. (2001). The controversy over score banding in personnel selection: Answers to 10 key questions. Personnel Psychology, 54, 149-185.
10. Carroll, J. B. (1993). Human cognitive abilities: A survey of factor-analytic studies. New York, NY: Cambridge University Press.
11. Cascio, W. F., Outtz, J., Zedeck, S., & Goldstein, I. L. (1991). Statistical implications of six methods of test score use in personnel selection. Human Performance, 4, 233-264.
12. Chan, D., & Schmitt, N. (1997). Video versus paper-and-pencil method of assessment in situational judgment tests: Subgroup differences in test performance and face validity perceptions. Journal of Applied Psychology, 82, 143-159.
13. Chang, H. H., & Ying, Z. (2008). To weight or not to weight? Balancing influence of initial items on adaptive testing. Psychometrika, 73, 441-450.
14. Cohen, J. (1990). Things I have learned (so far). American Psychologist, 45, 1304-1312.
15. Cronbach, L. J. (1990). Essentials of psychological testing (5th ed.). New York, NY: Harper Collins.
16. De Corte, W., Lievens, F., & Sackett, P. R. (2006). Predicting adverse impact and mean criterion performance in multistage selection. Journal of Applied Psychology, 91, 523-537.
17. De Corte, W., Lievens, F., & Sackett, P. R. (2007). Combining predictors to achieve optimal trade-offs between selection quality and adverse impact. Journal of Applied Psychology, 92, 1380-1393.
18. Donnellan, M. B., Oswald, F. L., Baird, B. M., & Lucas, R. E. (2006). The Mini-IPIP scales: Tiny-yet-effective measures of the Big Five factors of personality. Psychological Assessment, 18, 192-203.
19. Dorans, N. J., Pommerich, M., & Holland, P. W. (2007). Potential solutions to practical equating issues. New York, NY: Springer.
20. Edwards, J. R. (1994). The study of congruence in organizational behavior research: Critique and a proposed alternative. Organizational Behavior and Human Decision Processes, 58, 51-100.
21. Fan, X. (1998). Item response theory and classical test theory: An empirical comparison of their item/person statistics. Educational and Psychological Measurement, 58, 357-381.
22. Gibson, W. M., & Weiner, J. A. (1998). Generating random parallel test forms using CTT in a computer-based environment. Journal of Educational Measurement, 35, 297-310.
23. Goldstein, I. L., Zedeck, S., & Schneider, B. (1993). An exploration of the job analysis-content validity process. In N. Schmitt & W. C. Borman (Eds.), Personnel selection in organizations (pp. 3-34). San Francisco, CA: Jossey-Bass.
24. Green, B. F. (1977). Parameter sensitivity in multivariate methods. Multivariate Behavioral Research, 12, 264-288.
25. Guion, R. M. (1998). Assessment, measurement, andprediction for personnel decisions. Mahwah, NJ: Lawrence Erlbaum.
26. Haladyna, T., & Hess, R. (1999). An evaluation of conjunctive and compensatory standard-setting strategies for test decisions. Educational Assessment, 6, 129-153.
27. Hausknecht, J. P., Halpert, J. A., Di Paolo, N. T., & Moriarty Gerrard, M. O. (2007). Retesting in selection: A meta-analysis of coaching and practice effects for tests of cognitive ability. Journal of Applied Psychology, 92, 373-385.
28. Huselid, M. A., Jackson, S. E., & Schuler, R. S. (1997). Technical and strategic human resource management effectiveness as determinants of firm performance. Academy of Management Journal, 40, 171-188.
29. Janz, T., Hellervik, L., & Gilmore, D. (1986). Behavior description interviewing: New, accurate, cost effective. Boston, MA: Allyn and Bacon.
30. Johnson, J. W. (2007). Synthetic validity: A technique of use (finally). In S. M. Mc Phail (Ed.), Alternative validation strategies: Developing new and leveraging existing validity evidence (pp. 122-158). San Francisco, CA: Jossey-Bass.
31. Johnson, J. W., Abrahams, N., & Held, J. D. (2004, April). A procedure for selecting predictors considering validity and adverse impact. Poster presented at the 19th Annual Conference of the Society for Industrial and Organizational Psychology, Chicago, IL.
32. Kehoe, J. F., & Olson, A. (2005). Cut scores and employment discrimination litigation. In F. J. Landy (Ed.), Employment discrimination litigation: Behavioral, quantitative, and legal perspectives (pp. 410-449). San Francisco, CA: Jossey-Bass.
33. Kim, D-I., Choi, S. W., Lee, G., & Um, K. R. (2008). A comparison of the common-item and random-groups equating designs using empirical data. International Journal of Selection and Assessment, 16, 83-92.
34. Koopman, R. F. (1988). On the sensitivity of a composite to its weights. Psychometrika, 53, 547-552.
35. Kravitz, D. A. (2008). The validity-diversity dilemma: Beyond selection—The role of affirmative action. Personnel Psychology, 61, 173-193.
36. Kristof-Brown, A. L., Zimmerman, R. D., & Johnson, E. C. (2005). Consequences of individuals’ fit at work: A meta-analysis of person-job, person-organization, person-group, and person-supervisor fit. Personnel Psychology, 58, 281-342.
37. Lievens, F., Buyse, T., & Sackett, P. R. (2005). Retest effects in operational selection settings: Development and test of a framework. Personnel Psychology, 58, 981-1007.
38. Lievens, F., Reeve, C. L., & Heggestad, E. D. (2007). An examination of psychometric bias due to retesting on cognitive ability tests in selection settings. Journal of Applied Psychology, 92, 1672-1682.
39. Lievens, F., & Sackett, P. R. (2007). Situational judgment tests in high-stakes settings: Issues and strategies with generating alternate forms. Journal of Applied Psychology, 92, 1043-1055.
40. Livingston, S. A. (2004). Equating test scores (without IRT). Princeton, NJ: Educational Testing Service.
41. Mac Donald, P., & Paunonen, S. V. (2002). A Monte Carlo comparison of item and person characteristics based on item response theory versus classical test theory. Educational and Psychological Measurement, 62, 921-943.
42. Mc Culloch, M. C., & Turban, D. B. (2007). Using person-organization fit to select employees for high-turnover jobs. International Journal of Selection and Assessment, 15, 63-71.
43. Meade, A. W., & Bauer, D. J. (2007). Power and precision in confirmatory analytic tests of measurement invariance. Structural Equation Modeling: A Multidisciplinary Journal, 14, 611-635.
44. Millsap, R. E., & Kwok, O. (2004). Evaluating the impact of partial factorial invariance on selection in two populations. Psychological Methods, 9, 93-115.
45. Mueller, L., Norris, D., & Oppler, S. (2007). Implementation based on alternate validation procedures: Ranking, cut scores, banding, and compensatory models. In S. M. Mc Phail (Ed.), Alternative validation strategies: Developing new and leveraging existing validity evidence (pp. 349-405). San Francisco, CA: Jossey-Bass.
46. Murphy, K. R. (1986). When your top choice turns you down: Effect of rejected offers on the utility of selection tests. Psychological Bulletin, 99, 133-138.
47. Nye, C. D., Do, B.-R., Drasgow, F., & Fine, S. (2008). Two-step testing in employment selection: Is score inflation a problem? International Journal of Selection and Testing, 16, 112-120.
48. Oswald, F. L., Friede, A. J., Schmitt, N., Kim, B. H., & Ramsay, L. J. (2005). Extending a practical method for developing alternate test forms using independent sets of items. Organizational Research Methods, 8, 149-164.
49. Peterson, N. S., Kolen, M. J., & Hoover, H. D. (1989). Scaling, norming, and equating. In R. L. Linn (Ed.), Educational measurement (3rd ed.). New York, NY: American Council on Education and MacMillan.
50. Ployhart, R. E., & Holtz, B. C., (2008). The diversity-validity dilemma: Strategies for reducing racioethnic and sex subgroup differences and adverse impact in selection. Personnel Psychology, 61, 153-172.
51. Potosky, D., Bobko, P., & Roth, P. L. (2005). Forming composites of cognitive ability and alternative measures to predict job performance and reduce adverse impact: Corrected estimates and realistic expectations. International Journal of Selection and Assessment, 13, 304-315.
52. Pyburn, K. M., Jr., Ployhart, R. E., & Kravitz, D. (2008). The validity-diversity dilemma: Overview and legal context. Personnel Psychology, 61, 143-151.
53. Raymond, M. R., Neustel, S., & Anderson, D. (2007). Retest effects on identical and parallel forms in certification and licensure testing. Personnel Psychology, 60, 367-396.
54. Ree, M., Carretta, T., & Earles, J. (1998). In top-down decisions, weighting variables does not matter: A consequence of Wilks’ theorem. Organizational Research Methods, 1, 407-420.
55. Roth, P. L., Bevier, C. A., Bobko, P., Switzer, F. S., & Tyler, P. (2001). Ethnic group differences in cognitive ability in employment and educational settings: A meta-analysis. Personnel Psychology, 54, 297-330.
56. Sackett, P. R., & Ellingson, J. E. (1997). The effect of forming multi-predictor composites on group differences and adverse impact. Personnel Psychology, 50, 707-721.
57. Sackett, P. R., & Roth, L. (1991). A Monte Carlo examination of banding and rank order methods of test score use in personnel selection. Human Performance, 4, 279-295.
58. Sackett, P. R., & Roth, L. (1996). Multi-stage selection strategies: A Monte Carlo investigation of effects on performance and minority hiring. Personnel Psychology, 49, 549-572.
59. Sackett, P. R., Schmitt, N., Ellingson, J. E., & Kabin, M. (2001). High-stakes testing in employment, creden-tialing, and higher education: Prospects in a post-affirmative-action world. American Psychologist, 56, 302-318.
60. Schmidt, F. L. (1971). The relative efficiency of relative and simple unit predictor weights in applied differential psychology. Educational and Psychological Measurement, 31, 699-714.
61. Schmidt, F. L. (1991). Why all banding procedures in personnel selection are logically flawed. Human Performance, 4, 265-277.
62. Schmidt, F. L., Mack, M. J., & Hunter, J. E. (1984). Selection utility in the occupation of U.S. park ranger for three modes of test use. Journal of Applied Psychology, 69, 490-497.
63. Schmitt, N., & Chan, D. (2006). Situational judgment tests: Method or construct? In J. A. Weekley & R. E. Ployhart (Eds.), Situational judgment tests: Theory, measurement, and application (pp. 135-156). Mahwah, NJ: Lawrence Erlbaum.
64. Schmitt, N., & Kuljanin, G. (2008). Measurement invariance: Review of practice and implications. Human Resource Management Review, 18, 210-222.
65. Schmitt, N., & Oswald, F. L. (2004). Statistical weights of ability and diversity in selection decisions based on various methods of test-score use. In H. Aguinis (Ed.), Test-score banding in human resource selection: Technical, legal, and societal issues (pp. 113-131). Westport, CT: Praeger.
66. Schmitt, N., Rogers, W., Chan, D., Sheppard, L., & Jennings, D. (1997). Adverse impact and efficiency among various predictor combinations. Journal of Applied Psychology, 82, 719-730.
67. Segall, D. O. (2001, April). Detecting test compromise in high-stakes computerized adaptive testing: A verification testing approach. Paper presented at the annual meeting of the National Council on Measurement in Education, Seattle, WA.
68. Society for Industrial and Organizational Psychology. (2003). Principles for the validation and use of personnel selection procedures (4th ed.). Bowling Green, OH: Author.
69. Stanton, J. M., Sinar, E. F., Balzer, W. K., & Smith, P. C. (2002). Issues and strategies for reducing the length of self-report scales. Personnel Psychology, 55, 167-194.
70. Stark, S., Chernyshenko, O. S., & Drasgow, F. (2004). Examining the effects of differential item functioning and differential test functioning on selection decisions: When are statistically significant effects practically important? Journal of Applied Psychology, 89, 497-508.
71. Tett, R. P., Jackson, D. N., Rothstein, M., & Reddon, J. R. (1999). Meta-analysis of bi-directional relations in personality-job performance research. Human Performance, 12, 1-29.
72. Tippins, N. T., Beaty, J., Drasgow, F., Gibson, W. M., Pearlman, K., Segall, D. O., & Shepherd, W. (2006). Unproctored Internet testing in employment settings. Personnel Psychology, 59, 189-225.
73. Truxillo, D. M., & Bauer, T. N. (1999). Applicant reactions to test score banding in entry-level and promotional contexts. Journal of Applied Psychology, 84, 322-339.
74. Truxillo, D. M., & Bauer, T. N. (2000). The roles of gender and affirmative action in reactions to test score use methods. Journal of Applied Social Psychology, 30, 1812-1828.
75. U.S. Department of Labor. (1999). Testing and assessment: An employer’s guide to good practices. Washington, DC: U.S. Department of Labor, Employment and Training Administration.
76. Vandenberg, R. J., & Lance, C. E. (2000). A review and synthesis of the measurement invariance literature: Suggestions, practices and recommendations for organizational research. Organizational Research Methods, 3, 4-70.
77. van der Linden, W. J. (2005). Linear models for optimal test design. New York, NY: Springer.
78. Viswesvaran, C., & Ones, D. S., (1999). Meta-analyses of fakability estimates: Implications for personality measurement. Educational and Psychological Measurement, 59, 197-210.
79. Weekley, J. A., & Jones, C. (1997). Video-based situational testing. Personnel Psychology, 50, 25-49.