Subject Matter Expert Judgments Regarding the Relative Importance of Competencies are not Useful for Choosing the Test Batteries that Best Predict Performance

International Journal of Selection and Assessment

Volumn 21, Issue 4, 2013, Pages 419-429

  Murphy, Kevin R ^a   Deckert, Paige J ^b   Kinney, Theodore B ^c   Kung, Mei Chuan ^c  

^a Department of Environmental and Radiological Health Sciences   (United States)

^b PENNSYLVANIA STATE UNIVERSITY   (United States)

^c Select International Inc   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84887935768     PISSN: 0965075X     EISSN: 14682389     Source Type: Journal    
DOI: 10.1111/ijsa.12051     Document Type: Article

References (43)

1. Brown, K. G., Le, H., & Schmidt, F. L. (2006). Specific aptitude theory revisited: Is there incremental validity for training performance? International Journal of Selection and Assessment, 14, 87-100.
2. Carrier, M. R., Dalessio, A. T., & Brown, S. H. (1990). Correspondence between estimates of content and criterion-related validity values. Personnel Psychology, 43, 85-100.
3. Cascio, W. F. (1998). Applied psychology in human resource management (5th ed.) Upper Saddle River, NJ: Prentice-Hall.
4. Dierdorff, E. C., & Wilson, M. A. (2003). A meta-analysis of job analysis reliability. Journal of Applied Psychology, 88, 635-646.
5. Earles, J. A., & Ree, M. J. (1992). The predictive validity of the ASVAB for training grades. Educational and Psychological Measurement, 52, 721-725.
6. Fleishman, E. A., & Mumford, M. D. (1991). Evaluating classifications of job behavior: A construct validation of ability requirement scales. Personnel Psychology, 44, 523-575.
7. Goldstein, I. L., & Zedeck, S. (1996). Content validation. In R. Barrett (Ed.), Fair employment strategies in human resource management (pp. 27-37). Westport, CT: Quorum Books.
8. Goldstein, I. L., Zedeck, S., & Schneider, B. (1993). An exploration of the job analysis-content validity process. In N. Schmitt & W. C. Borman (Eds.), Personnel selection in organizations (pp. 3-34). San Francisco, CA: Jossey-Bass.
9. Guardians of NY v. Civil Service Commission (CA2 1980) 630 F.2d 79.
10. Guion, R. M. (1998). Assessment, measurement, and prediction for personnel decisions. Mahwah, NJ: Lawrence Erlbaum.
11. Hartigan, J. A., & Wigdor, A. K. (1989). Fairness in employment testing: Validity generalization, minority issues, and the General Aptitude Test Battery. Washington, DC: National Academy Press.
12. Harvey, R. J. (1991). Job analysis. In M. D. Dunnette & L. M. Hough (Eds.), Handbook of industrial and organizational psychology (Vol. 2, pp. 71-163). Palo Alto, CA: Consulting Psychologists.
13. Hunter, J. E. (1986). Cognitive ability, cognitive aptitudes, job knowledge and job performance. Journal of Vocational Behavior, 29, 340-362.
14. Jeanneret, P. R., & Strong, M. H. (2003). Linking O*NET job information to job requirement predictors: An O*NET application. Personnel Psychology, 56, 465-492.
15. Jones, G. E., & Ree, M. J. (1998). Aptitude test score validity: No moderating effect due to job ability requirement differences. Educational and Psychological Measurement, 58, 284-294.
16. Kehoe, J. (2012). What to make of content validity evidence for cognitive tests? Comments on Schmidt (2012). International Journal of Selection and Assessment, 20, 14-18.
17. Kehoe, J., & Murphy, K. (2010). Modern applications of the concept of validity and the validation process. In J. Farr & N. Tippins (Eds.), Handbook of employee selection (pp. 99-123). Mahwah, NJ: Erlbaum.
18. Lawshe, C. H. (1975). A quantitative approach to content validity. Personnel Psychology, 28, 563-575.
19. Morgeson, F. P., Delaney-Klinger, K., Mayfield, M. S., Ferrara, P., & Campion, M. A. (2004). Self-presentation processes in job analysis: A field experiment investigating inflation in abilities, tasks and competencies. Journal of Applied Psychology, 89, 674-686.
20. Murphy, K. R. (2009a). Content validation is useful for many things, but validity isn't one of them. Industrial and Organizational Psychology: Perspectives on Science and Practice, 2, 453-464.
21. Murphy, K. R. (2009b). Is content-related evidence useful in validating selection tests? Industrial and Organizational Psychology: Perspectives on Science and Practice, 2, 517-526.
22. Murphy, K. R. (2009c). Validity, validation and values. The Academy of Management Annals, 3, 421-461.
23. Murphy, K. R., Dzieweczynski, J. L., & Zhang, Y. (2009). Positive manifold limits the relevance of content-matching strategies for validating selection test batteries. Journal of Applied Psychology, 94, 1018-1031.
24. Nagoshi, C. T., & Johnson, R. C. (1986). The ubiquity of g. Personality and Individual Differences, 7, 201-207.
25. Peterson, N. G., Mumford, M. D., Borman, W. C., Jeanneret, P. R., & Fleishman, E. A. (1999). An occupational information system for the 21st century: The development of O*NET. Washington, DC: American Psychological Association.
26. Ployhart, R. E. (2012). The content validity of cognitively oriented tests: Commentary on Schmidt (2012). International Journal of Selection and Assessment, 20, 19-23.
27. Prediger, D. (1987). Validity of the new armed services vocational aptitude battery job cluster scores in career planning. Career Development Quarterly, 36, 113-125.
28. Ree, M. J., Carretta, T. R., & Earles, J. A. (1998). In top-down decisions, weighting variables does not matter. A consequence of Wilks' Theorem. Organizational Research Methods, 1, 407-420.
29. Ree, M. J., & Earles, J. A. (1991). Predicting training success: Not much more than g. Personnel Psychology, 44, 321-332.
30. Ree, M. J., & Earles, J. A. (1992). Intelligence is the best predictor of job performance. Current Directions in Psychological Science, 1, 86-89.
31. Ree, M. J., & Earles, J. A. (1994). The ubiquitous predictiveness of g. In M. G. Rumsey, C. B. Walker, & J. H. Harris (Eds.), Personnel selection and classification (pp. 127-136). Hillsdale, NJ: Erlbaum.
32. Ree, M. J., Earles, J. A., & Teachout, M. S. (1994). Predicting job performance: Not much more than g. Journal of Applied Psychology, 79, 518-524.
33. Rice, W. R., & Gaines, S. D. (1994). 'Heads I win, tails you lose': Testing directional alternative hypotheses in ecological and evolutionary research. Tree, 9, 235-237.
34. Salgado, J. F., Anderson, N., Moscoso, S., Bertua, C., & de Fruyt, F. (2003). International validity generalization of GMA and cognitive abilities: A European Community meta-analysis. Personnel Psychology, 56, 573-605.
35. Schmidt, F. L. (2012a). Cognitive tests used in selection can have content validity as well as criterion validity: A broader research review and implications for practice. International Journal of Selection and Assessment, 20, 1-13.
36. Schmidt, F. L. (2012b). Content validity and cognitive tests: Response to Kehoe (2012), Ployhart (2012), and Sackett (2012). International Journal of Selection and Assessment, 20, 30-35.
37. Schmidt, F. L., & Hunter, J. E. (1998). The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings. Psychological Bulletin, 124, 262-274.
38. Society for Industrial and Organizational Psychology, Inc. (2003). Principles for the validation and use of personnel selection procedures (4th ed.) Bowling Green, OH: Author.
39. Tannenbaum, R. J. (1992). Job analysis of the knowledge and abilities important for newly licensed (certified) Spanish teachers (Report ETS-RR-92-48). Princeton, NJ: Educational Testing Service.
40. Tett, R. P., Jackson, D. N., & Rothstein, M. (1991). Personality measures as predictors of job performance: A meta-analytic review. Personnel Psychology, 44, 703-742.
41. Thurstone, L. L. (1947). Multiple factor analysis: The development and expansion of vectors of the mind. Chicago, IL: University of Chicago Press.
42. Tross, S. A., & Maurer, T. J. (2000). The relationship between SME job experience and job analysis ratings: Findings with and without statistical control. Journal of Business and Psychology, 15, 97-110.
43. Uniform Guidelines on Employee Selection Procedures. 29 C.F.R. 1607 (1978).