Current concepts of validity, validation, and generalizability

Handbook of Employee Selection

Volumn , Issue , 2010, Pages 99-123

  Kehoe, Jerard F ^a   Murphy, Kevin R ^b  

^a Selection and Assessment Consulting   (United States)

^b PENNSYLVANIA STATE UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84909279917     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.4324/9780203809808     Document Type: Chapter

References (80)

1. American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (1985). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.
2. American Psychological Association. (1954). Technical Recommendations for Psychological Tests and Diagnostic Techniques. Washington, DC: Author.
3. Angoff, W. H. (1988). Validity: An evolving concept. In H. Wainer & H. Braun (Eds.), Test validity (pp. 19-32). Hillsdale, NJ: Lawrence Erlbaum.
4. Austin, J. T., & Villanova, P. (1992). The criterion problem 1917-1992. Journal of Applied Psychology, 77, 836-874.
5. Barrett, G. V., Phillips, J. S., & Alexander, R. A. (1981). Concurrent and predictive validity designs: A critical reanalysis. Journal of Applied Psychology, 66, 1-6.
6. Barrick, M. R., & Mount, M. K. (1991). The Big Five personality dimensions and job performance: A metaanalysis. Personnel Psychology, 44, 1-26.
7. Binning, J. F. (2007). Validation strategies. In S. G. Rogelberg (Ed.), Encyclopedia of industrial and organizational psychology (Vol. 2, pp. 859-864). Thousand Oaks, CA: Sage.
8. Binning, J. F., & Barrett, G. V. (1989). Validity of personnel decisions: A conceptual analysis of the inferential and evidential bases. Journal of Applied Psychology, 74, 478-494.
9. Brannick, M. T. (2001) Implications of empirical Bayes meta-analysis for test validation. Journal of Applied Psychology, 86, 468-480.
10. Brennan, R. L. (1983). Elements of generalizability theory. Iowa City, IA: American College Test Program.
11. Campbell, D. T., & Fiske, D. W. (1959). Convergent and discriminant validity in the multi-trait multi-method matrix. Psychological Bulletin, 56, 81-105.
12. Cook, T. D., & Campbell, D. T. (1979). Quasi-experimentation: Design and analysis issues for field settings. Chicago, IL: Rand McNally.
13. Cronbach, L. J. (1988). Five perspectives on the validity argument. In H. Wainer & H. Brown (Eds.), Test validity. Hillsdale, NJ: Lawrence Erlbaum.
14. Cronbach, L. J. (1989). Construct validation after thirty years. In R. Linn (Ed.), Intelligence: Measurement, theory, and public policy. Urbana: University of Illinois Press.
15. Cronbach, L. J., Gleser, G. C., Nanda, H., & Rajaratnam, N. (1972). The dependability of behavioral measurements: Theory of generalizability for scores and profiles. New York, NY: Wiley.
16. Cronbach, L. J., & Meehl, P. E. (1955). Construct validity in psychological tests. Psychological Bulletin, 52, 281-302.
17. Dimitrov, D. M. (2002). Reliability: Arguments for multiple perspectives and potential problems with generalization across studies. Educational and Psychological Measurement, 62, 783-801.
18. Goldstein, I. L., & Zedeck, S. (1996). Content validation. In R. Barrett (Ed.), Fair employment strategies in human resource management (pp. 27-37). Westport, CT: Quorum Books.
19. Goldstein, I. L., Zedeck, S., & Schneider, B. (1993). An exploration of the job analysis-content validity process. In N. Schmitt & W. C. Borman (Eds.), Personnel selection in organizations. San Francisco, CA: Jossey-Bass.
20. Gottfredson, L. S. (1986). Societal consequences of the g factor in employment. Journal ofVocationalBehavior, 29, 379-410.
21. Graham, J. M. (2006). Congeneric and (essentially) tau-equivalent estimates of score reliability: What they are and how to use them. Educational and Psychological Measurement, 66, 930-944.
22. Guion, R. M. (1977). Content validity, the source of my discontent. Applied Psychological Measurement, 1, 1-10.
23. Guion, R. M. (1978a). “Content validity” in moderation. Personal Psychology, 31, 205-213.
24. Guion, R. M. (1978b). Scoring content domain samples: The problem of fairness. Journal of Applied Psychology, 63, 499-506.
25. Guion, R. M. (1980). On trinitarian doctrines of validity. Professional Psychology, 11, 385-398.
26. Guion, R. M. (1998) Assessment, measurement and prediction for personnel decisions. Mahwah, NJ: Lawrence Erlbaum.
27. Guion, R. M., & Cranny, C. J. (1982). A note on concurrent and predictive validity designs. Journal of Applied Psychology, 67, 239-244.
28. Hausknecht, J. P., Halpert, J. A., Di Paolo, N. T., & Moriarty Gerrard, M. O. (2007). Retesting in selection: A meta-analysis of coaching and practice effects for tests of cognitive ability. Journal of Applied Psychology, 92, 373-385.
29. Herrnstein, R. J., & Murray, C. (1994). The bell curve: Intelligence and class structure in American life. New York, NY: Free Press.
30. Hoffman, C. C., Rashovsky, B., & D’Egidio, E. (2007). Job component validity: Background, current research, and applications. In S. M. McPhail (Ed.), Alternative validation strategies: Developing new and leveraging existing validity evidence. San Francisco, CA: Jossey-Bass.
31. Hogan, J., Davies, S., & Hogan, R. (2007). Generalizing personality-based validity evidence. In S. M. McPhail (Ed.). Alternative validation strategies: Developing new and leveraging existing validity evidence. San Francisco, CA: Jossey-Bass.
32. Hunter, J. E. (1986). Cognitive ability, cognitive aptitudes, job knowledge, and job performance. Journal of Vocational Behavior, 29, 340-362.
33. Jensen, A. R. (1980). Bias in mental testing. New York, NY: Free Press.
34. Kleiman, L. S., & Faley, R. H. (1985). The implications of professional and legal guidelines for court decisions involving criterion-related validity: A review and analysis. Personnel Psychology, 38, 803-833
35. Kraiger, K., & Teachout, M. S. (1990). Generalizability theory as construct-related evidence of the validity of job performance ratings. Human Performance, 3, 19-36.
36. Landy, F. J. (1986). Stamp collecting versus science: Validation as hypothesis testing. American Psychologist, 41, 1183-1192.
37. Langenfeld, T. E., & Crocker, L. M. (1994). The evolution of validity theory: Public school testing, the courts, and incompatible interpretations. Educational Assessment, 2, 149-165.
38. Lawshe, C. H. (1952). Employee selection. Personnel Psychology, 6, 31-34.
39. Lawshe, C. H. (1985). Inferences from personnel tests and their validities. Journal of Applied Psychology, 70, 237-238.
40. Lievens, F., Chasteen, C. S., Day, E. A., & Christiansen, N. D. (2006). Large-scale investigation of the role of trait activation theory for understanding assessment center convergent and discriminant validity. Journal of Applied Psychology, 91, 247-258.
41. Lievens, F., Reeve, C. L., & Heggestad E. D. (2007). An examination of psychometric bias due to retesting on cognitive ability tests in selection settings. Journal of Applied Psychology, 92, 1672-1682.
42. Mc Cormick, E. J. (1959). The development of processes for indirect or synthetic validity: III. Application of job analysis to indirect validity. A symposium. Personnel Psychology, 12, 402-413.
43. Mc Daniel, M. A., Whetzel, D. L., Schmidt, F. L., & Maurer, S. D. (1994). The validity of employment interviews: A comprehensive review and meta-analysis. Journal of Applied Psychology, 79, 599-616.
44. Mc Donald, R. P. (1999). Test theory: A unified treatment. Mahwah, NJ: Lawrence Erlbaum Associates.
45. Mc Henry, J. J, Hough, L. M., Toquam, J. L., Hanson, M. A., & Ashworth, S. (1990). Project A validity results: The relationship between predictor and criterion domains. Personnel Psychology, 43, 335-355.
46. Mc Kay, P. F., & Mc Daniel, M. A. (2006). A Re-examination of Black-White mean differences in work performance: More data, more moderators. Journal of Applied Psychology, 91, 538-554.
47. Mc Kenzie, S. B., Podsakoff, P. M., & Jarvis, C. B. (2005). The problem of measurement model misspecifica-tion in behavioral and organizational research and some recommended solutions. Journal of Applied Psychology, 90, 710-730.
48. Messick, S. (1988). The once and future issues of validity: Assessing the meaning and consequences of measurement. In H. Wainer & H. Brown (Eds.), Test validity. Hillsdale, NJ: Lawrence Erlbaum.
49. Messick, S. (1989). Meaning and values in test validation: The science and ethics of assessment. Educational Researcher, 18, 5-11.
50. Murphy, K. R. (2000). Impact of assessment of validity generalization and situational specificity on the science and practice of personnel selection. International Journal of Selection and Assessment, 8, 194-206.
51. Murphy, K. R. (2003). Validity generalization: A critical review. Mahwah, NJ: Lawrence Erlbaum.
52. Murphy, K. R. (2010). How a broader definition of the criterion domain changes our thinking about adverse impact. In J. Outtz (Ed.), Adverse impact (pp. 137-160). San Francisco, CA: Jossey-Bass.
53. Murphy, K. R., & Cleveland, J. (1995). Understanding performance appraisal: Social, organizational and goal-oriented perspectives. Newbury Park, CA: Sage.
54. Murphy, K. R., & Davidshofer, C.O. (2005). Psychological testing: Principles and applications (6th ed). Upper Saddle River, NJ: Prentice Hall.
55. Murphy, K. R., & DeShon, R. (2000a). Inter-rater correlations do not estimate the reliability of job performance ratings. Personnel Psychology, 53, 873-900.
56. Murphy, K. R., & DeShon, R. (2000b). Progress in psychometrics: Can industrial and organizational psychology catch up? Personnel Psychology, 53, 913-924.
57. Murphy, K. R., & Shiarella, A. (1997). Implications of the multidimensional nature of job performance for the validity of selection tests: Multivariate frameworks for studying test validity. Personnel Psychology, 50, 823-854.
58. Nathan, B. R., & Alexander, R. A. (1988). A comparison of criteria for test validation: A meta-analytic investigation. Personnel Psychology, 41, 517-535.
59. Newman, D. A., Jacobs, R. R., & Bartram, D. (2007). Choosing the best method for local validity estimation: Relative accuracy of meta-analysis versus a local study versus Bayes-analysis. Journal of Applied Psychology, 92, 1394-1413.
60. Nunnally, J. (1975). Psychometric theory 25 years ago and now. Educational Researcher, 4, 7-14, 19-21.
61. Peterson, N. G., Wise, L. L., Arabian, J., & Hoffman, R. G. (2001). Synthetic validation and validity generalization when empirical validation is not possible. In J. Campbell & D. Knapp (Eds.), Exploring the upper limits of personnel selection and classification (pp. 411-451). Mahwah, NJ: Lawrence Erlbaum.
62. Ree, M. J., & Earles, J. A. (1992). Intelligence is the best predictor of job performance. Current Directions in Psychological Science, 1, 86-89.
63. Roth, P. L., BeVire, C. A., Bobko, P., Switzer, F. S. III, & Tyler, P. (2001). Ethnic group differences in cognitive ability in employment and education settings: A meta-analysis. Personnel Psychology, 54, 297-330.
64. Sackett, P. R. (1987). Assessment centers and content validity: Some neglected issues. Personnel Psychology, 40, 13-25.
65. Salgado, J. F., Anderson, N., Moscoso, S., Bertua, C., & de Fruyt, F. (2003). International validity generalization of GMA and cognitive abilities: A European Community meta-analysis. Personnel Psychology, 56, 573-605.
66. Schmidt, F. L. (1992). What do data really mean? Research findings, meta-analysis, and cumulative knowledge in psychology. American Psychologist, 47, 1173-1181.
67. Schmidt, F. L., & Hunter, J. E. (1977). Development of a general solution to the problem of validity generalization. Journal of Applied Psychology, 62, 643-661.
68. Schmidt, F. L., & Hunter, J. E. (1980). The future of criterion-related validity. Personnel Psychology, 33, 41-60.
69. Schmidt, F. L., & Hunter, J. E. (1998). The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings. Psychological Bulletin, 124, 262-274.
70. Schmidt, F. L., Ones, D. S., & Hunter, J. E. (1992). Personnel selection. Annual Review of Psychology, 43, 627-670.
71. Schmidt, F. L., & Raju, N. S. (2007). Updating meta-analytic research findings: Bayesian approaches versus the medical model. Journal of Applied Psychology, 92, 297-308.
72. Schmidt, F. L., Viswesvaran, C., & Ones, D. S. (2000). Reliability is not validity and validity is not reliability. Personnel Psychology, 53, 901-912.
73. Schmitt, N., Gooding, R. Z., Noe, R. A., & Kirsch, M. (1984). Meta-analysis of validity studies published between 1964 and 1982 and the investigation of study characteristics. Personnel Psychology, 37, 407-422.
74. Society for Industrial and Organizational Psychology (1987). Principles for the validation and use of personnel selection procedures (3rd ed.). Bowling Green, OH: Author.
75. Society for Industrial and Organizational Psychology (2003). Principles for the validation and use of personnel selection procedures (4th ed.). Bowling Green, OH: Author.
76. Steel, P. G., Huffcutt, A. I., & Kammeyer-Mueller, J. (2006). From the work one knows the worker: A systematic review of challenges, solutions and steps to creating synthetic validity. International Journal of Selection and Assessment, 14,16-36.
77. Stelly, D. J., & Goldstein, H. W. (2007). Application of content validation methods to broader constructs. In S. M. Mc Phail (Ed.), Alternative validation strategies: Developing new and leveraging existing validity evidence. San Francisco, CA: Jossey-Bass.
78. Thorndike, R. M., & Lohman, D. F. (1990). A century of ability testing. Chicago, IL: Riverside.
79. Uniform guidelines on employee selection procedures. (1978). Federal Register, 43(166), 38290-39309.
80. Viswesvaran, C., Schmidt, F. L., & Ones, D. S. (2005). Is there a general factor in ratings of performance? A meta-analytic framework for disentangling substantive and error influences. Journal of Applied Psychology, 90, 108-131.