Test development and use: New twists on old questions

Human Resource Management

Volumn 44, Issue 3, 2005, Pages 219-235

  Cascio, Wayne F ^a,b,c,d,e   Aguinis, Herman ^b  

^a UNIVERSITY OF COLORADO   (United States)

^b UNIVERSITY OF COLORADO   (United States)

^c CPP Inc   (United States)

^d Society for Human Resource Management Foundation   (United States)

^e Academy of Management   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 25444526653     PISSN: 00904848     EISSN: None     Source Type: Journal    
DOI: 10.1002/hrm.20068     Document Type: Review

References (73)

1. Aguilera v. Cook County Police and Corrections Merit Board. 760 F.2d 844 (7th Circuit, 1985).
2. Aguinis, H. (2001). Estimation of sampling variance of correlations in meta-analysis. Personnel Psychology, 54, 569-590.
3. Aguinis, H. (2004). Regression analysis for categorical moderators. New York: Guilford.
4. Aguinis, H., & Whitehead, R. (1997). Sampling variance in the correlation coefficient under indirect range restriction: Implications for validity generalization. Journal of Applied Psychology, 82, 528-538.
5. American Educational Research Association (AERA), American Psychological Association (APA), & National Council on Measurement in Education (NCME). (1999). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.
6. Angoff, W. H. (1971). Scales, norms, and equivalent scores. In R. L. Thorndike (Ed.), Educational measurement (pp. 508-600). Washington, DC: American Council on Education.
7. Bernard v. Gulf Oil Corp. 890 F.2d 735 (5th Circuit, 1989).
8. Borman, W. C., & Motowidlo, S. J. (1993). Expanding the criterion domain to include elements of contextual performance. In N. Schmitt & W. C. Borman (Eds.), Personnel selection in organizations (pp. 71-98). San Francisco: Jossey-Bass.
9. Borman, W. C., & Motowidlo, S. J. (1997). Task performance and contextual performance: The meaning for personnel selection research. Human Performance, 10, 99-109.
10. Browne, M. W. (1975). Predictive validity of a linear regression equation. British Journal of Mathematical and Statistical Psychology, 28, 79-87.
11. Browne, M. W. (2000). Cross-validation methods. Journal of Mathematical Psychology, 44, 108-132.
12. Campbell, J. P., McCloy, R. A., Oppler, S. H., & Sager, C. E. (1993). A theory of performance. In N. Schmitt & W. C. Borman (Eds.), Personnel psychology in organizations (pp. 35-70). San Francisco: Jossey-Bass.
13. Cascio, W. F. (2000). Costing human resources: The financial impact of behavior in organizations (4th ed.). Cincinnati, OH: South-Western College Publishing.
14. Cascio, W. F., & Aguinis, H. (2001). The federal Uniform Guidelines on Employee Selection Procedures (1978): An update on selected issues. Review of Public Personnel Administration, 21, 200-218.
15. Cascio, W. F., & Aguinis, H. (2005). Applied psychology in human resource management (6th ed.). Upper Saddle River, NJ: Prentice Hall.
16. Cascio, W. F., Alexander, R. A., & Barrett, G. V. (1988). Setting cutoff scores: Legal, psychometric, and professional issues and guidelines. Personnel Psychology, 41, 1-24.
17. Claudy, J. G. (1978). Multiple regression and validity estimation in one sample. Applied Psychological Measurement, 2, 595-607.
18. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum.
19. Coleman, V. I., & Borman, W. C. (2000). Investigating the underlying structure of the citizenship performance domain. Human Resource Management Review, 10, 25-44.
20. Cotter, K. L., & Raju, N. S. (1982). An evaluation of formula-based population squared cross-validity estimates in prediction. Educational and Psychological Measurement, 40, 101-112.
21. Deadrick, D. L. & Madigan, R. M. (1990). Dynamic criteria revisited: A longitudinal study of performance stability and predictive validity. Personnel Psychology, 43, 717-744.
22. Drasgow, F., Dorans, N. J., & Tucker, L. R. (1979). Estimators of the squared cross-validity coefficient: A Monte Carlo investigation. Applied Psychological Measurement, 3, 171-192.
23. DuBois, C. L., Sackett, P. R., Zedeck, S., & Fogli, L. (1993). Further exploration of typical and maximum performance criteria: Definitional issues, prediction, and white-black differences. Journal of Applied Psychology, 78, 205-211.
24. EEOC v. Atlas Paper Box Co. 868 F.2d at 1487 (6th Circuit, 1989).
25. Efron, B., & Gong, G. (1983). A leisurely look at the bootstrap, the jackknife, and cross-validation. The American Statistician, 37, 36-48.
26. Ezekiel, M. (1930). Methods of correlational analysis. New York: Wiley.
27. Hartigan, J. A., & Wigdor, A. K. (Eds.). (1989). Fairness in employment testing: Validity generalization, minority issues, and the General Aptitude Test Battery. Washington, DC: National Academy Press.
28. Hofmann, D. A., Jacobs, R., & Baratta, J. E. (1993). Dynamic criteria and the measurement of change. Journal of Applied Psychology, 78, 194-204.
29. Hulin, C. L. (2001). Applied psychology and science: Differences between research and practice. Applied Psychology: An International Review, 50, 225-234.
30. Hulin, C. L., Henry, R. A., & Noon, S. L. (1990). Adding a dimension: Time as a factor in the generalizability of predictive relationships. Psychological Bulletin, 107, 328-340.
31. Hunter, J. E., & Schmidt, F. L. (2004). Methods of meta-analysis: Correcting error and bias in research findings (2nd ed.). Thousand Oaks, CA: Sage.
32. Johns, G. (1993). Constraints on the adoption of psychology-based personnel practices: Lessons from organizational innovation. Personnel Psychology, 46, 569-592.
33. Johnson, J. W. (2001). The relative importance of task and contextual performance dimensions to supervisor judgments of overall performance. Journal of Applied Psychology, 86, 984-996.
34. Keil, C. T., & Cortina, J. M. (2001). Degradation of validity over time: A test and extension of Ackerman's model. Psychological Bulletin, 127, 673-697.
35. Krantz, D. H. (1999). The null hypothesis testing controversy in psychology. Journal of the American Statistical Association, 44, 1372-1381.
36. Landy, F. J. (2003). Validity generalization: Then and now. In K. R. Murphy (Ed.), Validity generalization: A critical review (pp. 155-195). Mahwah, NJ: Lawrence Erlbaum Associates.
37. Landy, F. J., & Vasey, J. (1991). Job analysis: The composition of SME samples. Personnel Psychology, 44, 27-50.
38. Larson, S. C. (1931). The shrinkage of the coefficient of multiple correlation. Journal of Educational Psychology, 22, 45-55.
39. Latham, G. P. (2001). The reciprocal transfer of learning from journals to practice. Applied Psychology: An International Review, 50, 201-211.
40. Lautenschlager, G. L. (1990). Sources of imprecision in formula cross-validated multiple correlations. Journal of Applied Psychology, 75, 460-462.
41. Mael, F. A., & Hirsch, A. C. (1993). Rainforest empiricism and quasi-rationality: Two approaches to objective biodata. Personnel Psychology, 46, 719-738.
42. Markus, K. A. (2001). The converse inequality argument against tests of statistical significance. Psychological Methods, 6, 147-160.
43. Mitchell, T. W. (1985). An evaluation of the validity of correlational research conducted in organiza-tions. Academy of Management Review, 10, 192-205.
44. Mitchell, T. W., & Klimoski, R. J. (1986). Estimating the validity of cross-validity estimation. Journal of Applied Psychology, 71, 311-317.
45. Mosier, C. I. (1951). Symposium: The need and means of cross-validation. Educational and Psychological Measurement, 11, 5-11.
46. Muchinsky, P. M. (2004). When the psychometrics of test development meets organizational realities: A conceptual framework for organizational change, examples, and recommendations. Personnel Psychology, 57, 175-209.
47. Murphy, K. R. (1983). Fooling yourself with cross-validation: Single sample designs. Personnel Psychology, 36, 111-118.
48. Murphy, K. R. (1984). Cost-benefit considerations in choosing among cross-validation methods. Personnel Psychology, 37, 15-22.
49. Nickerson, R. S. (2000). Null hypothesis significance testing: A review of an old and continuing controversy. Psychological Methods, 5, 241-301.
50. Organ, D. W. (1997). Organizational citizenship behavior: It's construct clean-up time. Human Performance, 10, 85-97.
51. Plake, B. S., & Hambleton, R. K. (2001). The analytic judgment method for setting standards on complex performance assessments. In G. J. Cizek (Ed.), Setting performance standards: Concepts methods, and perspectives (pp. 283-312). Mahwah, NJ: Lawrence Erlbaum.
52. Raju, N. S., Anselmi, T. V., Goodman, J. S., & Thomas, A. (1998). The effect of correlated artifacts and true validity on the accuracy of parameter estimation in validity generalization. Personnel Psychology, 51, 453-465.
53. Raju, N. S., Bilgic, R., Edwards, J. E., & Fleer, P. F. (1997). Methodology review: Estimation of population validity and cross-validity, and the use of equal weights in prediction. Applied Psychological Measurement, 21, 291-305.
54. Raju, N. S., Bilgic, R., Edwards, J. E., & Fleer, P. F. (1999). Accuracy of population validity and cross-validity estimation: An empirical comparison of formula-based, traditional empirical, and equal weight procedures. Applied Psychological Measurement, 23, 99-115.
55. Rynes, S. L., Colbert, A. E., & Brown, K. G. (2002). HR professionals' beliefs about effective human resource practices: Correspondence between research and practice. Human Resource Management, 41, 149-174.
56. Sackett, P. R., Zedeck, S., & Fogli, L. (1988). Relations between measures of typical and maximum job performance. Journal of Applied Psychology, 73, 482-486.
57. Schmidt, F. L. (1996). Statistical significance testing and cumulative knowledge in psychology: Implications for training of researchers. Psychological Methods, 1, 115-129.
58. Schmidt, F. L., & Hunter, J. E. (1977). Development of a general solution to the problem of validity generalization. Journal of Applied Psychology, 62, 529-540.
59. Schmidt, F. L., & Hunter, J. E. (1998). The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings. Psychological Bulletin, 124, 262-274.
60. Schmidt, F. L., & Kaplan, L. B. (1971). Composite vs. multiple criteria: A review and resolution of the controversy. Personnel Psychology, 24, 419-434.
61. Schmitt, N., Coyle, B. W., & Rauschenberger, J. (1977). A Monte Carlo evaluation of three formula estimates of cross-validated multiple correlation. Psychological Bulletin, 84, 751-758.
62. Schmitt, N., & Ployhart, R. E. (1999). Estimates of cross-validity for stepwise regression and with predictor selection. Journal of Applied Psychology, 84, 50-57.
63. Society for Industrial and Organizational Psychology. (SIOP) (2003). Principles for the validation and use of personnel selection procedures (4th ed.). Bowling Green, OH: Author.
64. Steele-Johnson, D., Beauregard, R. S., Hoover, P. B., & Schmidt, A. M. (2000). Goal orientation and task demand effects on motivation, affect, and performance. Journal of Applied Psychology, 85, 724-738.
65. Steele-Johnson, D., Osburn, H. G., & Pieper, K. F. (2000). A review and extension of current models of dynamic criteria. International Journal of Selection and Assessment, 8, 110-136.
66. St. John, C. H., & Roth, P. L. (1999). The impact of cross-validation adjustments on estimates of effect size in business policy and strategy research. Organizational Research Methods, 2, 157-174.
67. Task Force on Statistical Inference. (2000). Narrow and shallow. American Psychologist, 55, 965.
68. Truxillo, D. M., Donahue, L. M., & Sulzer, J. L. (1996). Setting cutoff scores for personnel selection tests: Issues, illustrations, and recommendations. Human Performance, 9, 275-295.
69. Tryon, W. W. (2001). Evaluating statistical difference, equivalence, and indeterminacy using inferential confidence intervals: An integrated alternative method of conducting null hypothesis statistical tests. Psychological Methods, 6, 371-386.
70. Tukey, J. W. (1958). Bias and confidence in not quite large samples. Annals of Mathematical Statistics, 29, 614.
71. Uniform Guidelines on Employee Selection Procedures. (1978). Federal Register, 43, 38290-38315.
72. Wainer, H. (1999). One cheer for null hypothesis significance testing. Psychological Methods, 4, 212-213.
73. Wherry, R. J. (1931). A new formula for predicting the shrinkage of multiple correlation. Annals of Mathematics and Statistics, 2, 440-457.