Unlocking the key to biodata scoring: A comparison of empirical, rational, and hybrid approaches at different sample sizes

Personnel Psychology

Volumn 65, Issue 2, 2012, Pages 385-428

  Cucina, Jeffrey M ^a   Caputo, Pat M ^a   Thibodeaux, Henry F ^b   Maclane, Charles N ^c  

^a US Customs and Border Protection ^*  (United States)

^b US Office of Personnel Management   (United States)

^c Independent Consulting   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84860653690     PISSN: 00315826     EISSN: 17446570     Source Type: Journal    
DOI: 10.1111/j.1744-6570.2012.01244.x     Document Type: Article

References (67)

1. Aamodt MG, Pierce WL. (1987). Comparison of the rare response and vertical percent methods for scoring the biographical information blank. Educational and Psychological Measurement, 47, 505-511.
2. Bergman ME, Drasgow F, Donovan, MA, Juraska SE. (2003). Scoring of Situational Judgment Tests. Paper presented at the 18th Annual Conference of the Society for Industrial and Organizational Psychology, Orlando, FL.
3. Bergman ME, Drasgow F, Donovan, MA, Henning JB, Juraska SE. (2006). Scoring situational judgment tests: Once you get the data, your troubles begin. International Journal of Selection and Assessment, 14(3), 223-235.
4. Brown SH. (1994). Validating biodata. In Stokes GS, Mumford MD, Owens WA (Eds.), Biodata handbook. Palo Alto, CA : CPP.
5. Carlson KD, Scullen SE, Schmidt FL, Rothstein H, Erwin F. (1999). Generalizable biographical data validity can be achieved without multi-organizational development and keying. Personnel Psychology, 52(3), 731-755.
6. Cascio WF. (1982). Applied psychology in personnel management (2nd ed.). Reston, VA : Reston.
7. Cohen J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ : Erlbaum.
8. Cohen J. (1992). A power primer. Psychological Bulletin, 112(1), 155-159.
9. Cucina JM. (2007, June). A comparison of alternative methods of scoring a broad-bandwidth personality inventory to predict freshman GPA. International Public Management Association for Human Resources Assessment Council (IPMAAC) Student Paper Competition Winner, 2007. Paper presented at the 31st Meeting of IPMAAC, St. Louis, MO
10. Cucina JM, Bayless JM, Thibodeaux HF, Busciglio HH, MacLane CN. (2009, April). How to score biodata measures: A master tutorial. Master tutorial presented at the 24th Meeting of the Society for Industrial and Organizational Psychology, New Orleans, LA
11. Cucina JM, Vasilopoulos NL, McElreath JM. (2008, April). Using empirical keying to score personality measures. Paper presented at the 23rd Meeting of the Society for Industrial and Organizational Psychology, San Francisco, CA
12. Dalessio AT. (1994). Predicting insurance agent turnover using a video-based situational judgment test. Journal of Business and Psychology, 9, 23-32.
13. Davis B. (2001, June). Empirical keying of personality measures to maximize validity. Paper presented at the 25th Annual IPMAAC Conference on Professional Personnel Assessment, Newport Beach, CA
14. Davis BW. (1997). An integration of biographical data and personality research through Sherwood Forest empiricism: Robbing from personality to give to biodata. Unpublished doctoral dissertation, Louisiana State University.
15. Davis BW. (2002, April). Empirical keying an existing personality measure to improve predictive validity. Paper presented at the 17th Annual Conference of the Society for Industrial and Organizational Psychology, Toronto, Canada
16. Dean MA, Russell CJ. (2001, August). Bootstrap cross-validation efficiencies in personnel selection. Paper presented at the Annual Academy of Management Meeting, Washington, DC
17. Dean MA, Russell CJ, Farmer W. (2002). Noncognitive predictors of air traffic controller performance. In Eibfeldt H, Heil M, Broach D, (Eds.), Staffing the air traffic management (ATM) system: The selection of air traffic controllers (pp. 59-72). Brookfield, VT: Ashgate
18. Devlin SE, Abrahams NM, Edwards JE. (1992). Empirical keying of biographical data: Cross-validity as a function of scaling procedure and sample size. Military Psychology, 4(3), 119-136.
19. Dunnette MD. (1962). Personnel management. Annual Review of Psychology, 13, 285-314.
20. England GW. (1961). Development and use of weighted application blanks. Minneapolis, MN : Industrial Relations Center, University of Minnesota.
21. England GW. (1971). Development and use of weighted application blanks. Minneapolis, MN : Industrial Relations Center, University of Minnesota.
22. Gandy JA, Dye DA, MacLane CN. (1994). Federal government selection: The Individual Achievement Record. In Stokes GS, Mumford MD, Owens WA (Eds.), Biodata handbook: Theory, research, and use of biographical information in selection and performance prediction. Palo Alto, CA : CPP.
23. Gandy JA, Outerbridge AN, Sharf JC, Dye DA. (1989). Development and initial validation of the Individual Achievement Record (IAR). (PRD-90-01). Washington, DC : U.S. Office of Personnel Management.
24. Goldberg LR. (1972). Parameters of personality inventory construction and utilization: A comparison of prediction strategies and tactics. Multivariate Behavioral Research Monographs, 72(2), 1-59.
25. Guion RM. (1965). Personnel testing. New York, NY : McGraw-Hill.
26. Guttman L. (1941). An outline of the statistical theory of prediction. In Horst P (Ed.), The prediction of personal adjustment. New York, NY : Social Science Research Council.
27. Hogan JB. (1994). Empirical keying of background data measures. In Stokes GS, Mumford MD, Owens WA (Eds.), Biodata handbook. Palo Alto, CA : CPP.
28. Hough L, Paullin C. (1994). Construct-oriented scale construction: The rational approach. In Stokes GS, Mumford MD, Owens WA, (Eds.), Biodata handbook: Theory, research, and use of biographical information in selection and performance prediction. Palo Alto, CA : CPP.
29. Hunter JE, Hunter RF. (1984). Validity and utility of alternative predictors of job performance. Psychological Bulletin, 96(1), 72-98.
30. Karas M, West J. (1999) Construct-oriented biodata development for selection to a differentiated performance domain. International Journal of Selection and Assessment, 7, 86-96.
31. Kelley TL. (1939). The selection of upper and lower groups for the validation of test items. Journal of Educational Psychology, 30(1), 17-24.
32. Kluger AN, Reilly RR, Russell CJ. (1991). Faking biodata tests: Are option-keyed instruments more resistant? Journal of Applied Psychology, 76(6), 889-896.
33. Lecznar WB, Dailey JT. (1950). Keying biographical inventories in classification test batteries. American Psychologist, 5, 279.
34. Lefkowitz J, Gebbia MI, Balsam T, Dunn L. (1999). Dimensions of biodata and their relationships to item validity. Journal of Occupational and Organizational Psychology, 72, 331-350.
35. Lubin A, Osburn HG. (1957). A theory of pattern analysis for the prediction of a quantitative criterion. Psychometrika, 22(1), 63-73.
36. Lubin A, Osburn HG. (1960). The use of configural analysis for the prediction of a qualitative criterion. Educational & Psychological Measurement, 20, 275-282.
37. Mael FA, Hirsch AC. (1993). Rainforest empiricism and quasi-rationality: Two approaches to objective biodata. Personnel Psychology, 46(4), 719-738.
38. Malone MP. (1978). Predictive efficiency and discriminatory impact of verifiable biographical data as a function of data analysis procedure. Unpublished doctoral dissertation, University of Minnesota.
39. May K, Hittner JB. (1997). Tests for comparing dependent correlations revisited: A Monte Carlo study. Journal of Experimental Education, 65(3), 257-270.
40. Mead AD. (2000). Properties of a resampling validation technique for empirically scoring psychological assessments. Unpublished doctoral dissertation, University of Illinois at Urbana-Champaign.
41. Meehl PE. (1950). Configural scoring. Journal of Consulting Psychology, 14, 165-171.
42. Meng XL, Rosenthal R, Rubin DB. (1992). Comparing correlated correlation coefficients. Psychological Bulletin, 111(1), 172-175.
43. Miner JB. (1965). Studies in management education. New York, NY : Springer.
44. Mitchell TW, Klimoski RJ. (1982). Is it rational to be empirical? A test of methods for scoring biographical data. Journal of Applied Psychology, 67(4), 411-418.
45. Mumford MD, Owens WA. (1987). Methodology review: Principles, procedures, and findings in the application of background data measures. Applied Psychological Measurement, 11(1), 1-31.
46. Mumford MD, Stokes GS. (1992). Developmental determinants of individual action: Theory and practice in applying background measures. In Dunnette MD, Hough LM (Eds.), Handbook of industrial and organizational psychology. Palo Alto, CA : Consulting Psychologists Press.
47. O'Connor BP. (2000). SPSS, SAS, and MATLAB programs for determining the number of components using parallel analysis and Velicer's MAP test. Behavior Research Methods, Instruments, & Computers, 32, 396-402.
48. Olson-Buchanan JB, Drasgow F, Moberg PJ, Mead AD, Keenan PA, Donovan MA. (1998). Interactive video assessment of conflict resolution skills. Personnel Psychology, 51, 1-24.
49. Osburn HG, Lubin A. (1957). The use of configural analysis for the evaluation of test scoring methods. Psychometrika, 22(4), 359-371.
50. Reiter-Palmon R, Connelly MS. (2000). Item selection counts: A comparison of empirical key and rational scale validities in theory-based and non-theory-based item pools. Journal of Applied Psychology, 85(1), 143-151.
51. Rothstein HR, Schmidt FL, Erwin FW, Owens WA, Sparks CP. (1990). Biographical data in employment selection: Can validities be made generalizable? Journal of Applied Psychology, 75(2), 175-184.
52. Schmidt FL, Hunter JE. (1998). The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings. Psychological Bulletin, 124(2), 262-274.
53. Schmidt FL, Hunter JE, Ury VW. (1976). Statistical power in criterion-related validation studies. Journal of Applied Psychology, 61(4), 473-485.
54. Schoenfeldt LF. (1999). From dust bowl empiricism to rational constructs in biographical data. Human Resource Management Review, 9, 147-167.
55. Sharf JC. (1994). The impact of legal and equal employment opportunity issues on personal history inquiries. In Stokes GS, Mumford MD, Owens WA (Eds.), Biodata handbook: Theory, research, and use of biographical information in selection and performance prediction. Palo Alto, CA : CPP.
56. Stead WH, Shartle CL. (1940). Occupational counseling techniques: Their development and application. New York, NY : American.
57. Stokes GS, Hogan JB, Snell AF. (1993). Comparability of incumbent and applicant samples for the validation of biodata keys: The influence of social desirability. Personnel Psychology, 46, 739-762.
58. Stokes GS, Searcy CA. (1999). Specification of scales in biodata form development: Rational vs. empirical and global vs. specific. International Journal of Selection and Assessment, 7(2), 72-85.
59. Strong EK. (1926). An interest test for personnel managers. Journal of Personnel Research, 5, 194-203.
60. Taylor CW, Ellison RL. (1967). Biographical predictors of scientific performance. Science, 155(3766), 1075-1080.
61. Telenson PA, Alexander RA, Barrett GV. (1983). Scoring the biographical information blank: A comparison of three weighting techniques. Applied Psychological Measurement, 7(1), 73-80.
62. Thayer PW. (1977). Somethings old, somethings new. Personnel Psychology, 30, 513-524.
63. Tompkins SS, Miner JB. (1957). The Tompkins-Horn picture arrangement test. New York, NY : Springer.
64. U.S. Department of Labor. (1970). Manual for the USES general aptitude test battery. Section III: Development. Washington, DC : Author.
65. U.S. Department of Labor. (1974). Descriptive rating scale (Form MA-7-66, Rev. 3-74). Washington, DC : U.S. Department of Labor, Manpower Administration.
66. Wallace SR, Clarke WV, Dry RJ. (1956). The activity vector analysis as a selector of life insurance salesmen. Personnel Psychology, 9, 337-344.
67. Weekley JA, Jones C. (1997). Video-based situational testing. Personnel Psychology, 50, 25-49.