Employment interview reliability: New meta-analytic estimates by structure and format

International Journal of Selection and Assessment

Volumn 21, Issue 3, 2013, Pages 264-276

  Huffcutt, Allen I ^a   Culbertson, Satoris S ^b   Weyhrauch, William S ^b  

^a BRADLEY UNIVERSITY   (United States)

^b KANSAS STATE UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84881585521     PISSN: 0965075X     EISSN: 14682389     Source Type: Journal    
DOI: 10.1111/ijsa.12036     Document Type: Article

References (106)

1. Anderson, R. C. (1954). The guided interview as an evaluative instrument. Journal of Educational Research, 48, 203-209.
2. Asher, J. J. (1970). Reliability of a novel format for the selection interview. Psychological Reports, 26, 451-456.
3. Barrick, M. R., Swider, B. W., & Stewart, G. L. (2010). Initial evaluations in the interview: Relationships with subsequent interviewer evaluations and employment offers. Journal of Applied Psychology, 95, 1037-1046.
4. Bass, B. M. (1951). Situational tests: I. Individual interviews compared with leaderless group discussions. Educational and Psychological Measurement, 11, 67-75.
5. Berry, C. M., Sackett, P. R., & Landers, R. N. (2007). Revisiting interview-cognitive ability relationships: Attending to specific range restriction mechanisms in meta-analysis. Personnel Psychology, 60, 837-874.
6. Bosshardt, M. J. (1993). Situational interviews versus behavior description interviews: A comparative validity study. Unpublished doctoral dissertation, University of Minnesota.
7. Broadhurst, A. (1974). Reliability of interview in selecting students for postgraduate study in clinical psychology. Psychological Reports, 34, 819-825.
8. Buchner, L. M. (1990). Increases in interrater reliability of situational interviews as a function of the number of benchmark answers. Applied H.R.M. Research, 1, 27-31.
9. Burgess, M. M., Calkins, V., & Richards, J. M. (1972). The structured interview: A selection device. Psychological Reports, 31, 867-877.
10. Burnett, J. R., & Motowidlo, S. J. (1998). Relations between different sources of information in the structured selection interview. Personnel Psychology, 51, 963-983.
11. Campion, M. A., Campion, J. E., & Hudson, J. P., Jr. (1994). Structured interviewing: A note on incremental validity and alternate question types. Journal of Applied Psychology, 79, 998-1002.
12. Castle, P., & Garforth, F. (1951). Selection, training and status of supervisors: I. Selection. Occupational Psychology, 25, 109-123.
13. Chung-Yan, G. A., Cronshaw, S. F., & Hausdorf, P. A. (2005). Information exchange article: A criterion-related validation study of transit operators. International Journal of Selection and Assessment, 13, 172-177.
14. Conway, J. M., Jako, R. A., & Goodman, D. F. (1995). A meta-analysis of interrater and internal consistency reliability of selection interviews. Journal of Applied Psychology, 80, 565-579.
15. Cortina, J. M. (1993). What is coefficient alpha? An examination of theory and applications. Journal of Applied Psychology, 78, 98-104. Doi: 10.1037/0021.9010.78.1.98.
16. Davey, B. (1984). Are all oral panels created equal? A study of differential validity across oral panels. Presentation to the IPMA Assessment Council Annual Conference.
17. Fearing, F., & Fearing, F. M. (1942). Factors in the appraisal interview considered with particular reference to the selection of public personnel. The Journal of Psychology: Interdisciplinary and Applied, 14, 131-153.
18. Finesinger, J. E., Cobb, S., Chapple, E. D., & Brazier, M. A. B. (1948). Investigation of prediction of success in naval flight training. Part I: The squantum study (Report No. 81). Washington, DC: Civil Aeronautics Administration.
19. Fisher, J., Epstein, L. J., & Harris, M. R. (1967). Validity of the psychiatric interview: Predicting the effectiveness of the first Peace Corps volunteers in Ghana. Archives of General Psychiatry, 17, 744-750.
20. Gaines, L. K., & Lewis, B. R. (1982). Reliability and validity of the oral interview board in police promotions: A research note. Journal of Criminal Justice, 10, 403-419.
21. Gerstein, L. H., Barke, C. R., & Johnson, S. D. (1989). Internal validity studies of a telephone preemployment measure. Journal of Employment Counseling, 26, 77-83.
22. Ghiselli, E. E., Campbell, J. P., & Zedeck, S. (1981). Measurement theory for the behavioral sciences. New York: W. H. Freeman.
23. Gordon, M. J., & Lincoln, J. A. (1976). Family practice resident selection: Value of the interview. The Journal of Family Practice, 3, 175-177.
24. Green, P. C., Alter, P., & Carr, A. (1993, April). Reliability and validity of a behavior-based interview. In D. L. Denning (Chair), Psychometric analysis of the structured interview. Symposium conducted at the Eighth Annual Meeting of the Society for Industrial and Organizational Psychology, San Francisco.
25. Grove, D. A. (1981). A behavioral consistency approach to decision making in employment selection. Personnel Psychology, 34, 55-64.
26. Hakel, M. D. (1971). Similarity of post-interview trait rating intercorrelations as a contributor to interrater agreement in a structured employment interview. Journal of Applied Psychology, 55, 443-448.
27. Hargrave, G. E., & Hiatt, D. (1987). Law enforcement selection with the interview, MMPI, and CPI: A study of reliability and validity. Journal of Police Science and Administration, 15, 110-117.
28. Harris, R. M., Fisher, J., & Epstein, L. J. (1963). The reliability of the interview in psychiatric assessment for job placement. Comprehensive Psychiatry, 4, 19-28.
29. Hoffman, C. C., & Holden, L. M. (1993, April). Dissecting the interview: An application of generalizability theory. In D. L. Denning (Chair), Psychometric analysis of the structured interview. Symposium conducted at the Eighth Annual Meeting of the Society for Industrial and Organizational Psychology, San Francisco.
30. Hollingworth, H. L. (1923). Judging human character. New York: Appleton.
31. Hovland, C. I., & Wonderlic, E. F. (1939). Prediction of industrial success from a standardized interview. Journal of Applied Psychology, 23, 537-546.
32. Huffcutt, A. I., & Arthur, W. Jr. (1994). Hunter & Hunter (1984) revisited: Interview validity for entry-level jobs. Journal of Applied Psychology, 79, 184-190.
33. Huffcutt, A. I., Conway, J. M., Roth, P. L., & Klehe, U.-C. (2004). The impact of job complexity and study design on situational and behavior description interview validity. International Journal of Selection and Assessment, 12, 262-273.
34. Huffcutt, A. I., Roth, P. L., & McDaniel, M. A. (1996). A meta-analytic investigation of cognitive ability in employment interview evaluations: Moderating characteristics and implications for incremental validity. Journal of Applied Psychology, 81, 459-473.
35. Huffcutt, A. I., Van Iddekinge, C. H., & Roth, P. L. (2011). Understanding applicant behavior in employment interviews: A theoretical model of interviewee performance. Human Resource Management Review, 21, 353-367.
36. Huffcutt, A. I., Weekley, J. A., Wiesner, W. H., Degroot, T. G., & Jones, C. (2001). Comparison of situational and behavior description interview questions for higher-level positions. Personnel Psychology, 54, 619-644.
37. Hunter, J. E., & Schmidt, F. L. (2004). Methods of meta-analysis: Correcting error and bias in research findings. Thousand Oaks, CA: Sage.
38. Hunter, J. E., Schmidt, F. L., & Le, H. (2006). Implications of direct and indirect range restriction for meta-analysis methods and findings. Journal of Applied Psychology, 91, 594-612.
39. Janz, T. (1982). Initial comparison of patterned behavior description interviews versus unstructured interviews. Journal of Applied Psychology, 67, 577-580.
40. Johnson, E. K. (1991). The interview in a medical school setting: Manipulating structuring criteria. Proceedings from the Sixth Annual Conference of the Society for Industrial and Organizational Psychology.
41. Kennedy, R. (1986). An investigation of criterion-related validity for the structured interview. Unpublished master's thesis, East Carolina University.
42. Klehe, U., König, C. J., Richter, G. M., Kleinmann, M., & Melchers, K. G. (2008). Transparency in structured interviews: Consequences for construct and criterion-related validity. Human Performance, 21, 107-137.
43. Klehe, U., & Latham, G. (2006). What would you do - really or ideally? Constructs underlying the behavior description interview and the situational interview in predicting typical versus maximum performance. Human Performance, 19, 357-382.
44. Landy, F. J. (1976). The validity of the interview in police officer selection. Journal of Applied Psychology, 61, 193-198.
45. Latham, G. P., & Saari, L. M. (1984). Do people do what they say? Further studies on the situational interview. Journal of Applied Psychology, 69, 569-573.
46. Latham, G. P., Saari, L. M., Pursell, E. D., & Campion, M. A. (1980). The situational interview. Journal of Applied Psychology, 65, 422-427.
47. Latham, G. P., & Skarlicki, D. P. (1995). Criterion-related validity of the situational and patterned behavior description interviews with organizational citizenship behavior. Human Performance, 8, 67-80.
48. Latham, G. P., & Sue-Chan, C. (1998). Selecting employees in the 21st century: Predicting the contribution of I-O psychology to Canada. Canadian Psychology, 39, 14-22.
49. LeBreton, J. M., & Senter, J. L. (2008). Answers to 20 questions about interrater reliability and interrater agreement. Organizational Research Methods, 11, 815-852.
50. Lin, T.-R., Dobbins, G. H., & Farh, J.-L. (1992). A field study of race and age similarity effects on interview ratings in conventional and situational interviews. Journal of Applied Psychology, 77, 363-371.
51. Little, J. P., Schoenfelt, E. L., & Brown, R. D. (2000, April). The situational versus patterned behavior description interview for predicting customer service performance. Paper presented at the annual conference of the Society for Industrial and Organizational Psychology, New Orleans.
52. Maas, J. B. (1965). Patterned scaled expectation interview: Reliability studies on a new technique. Journal of Applied Psychology, 49, 431-433.
53. Martin, J. E. (1993). The effect of providing choices on the validity of a situational interview for resident advisors. Applies H.R.M. Research, 4, 69-78.
54. Maurer, T. J., Solamon, J. M., Andrews, K. D., & Troxtel, D. D. (2001). Interviewee coaching, preparation strategies, and response strategies in relation to performance in situational employment interviews: An extension of Maurer, Solamon, & Troxtel (1998). Journal of Applied Psychology, 86, 709-717.
55. Maurer, T. J., Solamon, J. M., & Lippstreu, M. (2008). How does coaching interviewees affect the validity of a structured interview? Journal of Organizational Behavior, 29, 355-371.
56. McCarthy, J. M., Van Iddekinge, C. H., & Campion, M. A. (2010). Are highly structured job interviews resistant to demographic similarity effects? Personnel Psychology, 63, 325-359.
57. McDaniel, M. A., Whetzel, D. L., Schmidt, F. L., & Maurer, S. (1994). The validity of employment interviews: A comprehensive review and meta-analysis. Journal of Applied Psychology, 79, 599-617.
58. McFarland, L. A., Ryan, A. M., & Kriska, S. D. (2002). Field study investigation of applicant use of influence tactics in a selection interview. The Journal of Psychology: Interdisciplinary and Applied, 136, 383-398.
59. McGraw, K. O., & Wong, S. P. (1996). Forming inferences about some intraclass correlation coefficients. Psychological Methods, 1, 30-46.
60. Melchers, K. G., König, C. J., Richter, G. M., Harstein, T., Kleinmann, M., & Klehe, U. (2008, April). Effects of assessor rotation on interview and AC construct validity. Paper presented at the 23rd annual conference of the Society for Industrial and Organizational Psychology, San Francisco.
61. Morgeson, F. P., Reider, M. H., & Campion, M. A. (2005). Selecting individuals in team settings: The importance of social skills, personality characteristics, and teamwork knowledge. Personnel Psychology, 58, 583-611.
62. Morse, M., & Hawthorne, J. W. (1946). Some notes on oral examinations. Public Personnel Review, 7, 15-18.
63. Moscoso, S., & Salgado, J. F. (2001). Psychometric properties of a structured behavioral interview to hire private security personnel. Journal of Business and Psychology, 16, 51-59.
64. Mosher, M. R. (1991, June). Development of a behaviorally consistent structured interview. Paper presented at the 27th International Applied Military Psychology Symposium, Stockholm, Sweden.
65. Motowidlo, S. J., Carter, G. W., Dunnette, M. D., Tippins, N., Werner, S., Burnett, J. R., & Vaughan, M. J. (1992). Studies of the structured behavioral interview. Journal of Applied Psychology, 77, 571-587.
66. Mussel, P., Behrmann, M., & Schuler, H. (2008, April). Explaining the psychometric properties of structured and unstructured interviews. Poster presented at the 23rd annual conference of the Society for Industrial and Organizational Psychology, San Francisco.
67. Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory (3rd ed.) New York: McGraw-Hill.
68. Orpen, C. (1985). Patterned behavior description interviews versus unstructured interviews: A comparative validity study. Journal of Applied Psychology, 70, 774-776.
69. Palacios, M. H., Newberry, L. A., & Bootzin, R. R. (1966). Predictive validity of the interview. Journal of Applied Psychology, 50, 67-72.
70. Poole, A., Catano, V. M., & Cunningham, D. P. (2007). Predicting performance in Canadian dental schools: The new CDA structured interview, a new personality assessment, and the DAT. Journal of Dental Education, 71, 664-676.
71. Posthuma, R. A., Morgeson, F. P., & Campion, M. A. (2002). Beyond employment interview validity: A comprehensive narrative of review of recent research and trends over time. Personnel Psychology, 55, 1-81.
72. Prien, E. P. (1962). Assessments of higher-level personnel: V. An analysis of interviewers' predictions of job performance. Personnel Psychology, 15, 319-334.
73. Raines, G. N., & Rohrer, J. H. (1955). The operational matrix of psychiatric practice. The American Journal of Psychiatry, 5, 721-733.
74. Roth, P. L., Van Iddekinge, C. H., Huffcutt, A. I., Eidson, C. E., Jr, & Bobko, P. (2002). Corrections for range restriction in structured interview ethnic group differences: The values may be larger than researchers thought. Journal of Applied Psychology, 87, 369-376.
75. Roth, P. L., Van Iddekinge, C. H., Huffcutt, A. I., Eidson, C. E., Jr, & Schmitt, M. J. (2005). Personality saturation in structured interviews. International Journal of Selection and Assessment, 13, 261-273.
76. Rynes, S., & Gerhart, B. (1990). Interviewer assessments of applicant 'fit': An exploratory investigation. Personnel Psychology, 43, 13-35.
77. Schleicher, D. J., Van Iddekinge, C. H., Morgeson, F. P., & Campion, M. A. (2010). If at first you don't succeed, try, try again: Understanding race, age, and gender differences in retesting score improvement. Journal of Applied Psychology, 95, 603-617.
78. Schmidt, F. L., Hunter, J. E., & Raju, N. S. (1988). Validity generalization and situational specificity: A second look at the 75% rule and Fisher's z transformation. Journal of Applied Psychology, 73, 665-672.
79. Schmidt, F. L., Le, H., & Ilies, R. (2003). Beyond alpha: An empirical examination of the effects of different sources of measurement error on reliability estimates for measures of individual differences constructs. Psychological Methods, 8, 206-224.
80. Schmidt, F. L., & Zimmerman, R. D. (2004). A counterintuitive hypothesis about employment interview validity and some supporting evidence. Journal of Applied Psychology, 89, 553-561.
81. Schmitt, N. (1996). Uses and abuses of coefficient alpha. Psychological Assessment, 8, 350-353. Doi: 10.1037/1040.3590.84.350.
82. Schmitt, N., & Ostroff, C. (1986). Operationalizing the 'behavioral consistency' approach: Selection test development based on a content-oriented strategy. Personnel Psychology, 59, 91-108.
83. Schneidman, E. S. (1943). A note on the experimental study of the appraisal interview. Journal of Applied Psychology, 27, 196-205.
84. Schuler, H., & Funke, U. (1989). The interview as a multimodal procedure. In R. W. Eder & G. R. Ferris (Eds.), The employment interview: Theory, research, and practice (pp. 183-192). Newbury Park, CA: Sage.
85. Schuler, H., Funke, U., Moser, K., & Donat, M. (1995). Personalauswahl in forschung und entwicklung. Eignung und leistung von wissenschaftlern und ingenieuren. [Personnel selection in research and development. Aptitudes and performance of scientists and engineers.] Goettingen: Hogrefe.
86. Scott, W. D. (1915). The scientific selection of salesmen. Advertising & Selling, 25, 5-6. 94-96.
87. Shaw, J. (1952). The function of the interview in determining fitness for teacher-training. Journal of Educational Research, 45, 667-681.
88. Shrout, P. E., & Fleiss, J. L. (1979). Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin, 86, 420-428.
89. Snow, A. J. (1924). An experiment in the validity of judging human ability. Journal of Applied Psychology, 8, 339-346.
90. Sparks, C. P. (1973). Validity of a selection program for process trainees (Personnel Research No. 73-1). Unpublished manuscript.
91. Stohr-Gilmore, M. K., Stohr-Gilmore, M. W., & Kistler, N. (1990). Improving selection outcomes with the use of situational interviews: Empirical evidence from a study of correctional officers for new generation jails. Review of Public Personnel Administration, 10, 1-18.
92. Tarico, V. S., Altmaier, E. M., Smith, W. L., Franken, E. A., Jr, & Berbaum, K. S. (1986). Development and validation of an accomplishment interview for radiology residents. Journal of Medical Education, 61, 845-847.
93. Taylor, P. J., & Small, B. (2002). Asking applicants what they would do versus what they did do: A meta-analytic comparison of situational and past behaviour employment interview questions. Journal of Occupational and Organizational Psychology, 75, 277-294.
94. Tross, S. A., & Maurer, T. J. (2008). The effect of coaching interviewees on subsequent interview performance in structured experience-based interviews. Journal of Occupational and Organizational Psychology, 81, 589-605.
95. U.S. Office of Personnel Management, Office of Examination Development, Alternative Examining Procedures Division. (1987). The structured interview. Washington, DC: Author.
96. Uhrbrock, R. S. (1948). The personnel interview. Personnel Psychology, 1, 273-302.
97. Van Iddekinge, C. H., Raymark, P. H., & Roth, P. L. (2005). Assessing personality with a structured employment interview: Construct-related validity and susceptibility to response inflation. Journal of Applied Psychology, 90, 536-552.
98. Van Iddekinge, C. H., Raymark, P. H., Roth, P. L., & Payne, H. S. (2006). Comparing the psychometric characteristics of ratings of face-to-face and videotaped structured interviews. International Journal of Selection and Assessment, 14, 347-359.
99. Viswesvaran, C., Ones, D. S., & Schmidt, F. L. (1996). Comparative analysis of the reliability of job performance ratings. Journal of Applied Psychology, 81, 557-574.
100. Walsh, U. R. (1975). A test of the construct and predictive validity of a structured interview. Doctoral Dissertation, Retrieved from ProQuest Dissertations and Theses. (288010492).
101. Wanous, J. P., Sullivan, S. E., & Malinak, J. (1989). The role of judgment calls in meta-analysis. Journal of Applied Psychology, 74, 259-264.
102. Weekley, J. A., & Gier, J. A. (1987). Reliability and validity of the situational interview for a sales position. Journal of Applied Psychology, 72, 484-487.
103. Whetzel, D. L., Baranowski, L. E., Petro, J. M., Curtin, P. J., & Fisher, J. L. (2003). A written structured interview by any other name is still a selection instrument. Applied H.R.M. Research, 8, 1-16.
104. Wiesner, W. H., & Cronshaw, S. F. (1988). A meta-analytic investigation of the impact of interview format and degree of structure on the validity of the employment interview. Journal of Occupational Psychology, 61, 275-290.
105. Wright, O. R., Carter, J. L., & Fowler, E. P. (1967). A differential analysis of an oral interview program. Public Personnel Review, 28, 242-246.
106. Zaccaria, M. A., Dailey, J. T., Tupes, E. C., Stafford, A. R., Lawrence, H. G., & Ailsworth, K. A. (1956). Development of an interview procedure for USAF officer applicants (AFPTRC-TN- 56-43, Project No. 7701). Lackland Air Force Base, TX: Personnel Research Laboratory, Air Force Personnel and Training Research Center, Air Research and Development Command.