A Multimedia Situational Test With a Constructed-Response Format: Its Relationship With Personality, Cognitive Ability, Job Experience, and Academic Performance

Journal of Personnel Psychology

Volumn 10, Issue 2, 2011, Pages 78-88

  Oostrom, Janneke K ^a   Born, Marise Ph ^a   Serlie, Alec W ^a   van der Molen, Henk T ^a  

^a ERASMUS UNIVERSITY ROTTERDAM   (Netherlands)

Author keywords

Academic performance; Assessment center; Construct validity; Criterion related validity; Multimedia; Situational judgment test; Webcam test

Indexed keywords

EID: 79959358270     PISSN: 18665888     EISSN: None     Source Type: Journal    
DOI: 10.1027/1866-5888/a000035     Document Type: Article

References (54)

1. Arthur W.J., Villado A. The importance of distinguishing between constructs and methods when comparing predictors in personnel selection research and practice. Journal of Applied Psychology 2008, 93:435-442.
2. Asher J.J., Sciarrino J.A. Realistic work sample tests: A review. Personnel Psychology 1974, 27:519-533.
3. Attali Y. Reliability of speeded number-right multiple-choice tests. Applied Psychological Measurement 2005, 29:357-368.
4. Bergman M.E., Drasgow F., Donovan M.A., Henning J.B., Juraska S.E. Scoring situational judgment tests: Once you get the data, your troubles begin. International Journal of Selection and Assessment 2006, 14:223-235.
5. Bernardin H.J., Buckley M.R. Strategies in rater training. Academy of Management Review 1981, 6:205-212.
6. Campbell D.T., Fiske D.W. Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin 1959, 56:81-105.
7. Chan D., Schmitt N. Situational judgment tests. Handbook of personnel selection 2005, 219-246. Blackwell, Oxford, UK. A. Evers, O. Smit-Voskuil, N. Anderson (Eds.).
8. Christian M.S., Edwards J.C., Bradley J.C. Situational judgment tests: Constructs assessed and a meta-analysis of their criterion-related validities. Personnel Psychology 2010, 63:83-117.
9. Clevenger J., Pereira G.M., Wiechmann D., Schmitt N., Harvey V.S. Incremental validity of situational judgment tests. Journal of Applied Psychology 2001, 86:410-417.
10. Costa P.T., McCrae R.R. Revised NEO Personality Inventory (NEO-PI-R) and NEO Five-Factor Inventory (NEO-FFI) 1992, Psychological Assessment Resources, Odessa, FL.
11. Cronbach L.J., Gleser G.C. Psychological tests and personnel decisions 1965, University of Illinois Press, Urbana, IL.
12. Drenth P.J.D. Test voor niet-verbale abstractie 1965, Swets & Zeitlinger, Amsterdam.
13. Funke U., Schuler H. Validity of stimulus and response components in a video test of social competence. International Journal of Selection and Assessment 1998, 6:115-123.
14. Gaugler B.B., Rosenthal D.B., Thornton G.C., Bentson C. Meta-analysis of assessment center validity. Journal of Applied Psychology 1987, 72:493-511.
15. Goldberg L.R. An alternative description of personality: The Big-Five factor structure. Journal of Personality and Social Psychology 1990, 59:1216-1229.
16. Griffin B., Hesketh B. Why openness to experience is not a good predictor of job performance. International Journal of Selection and Assessment 2004, 12:243-251.
17. Hu L., Bentler P.M. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling 1999, 6:1-55.
18. Hunter J.E., Hunter R.F. Validity and utility of alternative predictors of job performance. Psychological Bulletin 1984, 96:72-98.
19. Judge T.A., Bono J.E., Ilies R., Gerhardt M.W. Personality and leadership: A qualitative and quantitative review. Journal of Applied Psychology 2002, 87:765-778.
20. Koch, B. P. N. (1998). Handleiding G5. Unpublished manuscript.
21. Latham G.P., Saari L.M., Pursell E.D., Campion M.A. The situational interview. Journal of Applied Psychology 1980, 65:422-427.
22. Lievens F. Factors which improve the construct validity of assessment centers: A review. International Journal of Selection and Assessment 1998, 6:141-152.
23. Lievens F., Buyse T., Sackett P.R. The operational validity of a video-based situational judgment test for medical college admissions: Illustrating the importance of matching predictor and criterion construct domains. Journal of Applied Psychology 2005, 90:442-452.
24. Lievens F., Peeters H., Schollaert E. Situational judgment tests: A review of recent research. Personnel Review 2008, 37:426-441.
25. Lievens F., Sackett P.R. Video-based versus written situational judgment tests: A comparison in terms of predictive validity. Journal of Applied Psychology 2006, 91:1181-1188.
26. Lievens F., Thornton G.C. Assessment centers: Recent developments in practice and research. Handbook of selection 2005, 243-264. Blackwell, London. A. Evers, O. Smit-Voskuijl, N. Anderson (Eds.).
27. Lounsbury J.W., Sundstrom E., Loveland J.M., Gibson L.W. Intelligence, " Big Five" personality traits, and work drive as predictors of course grades. Personality and Individual Differences 2003, 35:1231-1239.
28. Loyens S.M.M., Rikers R.M.J.P., Schmidt H.G. The impact of students' conceptions of constructivist assumptions on academic achievement and drop-out. Studies in Higher Education 2007, 32:581-602.
29. McCrae R.R. Openness to experience: Expanding the boundaries of Factor V. European Journal of Personality 1994, 8:251-272.
30. McDaniel M.A., Hartman N.S., Whetzel D.L., Grubb W.L. Situational judgment tests, response instructions, and validity: A meta-analysis. Personnel Psychology 2007, 60:63-91.
31. McDaniel M.A., Morgeson F.P., Finnegan E.B., Campion M.A., Braverman E.P. Use of situational judgment tests to predict job performance: A clarification of the literature. Journal of Applied Psychology 2001, 86:730-740.
32. McDaniel M.A., Nguyen N.T. Situational judgment tests: A review of practice and constructs assessed. International Journal of Selection and Assessment 2001, 9:103-113.
33. Motowidlo S.J., Borman W.C., Schmit M.J. A theory of individual differences in task performance and contextual performance. Human Performance 1997, 10:71-83.
34. Motowidlo S.J., Brownlee A.L., Schmit M.J. Effects of personality characteristics on knowledge, skill, and performance in servicing retail customers. International Journal of Selection and Assessment 2008, 16:272-281.
35. Motowidlo S.J., Dunnette M.D., Carter G.W. An alternative selection procedure: The low-fidelity simulation. Journal of Applied Psychology 1990, 75:640-647.
36. Olson-Buchanan J.B., Drasgow F. Multimedia situational judgment tests: The medium creates the message. Situational judgment tests: Theory, measurement, and application 2006, 253-278. Erlbaum, Mahwah, NJ. J.A. Weekley, R.E. Ployhart (Eds.).
37. Oostrom J.K., Born M.Ph., Serlie A.W., Van der Molen H.T. Webcam testing: Validation of an innovative open-ended multimedia test. European Journal of Work and Organizational Psychology 2010, 19:532-550.
38. Oswald F.L., Schmitt N., Kim B.H., Ramsay L.J., Gillespie M.A. Developing a biodata measure and situational judgment inventory as predictors of college student performance. Journal of Applied Psychology 2004, 89:187-207.
39. Peterson N.G., Mumford M.D., Borman W.C., Jeanneret P.R., Fleishman E.A. An occupational information system for the 21st century: The development of O*Net 1999, American Psychological Association, Washington, DC.
40. Ployhart R.E., Holtz B.C. The diversity-validity dilemma: Strategies for reducing racioethnic and sex subgroup differences and adverse impact in selection. Personnel Psychology 2008, 61:153-172.
41. Poropat A.E. A meta-analysis of the five-factor model of personality and academic performance. Psychological Bulletin 2009, 135:322-338.
42. Schmitt N., Chan D. Situational judgment tests: Method or construct?. Situational judgment tests: Theory, measurement, and application 2006, 135-155. NJ, Erlbaum. J.A. Weekley, R.E. Ployhart (Eds.).
43. Schmitt N., Ostroff C. Operationalizing the " behavioral consistency" approach: Selection test development based on a content-oriented strategy. Personnel Psychology 1986, 39:91-108.
44. Semeijn J.H., Van der Velden R., Heijke H., Van der Vleuten C., Boshuizen H. Competence indicators in academic education and early labour market success of graduates in health sciences, RM-2005-2E 2005, Research Centre for Education and the Labour Market, Maastricht The Netherlands.
45. Sosik J.J., Kahai S.S., Avolio B.J. Transformational leadership and dimensions of creativity: Motivating idea generation in computer-mediated groups. Creativity Research Journal 1998, 11:111-121.
46. Spychalski A.C., Quinones M.A., Gaugler B.B., Pohley K. A survey of assessment center practices in organizations in the United States. Personnel Psychology 1997, 50:71-90.
47. Steiger J.H. Tests for comparing elements of a correlation matrix. Psychological Bulletin 1980, 87:245-251.
48. Stemler S.E., Sternberg R.J. Using situational judgment tests to measure practical intelligence. Situational judgment tests: Theory, measurement, and application 2006, 107-131. Erlbaum, Mahwah, NJ. J.A. Weekley, R.E. Ployhart (Eds.).
49. Thornton G.C., Byham W.C. Assessment centers and managerial performance 1982, Academic Press, New York, NY.
50. Trapmann S., Hell B., Hirn J.O.W., Schuler H. Meta-analysis of the relationship between the Big Five and academic success at university. Zeitschrift für Psychologie 2007, 215:132-151.
51. Van Leeuwen, R. G. J. (2004). Handleiding Captain. Unpublished manuscript.
52. Weekley J.A., Jones C. Video-based situational testing. Personnel Psychology 1997, 50:25-49.
53. Wernimont P.F., Campbell J.P. Signs, samples, and criteria. Journal of Applied Psychology 1968, 52:372-376.
54. Yukl G. Leadership in organizations 1998, Prentice Hall, Upper Saddle River, NJ.