Multimedia situational judgment tests: The medium creates the message

Situational Judgment Tests: Theory, Measurement, and Application

Volumn , Issue , 2013, Pages 253-278

  Olson Buchanan, Julie B ^a   Drasgow, Fritz ^b  

^a CALIFORNIA STATE UNIVERSITY   (United States)

^b UNIVERSITY OF ILLINOIS AT URBANA CHAMPAIGN   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84920765319     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.4324/9780203774878     Document Type: Chapter

References (67)

1. Ashworth, S. D., & Joyce, T. M. (1994, April). Developing scoring protocols for a computerized multimedia in-basket exercise. Paper presented at the annual conference of the Society for Industrial and Organizational Psychology, Nashville, TN.
2. Ashworth, S. D., & McHenry, J. J. (1992, April). Developing a multimedia in-basket: Lessons learn Paper presented at the annual conference of the Society for Industrial and Organizational Psychology, San Francisco, CA.
3. Barbera, K. M. (1996). Multimedia employment tests: The influence of attitudes and experiences on validity. Unpublished doctoral dissertation, Department of Psychology, Bowling Green State University, Bowling Green, Ohio.
4. Bergman, M. E., Donovan, M. A., & Drasgow, F. (2001, April). Situational judgment, personality, and cognitive ability: Are we really assessing different constructs? Paper presented at the annual conference of the Society for Industrial and Organizational Psychology, San Diego, CA.
5. Bergman, M. E., Drasgow, F., Donovan, M. A., Juraska, S. E., & Nejdlik, J. B. (2004). Scoring situational judgment tests. Manuscript under review.
6. Borman, W. C., & Motowidlo, S. J. (1993). Expanding the criterion domain to include elements of contextual performance. In N. Schmitt & W. C. Borman, Personnel selection (pp. 71–98). San Francisco, CA: Jossey-Bass.
7. Borman, W. C., & Motowidlo, S. J. (1997). Task performance and contextual performance: The meaning for personnel selection research. Human Performance, 10, 99109.
8. Bom, M. Ph., Van der Maesen de Sombreff, P. E. A. M., & Van der Zee, K. I. (2001, April). A multimedia situational judgment test for social intelligence. Paper presented at the annual conference of the Society for Industrial and Organizational Psychology, San Diego, CA.
9. Breiman, L. Friedman, J. H., Olshen, R. A., & Stone, C. J. (1984). Classification and regression trees. Belmont, CA: Wadsworth.
10. Broadus, M. (1990). Multimedia in post secondary education. Computerdata, 15, 43.
11. Chan, D., & Schmitt, N. (1997). Video-based versus paper-and-pencil method of assessment in situational judgment tests: Subgroup differences in test performance and face validity perceptions, journal of Applied Psychology, 82, 300–310.
12. Chan, K. -Y., Drasgow, F., & Sawin, L. L. (1999). What is the shelf life of a test? The effect of time on the psychometrics of a cognitive ability test battery over 16 years. Journal of Applied Psychology, 84, 610–619.
13. Chiu, C. -Y., Hong, Y. -Y., Mischel, W., & Shoday, Y. (1995). Discriminative facility in social competence: Conditional versus dispositional encoding and monitoring-blunting of information. Social Cognition, 13, 49–70.
14. Clauser, B. E., Margolis, M. J., Clyman, S. G., & Ross, L. P. (1997). Development of automated scoring algorithms for complex performance assessments: A comparison of two approaches, journal of Educational Measurement, 34, 141–161.
15. Clyman, S. G., Melnick, D. E., & Clauser, B. E. (1999). Computer-based case simulations from medicine: Assessing skills in patient management. In A. Tekian, C. H. Mcguire, & W. E. Mcgahie, Innovative simulations for assessing professional competence (pp. 29–41). Chicago: Department of Medical Education, University of Illinois.
16. Dalessio, A. T. (1994). Predicting insurance agent turnover using a video-based situational judgment test, journal of Business and Psychology, 9, 23–32.
17. Desmarais, L. Burris, Dyer, P. J., Midkiff, K. R., Barbera, K. M., Curtis, J. R., Esrig, F. H., & Masi, D. L. (1992, May). Scientific uncertainties in the development of a multimedia test: Tradeoffs and decisions. Paper presented at the annual conference of the Society for Industrial and Organizational Psychology, Montreal, Quebec.
18. Desmarais, L. Burris, Masi, D. L., Olson, M. J., Barbara, K. M., & Dyer, P. J. (1994, April). Scoring a multimedia situational judgment test: Ibm's experience. Paper presented at the annual conference of the Society for Industrial and Organizational Psychology, Nashville, TN.
19. Drasgow, F. (2005). Innovative computerized test items. In K. Kempf-Leonard, Encyclopedia of social measurement (Vol. 2, pp. 283–290). San Diego, CA: Academic Press.
20. Drasgow, F., & OIson-Buchanan, J. B. (). (1999). Innovations in computerized assessment. Mahwah, NJ: Lawrence Erlbaum Associates.
21. Drasgow, F., OIson-Buchanan, J. B., & Moberg, P. J. (1999). Development of an interactive video assessment: Trials and tribulations. In F. Drasgow & J. B. Oison-Buchanan, Innovations in computerized assessment (pp. 177–196). Mahwah, NJ: Lawrence Erlbaum Associates.
22. Drasgow, F., Olson, J. B., Keenan, P. A., Moberg, P. J., & Mead, A. D. (1993). Computerized assessment. In G. R. Ferris & K. M. Rowland, Research in personnel and human resources management, (Vol. 11, pp. 163–206). Greenwich, CT: Jai Press.
23. Dyer, P. J., Desmarais, L. B., Midkiff, K. R., Colihan, J. P, & Olson, J. B. (1992, May). Designing a multimedia test: Understanding the organizational charge, building the team, and making the basic research commitments. Paper presented at the annual conference of the Society for Industrial and Organizational Psychology, Montreal, Quebec.
24. Flanagan, J. C. (1954). The critical incident technique. Psychological Bulletin. 51, 327–358.
25. Haak, S. W. (1993). Development of a computer tool including interactive video simulation for eliciting and describing clinical nursing judgment performance. Unpublished doctoral dissertation, The University of Utah, College of Nursing, Salt Lake City, Utah.
26. Hanson, M. A., Borman, W. C., Mogilka, H. J., Manning, C., & Hedge, J. W. (1999). Computerized assessment of skill for a highly technical job. In F. Drasgow & J. B. Oison-Buchanan, Innovations in computerized assessment (pp. 197–220). Mahwah, NJ: Lawrence Erlbaum Associates.
27. Hulin, C. L., Drasgow, F., & Parsons, C. K. (1983). Item response theory: Applications to psychological measurement. Homewood, IL: Dow Jones-Irwin
28. Hunter, D. R. (2003). Measuring general aviation pilot judgment: Using a situational judgment technique. The International journal of Aviation Psychology, 13, 373–386.
29. Hunter, J. E. (1986). Cognitive ability, cognitive aptitudes, job knowledge, and job performance. Journal of Vocational Behavior. 29, 340–362.
30. Hunter, J. E., & Hunter, R. F. (1984). Validity and utility of alternative predictors of job performance. Psychological Bulletin, 96, 72–98.
31. Interactive video training. (1991). Ibm Directions, 5, 8–9.
32. Johnson, D. F. (1983). Things we can measure through technology that we could not measure before. In New directions for testing and measurement: Measurement, technology, and individuality in education (Proceedings of the 1982 Educational Testing Services Invitational Conference, pp. 13–18). Princeton, NJ: Educational Testing Service.
33. Jones, C., & DeCotiis, T. A. (1986, August). Video-based selection of hospitality employees. The Cornell Hotel and Restaurant Administration Quarterly, pp. 67–73.
34. Keenan, P. A., & Olson, J. B. (1991, April). A model-based multi-media technique for assessing conflict management skills. Paper presented at the annual conference of the Society for Industrial and Organizational Psychology Conference, St. Louis, MO.
35. Kleinmann, M., & Straufi, B. (1998). Validity and application of computer-simulated scenarios in personnel assessment. International Journal of Selection and Assessment, 6, 97–106.
36. Konradt, U., Hertel, G., & Joder, K. (2003). Web-based assessment of call center agents: Development and validation of a computerized instrument. International Journal of Selection and Assessment, 11, 184–193.
37. Leary, T. (1957). Interpersonal diagnosis of personality. New York: Ronald.
38. Lievens, F., Buyse, T., & Sackett, P. R. (2005). The operational validity of a video-based situational judgment test for medical college admissions: Illustrating the importance of matching predictor and criterion construct domains. Journal of Applied Psychology, 90(3), 442–452.
39. Lievens, F., & Coetsier, P. (2002). Situational tests in student selection: An examination of predictive validity, adverse impact, and construct validity. International Journal of Selection and Assessment, 10, 245–257.
40. Lord, F. M. (1980). Applications of item response theory to practical testing problems. Hillsdale, NJ: Lawrence Erlbaum Associates.
41. McDaniel, M. A., Morgeson, F. P., Finnegan, E. B., Campion, M. A., & Braverman, E. P. (2001). Use of situational judgment tests to predict job performance: A clarification of the literature. Journal of Applied Psychology, 86, 730–740.
42. McHenry, J. J., & Schmitt, N. (1994). Multimedia testing. In M. G. Rumsey & C. B. Walker, Personnel selection and classification (pp. 193–232). Hillsdale, NJ: Lawrence Erlbaum Associates.
43. Mead, A. (1991, April). Hardware and software for multi-media computerized assessments. Paper presented at the annual conference of the Society for Industrial and Organizational Psychology, St. Louis, MO.
44. Mead, A. D., & Drasgow, F. (1993). Equivalence of computerized and paper-and-pencil cognitive ability tests: A meta-analysis. Psychological Bulletin, 114, 449–458.
45. Midkiff, K. R., Dyer, P. J., Desmarais, L. B., Rogg, K., & McCusker, C. R. (1992, May). The multimedia test: Friend or foe1. Paper presented at the annual conference of the Society for Industrial and Organizational Psychology Conference, Montreal, Quebec.
46. Mischel, W., & Shoda, Y. (1998). Reconciling processing dynamics and personality dimensions. Annual Review of Psychology, 49, 229–258.
47. Motowidlo, S. J., Borman, W. C., & Schmit, M. J. (1997). A theory of individual differences in task and contextual performance. Human Performance, 10, 71–83.
48. Motowidlo, S. J., Dunnette, M. D., & Carter, G. W. (1990). An alternative selection procedure: The low-fidelity simulation. Journal of Applied Psychology, 75, 640–647.
49. Naglieri, J. A., Drasgow, F., Schmit, M., Handler, L., Prifitera, A., Margolis, A., & Velasquez, R. (2004). Psychological testing on the Internet: New problems, old issues. American Psychologist. 59, 150–162.
50. OIson-Buchanan, J. B. (2001). Computer-based advances in assessment. In F. Drasgow & N. Schmitt, Measuring and analyzing behavior in organizations (pp. 44–87). San Francisco, CA: Jossey-Bass.
51. OIson-Buchanan, J. B., Drasgow, F., Moberg, P. J., Mead, A. D., Keenan, P. A., Donovan, M. A. (1998). Interactive video assessment of conflict resolution skills. Personnel Psychology, 51, 1–24.
52. Olson, J. B., & Keenan, P. A. (1994, April). Scoring interactive video assessments: A model-based approach. Paper presented at the annual conference of the Society for Industrial and Organizational Psychology Conference, Nashville, TN
53. Petty, L. C., & Rosen, E. F. (1987). Computer-based interactive video systems. Behavior Research Methods, Instruments, and Computers, 19, 160–166.
54. Pine, D. E. (1994, April). The development and validation of a video-based situational response test. Paper presented at the annual conference of the Society for Industrial and Organizational Psychology, Nashville, TN.
55. Powers, D. E., Burstein, J. C., Chodorow, M. S., Fowles, M. E., & Kukich, K. (2002). Comparing the validity of automated and human scoring of essays. Journal of Educational Computing Research, 26, 407–425
56. Reynolds, D. H., Sinar, E. F., Scott, D. R., & McClough, A. C. (2000, April). Evaluation of a Web-based selection procedure. Paper presented at the annual conference of the Society for Industrial and Organizational Psychology, New Orleans, LA.
57. Richman-Hirsch, W. L., OIson-Buchanan, J. B., & Drasgow, F. (2000). Examining the impact of administration medium on examinee perceptions and attitudes. Journal of Applied Psychology, 85, 880–887.
58. Schmitt, N., Gilliland, S. W., Landis, R. S., & Devine, D. (1993). Computer-based testing applied to selection of secretarial applicants. Personnel Psychology, 46, 149–165.
59. Shoda, Y., Mischel, W., & Wright, J. C. (1993). The role of situational demands and cognitive competencies in behavior organization and personality congruence. Journal of Personality and Social Psychology, 65, 1023–1035.
60. Sinar, E. F., Reynolds, D. H., & Paquet, S. L. (2003). Nothing but 'net'? Corporate image and web-based testing. International Journal of Selection and Assessment, 11, 150–157
61. Smiderle, D., Perry, B. A., & Cronshaw, S. F. (1994). Evaluation of video-based assessment in transit operator selection. Journal of Business and Psychology, 9. 3–22.
62. Thorndike, R. L. (1949). Personnel selection. New York: Wiley.
63. Trist, E. L. (1981). The evolution of sociotechnical systems: A conceptual framework and an action research program. Toronto: Ontario Quality of Working Life Centre.
64. Trist, E. L., & Bamforth, K. W. (1951). Some social and psychological consequences of the longwall method of coal-getting. Human Relations, 4, 1–38.
65. Walker, D., & Goldenberg, R. (2004, June). Bringing selection into the 21st century: A look at video-based testing within U. S. Customs and Border Protection. Paper presented at the annual conference of the International Public Management Association-Assessment Council, Seattle, WA.
66. Vlug, T., Furcon, J. E., Mondragon, N., & Mergen, C. Q. (2000, April). Validation and implementation of Web-based screening system in the Netherlands. Paper presented at the annual conference of the Society for Industrial and Organizational Pscyhology, New Orleans, LA.
67. Weekley, J. A., & Jones, C. (1997). Video-based situational testing. Personnel Psychology, 50, 25–49.