Foundations for Measurement

Technology-Enhanced Assessment of Talent

Volumn , Issue , 2011, Pages 19-65

  Scott, John C ^a   Mead, Alan D ^b  

^a APTMetrics Inc   (United States)

^b ILLINOIS INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

Assessment plan; Assessment specifications; High quality assessments; Technology enhanced assessments; Unproctored internet testing

Indexed keywords

EID: 84886201315     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.1002/9781118256022.ch2     Document Type: Chapter

References (63)

1. American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (1999). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.
2. Beaty, J.C., Grauer, E., & Davis, J. (2006). Unproctored internet testing: Important questions and empirical answers. Paper presented at the Practitioner forum conducted at the 21st Annual Meeting of the Society of Industrial and Organizational Psychology, Dallas, Texas.
3. Bejar, I.I. (1983). Achievement testing: Recent advances. In J.L. Sullivan & R.G. Niemi (Eds.), Quantitative applications in the social sciences. (No. 07-036). Thousand Oaks, CA: Sage.
4. Biskin, B.H., & Kolotkin, R.L. (1977). Effects of computerized administration on scores on the Minnesota Multiphasic Personality Inventory. Applied Psychological Measurement, 1(4), 543-549.
5. Boyle, S. (1984). The effect of variations in answer-sheet format on aptitude test performance. Journal of Occupational Psychology, 57, 323-326.
6. Breithaupt, K.J., Mills, C.N., & Melican, G.J. (2006). Facing the opportunities of the future. In D. Bartram & R.K. Hambleton (Eds.), Computer-based testing and the internet: Issues and advances, pp. 231-251.
7. Buchanan, T. (2002). Online assessment: Desirable or dangerous? Professional Psychology: Research and Practice, 33, 148-154.
8. Buchanan, T., Johnson, J.A., & Goldberg, L.R. (2005). Implementing a five-factor personality inventory for use on the Internet. European Journal of Psychological Assessment, 21(2), 115-127.
9. Bugbee, A.C., Jr. (1996). The equivalence of paper-and-pencil and computer-based testing. Journal of Research on Computing in Education, 28(3), 282-99.
10. Byrne, B.M. (1998). Structural equations modeling with LISREL, PRELIS, and SIMPLIS: Basic concepts, applications, and programming. Mahwah, NJ: Lawrence Erlbaum Associates.
11. Candell, G.L., & Drasgow, F. (1988). An iterative procedure for linking metrics and assessing item bias in item response theory. Applied Psychological Measurement, 12(3), 253-260.
12. Canoune, H.L., & Leyhe, E.W. (1985). Human versus computer interviewing. Journal of Personality Assessment, 49, 103-106.
13. Drasgow, F., Nye, C.D., & Tay, L. (2010). Indicators of quality assessment (pp. 22-59). In J.C. Scott & D.H. Reynolds (Eds.), Handbook of workplace assessment: Selecting and developing organizational talent. San Francisco: Jossey Bass.
14. Equal Employment Opportunity Commission, Civil Service Commission, United States Department of Labor, & Department of Justice. (1978). Uniform guidelines on employee selection procedures. Federal Register, 43(166), 38290-38315.
15. Fox, S., & Livingston, G. (2007). Latinos online: Hispanics with lower levels of education and English proficiency remain largely disconnected from the Internet [Electronic Version] http://pewhispanic.org/files/reports/73.pdf.
16. Greaud, V.A., & Green, B.F. (1986). Equivalence of conventional and computer presentation of speed tests. Applied Psychological Measurement, 10, 23-34.
17. Guion, R.M. (1998). Assessment, measurement, and prediction for personnel decisions. Mahwah, NJ: Lawrence Erlbaum Associates.
18. Hambleton, R.K., & Pitoniak, M.J. (2002). Testing and measurement: Advances in item response theory and selected testing practices. In J. Wixted (Ed.), Stevens'handbook of experimental psychology (3rd ed., Vol. 4, pp. 517-561). Hoboken, NJ: John Wiley & Sons.
19. Hausknecht, J.P., Halpert, J.A., Di Paolo, N.T., Moriarty, N.T., & Gerrard, M.O. (2007). Retesting in selection: A meta-analysis of practice effects for tests of cognitive ability. Journal of Applied Psychology.
20. Holland, W.H., & Wainer, H. (1993). Differential item functioning. Mahwah, NJ: Lawrence Erlbaum Associates.
21. Horrigan, J. (2009). Home broadband adoption increases sharply in 2009 with big jumps among seniors, low-income households, and rural residents even though prices have risen since last year. [Electronic Version] www.pewinternet.org/Press-Releases/2009/Home-broadband-adoption-increases-sharply-in-2009.aspx.
22. International Test Commission (2006). International guidelines on computer-based and internet-delivered testing. International Journal of Testing, 6(2), 143-171.
23. Jodoin, M.G. (2003, Spring). Measurement efficiency of innovative item formats in computer-based testing. Journal of Educational Measurement, 40(1), 1-15.
24. Kolen, M.J., & Brennan, R.L. (2004). Test equating, scaling, and linking: Methods and practices. New York: Springer-Verlag.
25. Lievens, F., Buyse, T., & Sackett, P.R. (2005). Retest effects in operational selection settings: Development and test of a framework. Personnel Psychology, 58, 981-1007.
26. Liu, C., Borg, I., & Spector, P.E. (2004). Measurement equivalence of the German job satisfaction survey used in a multinational organization:implications of Schwartz's culture model. Journal of Applied Psychology, 89(6), 1070-1082.
27. Mazzeo, J., & Harvey, A.L. (1988). The equivalence of scores from automated and conventional educational and psychological tests: A review of the literature (ETS RR-88-21). Princeton, NJ: Educational Testing Service.
28. McPhail, S.M. (Ed.) (2007). Alternative validation strategies: Developing new and leveraging existing validation evidence. San Francisco: Jossey-Bass.
29. McPhail, S.M., & Stelly, D.J. (2010). Validation strategies (pp. 671-710). In J.C. Scott & D.H. Reynolds (Eds.), Handbook of workplace assessment: Selecting and developing organizational talent. San Francisco: Jossey Bass.
30. Mead, A.D. (2010, April). Non-comparability of speeded computerized tests: Differential speededness? Paper presented at the Annual Meeting of the Society of Industrial and Organizational Psychology, Atlanta, Georgia.
31. Mead, A.D., & Blitz, D.L. (April, 2003). Comparability of paper and computerized non-cognitive measures: A review and integration. Paper presented at the annual meeting of the Society of Industrial and Organizational Psychology, Orlando, Florida.
32. Mead, A.D. & Drasgow, F. (1993). Equivalence of computerized and paper-and-pencil cognitive ability tests: A meta-analysis. Psychological Bulletin, 114(3), 449-458.
33. Mead, A.D., Segall, D.O., Williams, B.A., & Levine, M.V. (1999, April). Multidimensional assessment for multidimensional minds: Leveraging the computer to assess personality comprehensively, accurately, and briefly. A paper presented at the twelfth annual conference for the Society for Industrial and Organizational Psychology, St. Louis, Missouri.
34. Meade, A.W., Michels, L.C., & Lautenschlager, G.J. (2007). Are internet and paper-and-pencil personality tests truly comparable? An experimental design measurement invariance study. Organizational Research Methods, 10(2), 322-345.
35. Neuman, G., & Baydoun, R. (1998). Computerization of paper-andpencil tests: When are they equivalent? Applied Psychological Measurement, 22(1), 71-83.
36. Ones, D.S., & Viswesvaran, C. (1998). The effects of socially desirability and faking on personality and integrity assessment for personnel selection. Human Performance, 11, 245-269.
37. Outtz, J.L. (2010). Addressing the flaws in our assessment decisions. In J.C. Scott & D.H. Reynolds (Eds.), Handbook of workplace assessment: Selecting and developing organizational talent (pp. 711-728). San Francisco: Jossey-Bass.
38. Paek, P. (2005, August). Recent trends in comparability studies. PEM Research Report 05-05. Upper Saddle River, NJ: Pearson Educational Measurement.
39. Parshall, C.G., Davey, T., & Pashley, P.J. (2000). Innovative item types for computerized testing. In W.J. van der Linder & C.A.W. Glas (Eds.), Computerized adaptive testing: Theory and practice (pp. 129-148). Dordrecht: Kluwer.
40. Pearlman, K. (2009). Unproctored internet testing: Practical, legal, and ethical concerns. Industrial and Organizational Psychology, 2(1), 14-19.
41. Ployhart, R.P., & Oswald, F.L. (2004). Applications of mean and covariance structure analysis: Integrating correlational and experimental approaches. Organizational Research Methods, 7, 27-65.
42. Ployhart, R.E., Weekley, J.A., Holtz, B.C., & Kemp, C.F. (2002, April). Web-based vs. paper and pencil testing: A comparison of factor structures across applicants and incumbents. Paper presented at the 17th annual conference of the Society for Industrial and Organizational Psychology, Toronto, Ontario.
43. Pomplun, M., Frey, S., & Becker, D.F. (2002). The score equivalence of paper-and-pencil and computerized versions of a speeded test of reading comprehension. Educational and Psychological Measurement, 62, 337-354.
44. Rainie, L., Estabrook, L., & Witt, E. (2007). Information searches that solve problems [Electronic Version] www.pewinternet.org/Reports/2007/Information-Searches-That-Solve-Problems.aspx.
45. Raju, N.S., & Ellis, B.B. (2002). Differential item and test functioning. In F. Drasgow & N. Schmitt (Eds.), Measuring and analyzing behavior in organizations: Advances in measurement and data analysis (pp. 156-188). San Francisco: Jossey-Bass.
46. Raju, N.S., Laffitte, L.J., & Byrne, B.M. (2002). Measurement equivalence: A comparison of confirmatory factor analysis and item response theory. Journal of Applied Psychology, 87(3), 517-529.
47. Raymond, M.R., Neustel, S., & Anderson, D. (2008, March). The benefits of taking an identical version of a certification test on two occasions. Paper presented at the annual meeting of the National Council on Measurement in Education (NCME), New York, New York.
48. Reynolds, D.H., & Rupp, D.E. (2010). Advances in technologyfacilitated assessment. In J.C. Scott & D.H. Reynolds (Eds.), Handbook of workplace assessment: Selecting and developing organizational talent. San Francisco: Jossey-Bass.
49. Richman, W.L., Kiesler, S., Weisband, S., & Drasgow, F. (1999). A meta-analytic study of social desirability distortion in computeradministered questionnaires, traditional questionnaires, and interviews. Journal of Applied Psychology, 84(5), 754-775.
50. Rogers, J., Howard, K., & Vessey, J. (1993). Using significance tests to evaluate equivalence between two experimental groups. Psychological Bulletin, 113(3), 553-565.
51. Rosse, J.G., Stecher, M.D., Miller, J.L., & Levin, R.A. (1998). The impact of response distortion on preemployment personality testing and hiring decisions. Journal of Applied Psychology, 83(4), 634-644.
52. Ryan, A.M., Horvath, M., Ployhart, R.E., Schmitt, N., & Slade, L.A. (2000). Hypothesizing differential item functioning in global employee opinion surveys. Personnel Psychology, 53(3), 531-562.
53. Sacher, J., & Fletcher, J.D. (1978). Administering paper-and-pencil tests by computer, or the medium is not always the message. In D.J. Weiss (Ed.), Proceedings of the 1977 Computerized Adaptive Testing Conference (pp. 403-419). Minneapolis: University of Minnesota, Department of Psychology, Psychometric Methods Program.
54. Segall, D.O. (1996). Multidimensional adaptive testing. Psychometrika, 61, 331-354.
55. Society for Industrial and Organizational Psychology. (2003). Principles for the validation and use of personnel selection procedures (4th ed.). Bowling Green, OH: Author.
56. Stanton, J.M., & Rogelberg, S.G. (2001). Using internet/intranet web pages to collect organizational research data. Organizational Research Methods, 4, 199-216.
57. Stark, S., Chernyshenko, O.S. & Drasgow, F. (2006). Detecting differential item functioning with confirmatory factor analysis and item response theory: toward a unified strategy. Journal of Applied Psychology, 91(6), 1292-1306.
58. Tippins, N.T., Beaty, J., Drasgow, F., Gibson, W.M., Pearlman, K., Segall, D.O., & Shepherd, W.J. (2006). Unproctored internet testing in employment settings. Personnel Psychology, 59(1), 189-225.
59. Vandenberg, R.J., & Lance, C.E. (2000). A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3, 4-70.
60. Weiss, D.J. (2007). Adaptive-and electronic-testing: Past, present, and future. Invited address presented at the annual meeting of the National Council on Measurement in Education, Chicago, Illinois.
61. Weiss, D.J., & Gibbons, R.D. (2007). Computerized adaptive testing with the bifactor model. In D.J. Weiss (Ed.), Proceedings of the 2007 GMAC Conference on Computerized Adaptive Testing. Minneapolis: University of Minnesota, Department of Psychology, Psychometric Methods Program.
62. Zenisky, A.L., & Sireci, S.G. (2002). Technological innovations in largescale assessment. Applied Measurement in Education, 15(4), 337-362.
63. Zickar, M., Rosse, J., & Levin, R. (1996, April). Modeling the effects of faking on personality instruments. Paper presented at the 11th annual meeting of the Society for Industrial and Organizational Psychology, San Diego, California.