Exploring issues of examinee behavior: Insights gained from response-time analyses

Computer-Based Testing: Building the Foundation for Future Assessments

Volumn , Issue , 2005, Pages 237-266

  Schnipke, Deborah L ^a   Scrams, David J ^a  

^a Law School Admissions Council   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 85071808253     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.4324/9781410612250-20     Document Type: Chapter

References (83)

1. Baxter, B. (1941). An experimental analysis of the contributions of speed and level in an intelligence test. Journal of Educational Psychology, 32, 285-296.
2. Bergstrom, B., Gershon, R., & Lunz, M. E. (1994, April). Computer adaptive testing: Exploring examinee response time using hierarchical linear modeling. Paper presented at the annual meeting of the National Council on Measurement in Education, New Orleans, LA.
3. Bhola, D. S., Plake, B. S., & Roos, L. L. (1993, October). Setting an optimum time limit for a computer-administered test. Paper presented at the meeting of the Midwestern Educational Research Association, Chicago.
4. Blommers, P., & Lindquist, E. F. (1944). Rate of comprehension of reading: Its measurement and its relation to comprehension. Journal of Educational Psychology, 35(8), 449-473.
5. Bontempo, B. D., & Julian, E. R. (1997, March). Assessing speededness in variable-length computer adaptive testing. Paper presented at the annual meeting of the National Council on Measurement in Education, Chicago.
6. Bridges, K. R. (1985). Test-completion speed: Its relationship to performance on three course-based objective examinations. Educational and Psychological Measurement, 45, 29-35.
7. Davey, T. (1990, April). Modeling timed test performance. Paper presented at the meeting of the American Educational Research Association, Boston.
8. Davidson, W. M., & Carroll, J. B. (1945). Speed and level components in time-limit scores: A factor analysis. Educational and Psychological Measurement, 5, 411-427.
9. Embretson, S. E. (1983). Construct validity: Construct representation versus nomothetic span. Psychological Bulletin, 93, 179-197.
10. Embretson, S. E. (1997). Multicomponent response models. In W. J. van der Linden and R. K. Hambleton (Eds.), Handbook of modern item response theory (pp. 305-322). New York: Springer.
11. Foos, P. W. (1989). Completion time and performance on multiple-choice and essay tests. Bulletin of the Psychonomic Society, 27, 179-180.
12. Frederiksen, N., Mislevy, R. J., & Bejar, I.I. (Eds.). (1993). Test theory for a new generation of tests. Hillsdale, NJ: Lawrence Erlbaum Associates.
13. Fumeaux, W. D. (1961). Intellectual abilities and problem solving behavior. In H. J. Eysenck (Ed.), The handbook of abnormal psychology (pp. 167-192). London: Pitman.
14. Gitomer, D. H., Curtis, M. E., Glaser, R., & Lensky, D. B. (1987). Processing differences as a function of item difficulty in verbal analogy performance. Journal of Educational Psychology, 79, 212-219.
15. Guilford, J. P., & Zimmerman, W. F. (1953). The Guilford-Zimmerman aptitude survey. IV. Spatial Visualization Form B. Beverly Hills, CA: Sheridan Supply Company.
16. Gulliksen, H. (1950/1987). Theory of mental tests. Hillsdale, NJ: Lawrence Erlbaum Associates.
17. Hadadi, A., & Luecht, R. M. (1998, April). Effects of assessing speededness/non-speededness in computerized tests. Paper presented at the annual meeting of the National Council on Measurement in Education, San Diego, CA.
18. Halkitis, P. N., Jones, J. P., & Pradhan, J. (1996, April). Estimating testing time: The effects of item characteristics on response latency. Paper presented at the annual meeting of the American Educational Research Association, New York.
19. Holden, R. R. (1993, August). Response latency detection of lying on personnel tests. Paper presented at the annual meeting of the American Psychological Association, Toronto, Canada.
20. Holden, R. R., & Kroner, D. G. (1992). Relative efficacy of differential response latencies for detecting faking on a self-report measure of psychopathology. Psychological Assessment: A Journal of Consulting and Clinical Psychology, 4, 170-173.
21. Holden, R. R., Kroner, D. G., Fekken, G. C., & Popham, S. M. (1992). A model of personality test item response dissimulation. Journal of Personality and Social Psychology, 63, 272-279.
22. Jenkins, S. M., & Holmes, S. D. (1997). Computer usage and access patterns of actual and potential LSAT takers. (LSAC Computerized Testing Report 97-10). Newtown, PA: Law School Admission Council.
23. Jensen, A. R. (1982a). Reaction time and psychometric g. In H. J. Eysenck (Ed.), A model for intelligence. New York: Springer.
24. Jensen, A. R. (1982b). The chronometry of intelligence. In R. J. Sternberg (Ed.), Advances in the psychology of human intelligence (Vol. 1). London: Lawrence Erlbaum Associates.
25. Kendler, H. H. (1987). Historical foundations of modem psychology. Pacific Grove, CA: Brooks/Cole.
26. Kennedy, M. (1930). Speed as a personality trait. Journal of Social Psychology, 1, 286-292.
27. Klineberg, O. (1935). Race differences. New York: Harper.
28. Llabre, M. M., & Froman, T. W. (1987). Allocation of time to test items: A study of ethnic differences. Journal of Experimental Education, 55, 137-140.
29. Luce, R. D. (1986). Response times: Their role in inferring elementary mental organization. New York: Oxford University Press.
30. Masters, G. N., & Mislevy, R. J. (1993). New views of student learning: Implications for educational measurement. In N. Frederiksen, R. J. Mislevy, & I. I. Bejar (Eds.), Test theory for a new generation of tests (pp. 219-242). Hillsdale, NJ: Lawrence Erlbaum Associates.
31. Mislevy, R. J. (1993). Foundations of a new test theory. In N. Frederiksen, R. J. Mislevy, & I. I. Bejar (Eds.), Test theory for a new generation of tests (pp. 19-39). Hillsdale, NJ: Lawrence Erlbaum Associates.
32. Mislevy, R. J. (1994). Evidence and inference in educational assessment. Psychometrika, 59, 439-483.
33. Mislevy, R. J. (1996). Test theory reconceived. Journal of Educational Measurement, 33, 379-416.
34. Mislevy, R. J., & Verhelst, N. (1990). Modeling item responses when different subjects employ different solution strategies. Psychometrika, 55, 195-215.
35. Mislevy, R. J., Wingersky, M. S., Irvine, S. H., & Dann, P. L. (1991). Resolving mixtures of strategies in spatial visualization tasks. British Journal of Mathematical and Statistical Psychology, 44, 265-288.
36. Mollenkopf, W. G. (1960). Time limits and the behavior of test takers. Educational and Psychological Measurement, 20, 223-230.
37. Morrison, E. J. (1960). On test variance and the dimensions of the measurement situation. Educational and Psychological Measurement, 20, 231-250.
38. Myers, C. T. (1952). The factorial composition and validity of differently speeded tests. Psychometrika, 77(3), 347-352.
39. O'Neill, K., & Powers, D. E. (1993, April). The performance of examinee subgroups on a computer-administered test of basic academic skills. Paper presented at the meeting of the National Council on Measurement in Education, Atlanta, GA.
40. Parshall, C. G., Mittelholtz, D., & Miller, T. R. (1994, April). Response time: An investigation into determinants of item-level timing. In C. G. Parshall (Ed.), Issues in the development of a computer adaptive placement test. Symposium conducted at the meeting of the National Council on Measurement in Education, New Orleans, LA.
41. Posner, M. I., & Boies, S. (1971). Components of attention. Psychological Review, 78, 391-408.
42. Rasch, G. (1960). Probabilistic models for some intelligence and attainment tests. Copenhagen: Danish Institute for Educational Research.
43. Raven, J. (1956). Progressive matrices. London: Lewis.
44. Reese, C. M. (1993, April). Establishing time limits for the GRE computer adaptive tests. In W. D. Way (Chair), Practical problems in the development of large scale computer adaptive tests. Symposium conducted at the meeting of the National Council on Measurement in Education, Atlanta, GA.
45. Rindler, S. E. (1979). Pitfalls in assessing test speededness. Journal of Educational Measurement, 76(4), 261-270.
46. Roberts, A. H., & Greene, J. E. (1971). Cross-cultural study of relationships among four dimensions of time perspective. Perceptual and Motor Skills, 33, 163-173.
47. Roskam, E. E. (1997). Models for speed and time-limit tests. In W. J. van der Linden and R. K. Hambleton (Eds.), Handbook of modern item response theory (pp. 187-208). New York: Springer.
48. Samejima, F. (1973). Homogeneous case of the continuous response level. Psychometrika, 38, 203-219.
49. Samejima, F. (1974). Normal ogive model on the continuous response level in the multidimensional latent space. Psychometrika, 39, 111-121.
50. Samejima, F. (1983a). A latent trait model for differential strategies in cognitive processes (Technical Report ONR/RR-81-1). Knoxville, TN: University of Tennessee.
51. Samejima, F. (1983b). A general model for the homogeneous case of the continuous response (ONR Research Report 83-3). Arlington, VA: Office of Nsval Research, Personnel and Training Research Programs.
52. Schaeffer, G. A., Reese, C. M., Steffen, M., McKinley, R. L, & Mills, C. N. (1993). Field test of a computer-based ORE General Test (ETS Research Report 93-07). Princeton, NJ: Educational Testing Service.
53. Schaeffer, G. A., Steffen, M., Golub-Smith, M. L., Mills, C. N., & Durso, R. (1995). The introduction and comparability of the computer adaptive ORE General Test (ETS Research Report 95-20). Princeton, NJ: Educational Testing Service.
54. Scheiblechner, H. (1979). Specific objective stochastic latency mechanisms. Journal of Mathematical Psychology, 19, 18-38.
55. Scheiblechner, H. (1985). Psychometric models for speed-test construction: The linear exponential model. In S. E. Embretson (Ed.), Test design: Developments in psychology and psychometrics (pp. 219-244). Orlando, FL: Academic Press.
56. Schnipke, D. L. (1995). Assessing speededness in computer-based tests using item response times (Doctoral dissertation, Johns Hopkins University, 1995). Dissertation Abstracts International, 57, B759.
57. Schnipke, D. L., & Pashley, P. J. (1997, March). Assessing subgroup differences in item response times. Paper presented at the meeting of the National Council on Measurement in Education, Chicago.
58. Schnipke, D. L., & Scrams, D. J. (1997a). Modeling item response times with a two-state mixture model: A new method of measuring speededness. Journal of Educational Measurement, 34, 213-232.
59. Schnipke, D. L., & Scrams, D. J. (1997b). Representing response-time information in item banks (LSAC Computerized Testing Report 97-09). Newtown, PA: Law School Admission Council.
60. Scrams, D. J., & Schnipke, D. L. (1997). Making use of response times in standardized tests: Are accuracy and speed measuring the same thing? Paper presented at the meeting of the National Council on Measurement in Education, Chicago.
61. Scrams, D. J., & Schnipke, D. L. (1999, April). Response-time feedback on computeradministered tests. Paper presented at the meeting of the National Council on Measurement in Education, Montreal, Canada.
62. Segall, D. O. (1987). Relation between estimated ability and test time on the CAT-ASVAB. Unpublished manuscript.
63. Segall, D. O., Moreno, K. E., Kieckhaefer, W. F., Vicino, F. L., & McBride, J. R. (1997). Validation of the experimental CAT-ASVAB system. In W. A. Sands, B. K. Waters, & J. R. McBride (Eds.), Computerized adaptive testing: From inquiry to operation (pp. 103-114). Washington, DC: American Psychological Association.
64. Shannon, L. (1976). Age change in time perception in Native Americans, Mexican Americans, and Anglo-Americans. Journal of Cross-Cultural Psychology, 7, 117-122.
65. Snow, R. E., & Lohman, D. F. (1989). Implications of cognitive psychology for educational measurement. In R. L. Linn (Ed.), Educational Measurement (3rd ed., pp. 263-331). New York: American Council on Education and Macmillan.
66. Spearman, C. (1927). The abilities of man. New York: Macmillan.
67. Stemberg, S. (1970). Memory scanning: Mental processes revealed by reaction time experiments. In J. S. Antrobus (Ed.), Cognition and affect (pp. 13-58). Boston: Little, Brown.
68. Sutton, R. E. (1991). Equity and computers in the schools: A decade of research. Review of Educational Research, 67(4), 475-503.
69. Swanson, D. B., Featherman, C. M., Case, S. M., Luecht, R., & Nungester, R. (1997, March). Relationship of response latency to test design, examinee proficiency and item difficulty in computer-based test administration. Paper presented at the meeting of the National Council on Measurement in Education, Chicago.
70. Swygert, K. A. (1998). An examination of item response times on the GRE-CAT. Unpublished doctoral dissertation, University of North Carolina, Chapel Hill.
71. Tate, M. W. (1948). Individual differences in speed of response in mental test materials of varying degrees of difficulty. Educational and Psychological Measurement, 8, 353-374.
72. Tatsuoka, K. K. (1990). Toward an integration of item-response theory and cognitive error diagnosis. In N. Frederiksen, R. Glaser, A. Lesgold, & M. G. Shafto (Eds.), Diagnostic monitoring of skill and knowledge acquisition. Hillsdale, NJ: Lawrence Erlbaum Associates.
73. Tatsuoka, K. K., & Tatsuoka, M. M. (1978). Time-score analysis in criterion-referenced tests (CERL Report E-1). Urbana, IL: University of Illinois, Computer-Based Education Research Laboratory.
74. Tatsuoka, K. K., & Tatsuoka, M. M. (1980). A model for incorporating response-time data in scoring achievement tests. In D. J. Weiss (Ed.), Proceedings of the 1979 Computerized Adaptive Testing Conference (pp. 236-256). Minneapolis, MN: University of Minnesota, Department of Psychology, Psychometric Methods Program.
75. Thissen, D. (1983). Timed testing: An approach using item response theory. In D. J. Weiss (Ed.), New horizons in testing: Latent trait test theory and computerized adaptive testing (pp. 179-203). New York: Academic Press.
76. Tinsley, H. E. (1971). An investigation of the Rasch simple logistic model for tests of Intelligence or attainment. Unpublished doctoral dissertation, University of Minnesota.
77. Townsend, J. T., & Ashby, F. G. (1983). The stochastic modeling of elementary psychological processes. Cambridge: Cambridge University Press.
78. van der Linden, W. J., Scrams, D. J., & Schnipke, D. L. (1999). Using response-time constraints to control for differential speededness in computerized adaptive testing. Applied Psychological Measurement, 23, 195-210.
79. Verhelst, N. D., Verstralen, H. H. F. M., & Jansen, M. G. H. (1997). A logistic model for timelimit tests. In W. J. van der Linden & R. K. Hambleton (Eds.), Handbook of modern item response theory (pp. 169-185). New York: Springer.
80. Whitely, S. E. (1977a). Information-processing on intelligence test items: Some response components. Applied Psychological Measurement, 1, 465-476.
81. Whitely, S. E. (1977b). Relationships in analogy items: A semantic component of a psychometric task. Educational and Psychological Measurement, 37, 725-739.
82. Yamamoto, K. (1989). Hybrid model of IRT and latent class models (ETS Research Report 89-41). Princeton, NJ: Educational Testing Service.
83. Yamamoto, K., & Everson, H. T. (1995). Modeling the mixture of IRT and pattern responses by a modified hybrid model (ETS Research Report 95-16). Princeton, NJ: Educational Testing Service.