IRT-related factor analytic procedures for testing the equivalence of paper-and-pencil and Internet-administered questionnaires

Psychological Methods

Volumn 10, Issue 2, 2005, Pages 193-205

  Ferrando, Pere J ^a   Lorenzo Seva, Urbano ^a  

^a UNIVERSITAT ROVIRA I VIRGILI   (Spain)

Author keywords

Factor analysis; Internet tests; IRT; Measurement equivalence; Personality measurement

Indexed keywords

ARTICLE; FACTORIAL ANALYSIS; HUMAN; INTERNET; METHODOLOGY; PERSONALITY DISORDER; PSYCHOLOGICAL MODEL; PSYCHOLOGY; QUESTIONNAIRE;

FACTOR ANALYSIS, STATISTICAL; HUMANS; INTERNET; MODELS, PSYCHOLOGICAL; PERSONALITY DISORDERS; PSYCHOLOGY; QUESTIONNAIRES;

EID: 22544448084     PISSN: 1082989X     EISSN: None     Source Type: Journal    
DOI: 10.1037/1082-989X.10.2.193     Document Type: Article

References (62)

1. Aish, A. M., & Jöreskog, K. G. (1990). A panel model for political efficacy and responsiveness: An application of LISREL 7 with weighted least squares. Quality and Quantity, 24, 405-426.
2. Alsup, R., & Gillespie, D. F. (1997). Stability of attitudes toward abortion and sex roles: A two-factor measurement model at two points of time. Structural Equation Modeling, 4, 338-352.
3. Barak, A., & Cohen, L. (2002). Empirical examination of an online version of the self-directed search. Journal of Career Assessment, 10, 387-400.
4. Barak, A., & English, N. (2002). Prospects and limitations of psychological testing on the Internet. Journal of Technology in Human Services, 19, 65-89.
5. Browne, M. W., & Cudeck, R. (1989). Single sample cross-validation indices for covariance structures. Multivariate Behavioral Research, 24, 445-455.
6. Buchanan, T. (2000a). Internet research: Self-monitoring and judgments of attractiveness. Behavior Research Methods, Instruments & Computers, 32, 521-527.
7. Buchanan, T. (2000b). Potential of the Internet for personality research. In M. H. Birnbaum (Ed.), Psychological experiments on the Internet (pp. 121-265). San Diego, CA: Academic Press.
8. Buchanan, T., & Smith, J. L. (1999). Using the Internet for psychological research: Personality testing on the World Wide Web. British Journal of Psychology, 90, 125-144.
9. Christoffersson, A. (1975). Factor analysis of dichotomized variables. Psychometrika, 40, 5-31.
10. Committee on Professional Standards and Committee on Psychological Tests and Assessment. (1986). Guidelines for computer-based tests and interpretations. Washington, DC: American Psychological Association.
11. Cook, C., Heath, F., Thompson, R. L., & Thompson, B. (2001). Score reliability in Web-or-Internet-based surveys: Unnumbered graphic rating scales versus Likert-type scales. Educational and Psychological Measurement, 61, 697-706.
12. Cronbach, L. J. (1990). Essentials of psychological testing. New York: Harper & Row.
13. Cronk, B. C., & West, J. L. (2002). Personality research on the Internet: A comparison of Web-based and traditional instruments in take-home and in-class settings. Behavior Research Methods, Instruments & Computers, 34, 177-180.
14. Davis, R. N. (1999). Web-based administration of a personality questionnaire: Comparison with traditional methods. Behavior Research Methods, Instruments & Computers, 31, 572-577.
15. Epstein, J., Klinkenberg, W. D., Wiley, D., & McKinley, L. (2001). Insuring sample equivalence across Internet and paper-and-pencil assessments. Computers in Human Behavior, 17, 339-346.
16. Eysenck, S. B. G., Eysenck, H. J., & Barrett, P. T. (1985). A revised version of the Psychoticism scale. Personality and Individual Differences, 6, 21-29.
17. Ferrando, P. J. (2000). Testing the equivalence among different item response formats in personality measurement: A structural equation modeling approach. Structural Equation Modeling, 7, 271-286.
18. Ferrando, P. J. (2002). An IRT-based two-wave model for studying short-term stability in personality measurement. Applied Psychological Measurement, 26, 286-301.
19. Finch, J. F., & West, S. G. (1997). The investigation of personality structure: Statistical models. Journal of Research in Personality, 31, 439-485.
20. Fouladi, R. T., McCarthy, C. J., & Moller, N. P. (2002). Paper-and-pencil or online? Evaluating mode effects on measures of emotional functioning and attachment. Assessment, 9, 204-215.
21. Gati, I., & Saka, N. (2001). Internet-based versus paper-and-pencil assessment: Measuring career decision-making difficulties. Journal of Career Assessment, 9, 379-416.
22. Green, B. F. (1988). Construct validity of computer-based tests. In H. Wainer & H. I. Braun (Eds.), Test validity (pp. 77-86). Hillsdale, NJ: Erlbaum.
23. Green, B. F., Bock, R. D., Humphreys, L. G., & Reckase, M. D. (1984). Technical guidelines for assessing computerized adaptative tests. Journal of Educational Measurement, 21, 347-360.
24. Hetter, R. D., Segall, D. O., & Bloxom, B. M. (1997). Evaluating item calibration medium in computerized adaptive testing. In W. A. Sands, B. K. Waters, & J. R. McBride (Eds.), Computerized adaptive testing (pp. 161-168). Washington, DC: American Psychological Association.
25. Hewson, C. M., Laurent, D., & Vogel, C. M. (1996). Proper methodologies for psychological and sociological studies conducted via the Internet. Behavior Research Methods, Instruments & Computers, 28, 186-191.
26. Joinson, A. (1999). Social desirability, anonymity, and Internet-based questionnaires. Behavior Research Methods, Instruments & Computers, 31, 433-438.
27. Jöreskog, K. G. (1971). Statistical analysis of sets of congeneric tests. Psychometrika, 36, 109-133.
28. Jöreskog, K. G. (1990). New developments in LISREL: Analysis of ordinal variables using polychoric correlations and weighted least squares. Quality and Quantity, 24, 387-404.
29. Kapes, J. F., Martinez, L., Ip, C., Slivinski, T., & Hardwicke, S. (1998). Internet-based vs. paper-pencil occupational competency test administration: An equivalence study. Journal of Vocational Education Research, 23, 201-219.
30. Krantz, J. H., Ballard, J., & Scher, J. (1997). Comparing the results of laboratory and World-Wide Web samples on the determinants of female attractiveness. Behavior Research Methods, Instruments & Computers, 29, 264-269.
31. Kyllonen, P. C. (1991). Principles for creating computerized test battery. Intelligence, 15, 1-15.
32. Lord, F. M. (1952). A theory of test scores. Psychometrika Monographs, 7.
33. Lord, F. M. (1953). The relation of test score to the trait underlying the test. Educational and Psychological Measurement, 13, 517-548.
34. Lord, F. M. (1980). Applications of item response theory to practical testing problems. Hillsdale, NJ: Erlbaum.
35. Lunz, M. E., & Bergstrom, B. A. (1994). An empirical study of computerized adaptive test administration conditions. Journal of Educational Measurement, 31, 251-263.
36. McDonald, R. P. (1999). Test theory: A unified treatment. Mahwah, NJ: Erlbaum.
37. Meier, S. (1994). The chronic crisis in psychological measurement and assessment: A historical survey. San Diego, CA: Academic Press.
38. Mislevy, R. J., & Bock, R. D. (1990). BILOG 3: Item analysis and test scoring with binary logistic models. Mooresville, IN: Scientific Software.
39. Muthén, B. (1978). Contributions to factor analysis of dichotomous variables. Psychometrika, 43, 551-560.
40. Muthén, B. (1981). Factor analysis of dichotomous variables: American attitudes toward abortion. In D. J. Jackson & E. F. Borgatta (Eds.), Factor analysis and measurement in sociological research (pp. 201-213). London: Sage.
41. Muthén, B. (1984). A general structural equation model with dichotomous, ordered, categorical and continuous latent variable indicators. Psychometrika, 49, 115-132.
42. Muthén, B. (1993). Goodness of fit with categorical and other nonnormal variables. In K. A. Bollen & J. S. Long (Eds.), Testing structural equation models (pp. 205-234). Newbury Park, CA: Sage.
43. Muthén, L. K., & Muthén, B. (1999). Mplus user's guide. Los Angeles: Muthén & Muthen.
44. Pasveer, K. A., & Ellard, J. H. (1998). The making of a personality inventory: Help from the WWW. Behavior Research Methods, Instruments & Computers, 30, 309-313.
45. Pettit, F. A. (2002). A comparison of World-Wide Web and paper-and-pencil personality questionnaires. Behavior Research Methods, Instruments & Computers, 34, 50-54.
46. Pomplun, M., Frey, S., & Becker, D. F. (2002). The score equivalence of paper-and-pencil and computerized versions of a speeded test of reading comprehension. Educational and Psychological Measurement, 62, 337-354.
47. Potthast, M. J. (1993). Confirmatory factor analysis of ordered categorical variables with large models. British Journal of Mathematical and Statistical Psychology, 46, 273-286.
48. Reise, S. P. (1999). Personality measurement issues viewed through the eyes of IRT. In S. E. Embretson & S. L. Hershberger (Eds.), The new rules of measurement (pp. 219-241). Hillsdale, NJ: Erlbaum.
49. Reise, S. P., & Waller, N. G. (1990). Fitting the two-parameter model to personality data. Applied Psychological Measurement, 14, 45-58.
50. Samejima, F. (1977). A method of estimating item characteristic functions using the maximum likelihood estimate of ability. Psychometrika, 42, 163-191.
51. Schmidt, W. C. (1997). World-Wide Web survey research: Benefits, potential problems, and solutions. Behavior Research Methods, Instruments & Computers, 29, 274-279.
52. Smith, M. A., & Leigh, B. (1997). Virtual subjects: Using the Internet as an alternative source of subjects and research environment. Behavior Research Methods, Instruments & Computers, 29, 496-505.
53. Sórbom, D. (1979). Detection of correlated errors in longitudinal data. In J. Magidson (Ed.), Advances in factor analysis and structural equation models (pp. 171-184). Cambridge, MA: Abt Books.
54. Stanton, J. M. (1998). An empirical assessment of data collection using the Internet. Personnel Psychology, 51, 709-725.
55. Steinberg, L., & Thissen, D. (1995). Item response theory in personality research. In P. E. Shrout & S. T. Fiske (Eds.), Personality research, methods, and theory: A festschrift honoring Donald W. Fiske (pp. 161-181). Hillsdale, NJ: Erlbaum.
56. Steinberg, L., Thissen, D., & Wainer, H. (2000). Validity. In H. Wainer (Ed.), Computerized adaptive testing: A primer (pp. 185-229). Mahwah, NJ: Erlbaum.
57. Sykes, R. C., & Ito, K. (1997). The effects of computer administration on scores and item-parameter estimates of an IRT-based licensure examination. Applied Psychological Measurement, 21, 51-63.
58. Takane, Y., & de Leeuw, J. (1987). On the relationship between item response theory and factor analysis of discretized variables. Psychometrika, 52, 393-408.
59. Thissen, D. (2000). Reliability and measurement precision. In H. Wainer, (Ed.), Computerized adaptive testing: A primer (pp. 159-184). Mahwah, NJ: Erlbaum.
60. Vispoel, W. P., Boo, J., & Bleiler, T. (2001). Computerized and paper-and-pencil versions of the Rosenberg self-esteem scale: A comparison of psychometric features and respondent preferences. Educational and Psychological Measurement, 61, 461-474.
61. Waller, N. G., Tellegen, A., McDonald, R. P., & Lykken, D. T. (1996). Exploring nonlinear models in personality assessment: Development and validation of a negative emotionality scale. Journal of Personality, 64, 545-576.
62. Webster, J., & Compeau, D. (1996). Computer-assisted versus paper-and-pencil administration of questionnaires. Behavior Research Methods, Instruments & Computers, 28, 567-576.