Validated intraclass correlation statistics to test item performance models

Behavior Research Methods

Volumn 43, Issue 1, 2011, Pages 37-55

  Courrieu, Pierre ^a   Brand D'abrescia, Muriele ^b   Peereman, Ronald ^c   Spieler, Daniel ^d   Rey, Arnaud ^a  

^a CNRS   (France)

^b UNIVERSITÉ DE BOURGOGNE   (France)

^c UNIVERSITÉ PIERRE MENDÈS FRANCE   (France)

^d GEORGIA INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

Intraclass correlation; Item performance databases; Misfit detection; Model test

Indexed keywords

ADULT; ALGORITHM; ANALYSIS OF VARIANCE; ARTICLE; BEHAVIORAL SCIENCE; COMPUTER SIMULATION; EPIDEMIOLOGY; FEMALE; HUMAN; MALE; NEUROPSYCHOLOGICAL TEST; PHYSIOLOGY; POPULATION; PSYCHOLOGY; PSYCHOMOTOR PERFORMANCE; REACTION TIME; REGRESSION ANALYSIS; REPRODUCIBILITY; STANDARD; STATISTICAL ANALYSIS; STATISTICAL MODEL; STATISTICS;

ALGORITHMS; ANALYSIS OF VARIANCE; BEHAVIORAL SCIENCES; COGNITIVE SCIENCE; COMPUTER SIMULATION; DATA INTERPRETATION, STATISTICAL; FEMALE; HUMANS; MALE; MODELS, STATISTICAL; NEUROPSYCHOLOGICAL TESTS; POPULATION; PSYCHOMOTOR PERFORMANCE; REACTION TIME; REGRESSION ANALYSIS; REPRODUCIBILITY OF RESULTS; SAMPLING STUDIES; YOUNG ADULT;

EID: 79953681884     PISSN: 1554351X     EISSN: 15543528     Source Type: Journal    
DOI: 10.3758/s13428-010-0020-5     Document Type: Article

References (29)

1. Ahmed, M., & Shoukri, M. (2010). A Bayesian estimator of the intracluster correlation coefficient from correlated binary responses. Journal of Data Science, 8, 127-137.
2. Akaike, H. (1974). A new look at the statistical model identification.I. E. E. E. Transactions on Automatic Control, AC, 19, 716-723.
3. Baayen, R. H., Piepenbrock, R., & van Rijn, H. (1993). The CELEX Lexical Database (CD-ROM). Linguistic Data Consortium. Philadelphia, PA: University of Pennsylvania.
4. Balota, D. A., & Spieler, D. H. (1998). The utility of item-level analyses in model evaluation: A reply to Seidenberg and Plaut (1998). Psychological Science, 9, 238-240.
5. Balota, D. A., Yap, M. J., Cortese, M. J., Hutchison, K. A., Kessler, B., Loftis, B., et al. (2007). The English lexicon project. Behavior Research Methods, 39(3), 445-459.
6. Coltheart, M., Rastle, K., Perry, C., Langdon, R., & Ziegler, J. C. (2001). DRC: A dual route cascaded model of visual word recognition and reading aloud. Psychological Review, 108, 204-256.
7. Content, A., Mousty, P., & Radeau, M. (1990). Brulex: Une base de données lexicales informatisée pour le français écrit et parlé. L'Année Psychologique, 90, 551-566.
8. Ferrand, L., New, B., Brysbaert, M., Keuleers, E., Bonin, P., Méot, A., et al. (2010). The French Lexicon Project: Lexical decision data for 38, 840 French words and 38, 840 pseudowords. Behavior Research Methods, 42, 488-496.
9. Hannan, E. J., & Quinn, B. G. (1979). The determination of the order of an autoregression. Journal of the Royal Statistical Society, B, 41, 190-195.
10. Hansen, M. H., & Yu, B. (2001). Model selection and the principle of minimum description length. Journal of the American Statistical Association, 96, 746-774.
11. Kass, R. E., & Raftery, A. E. (1995). Bayes factors. Journal of the American Statistical Association, 90, 377-395.
12. McGraw, K. O., & Wong, S. P. (1996). Forming inferences about some intralass correlation coefficients. Psychological Methods, 1(1), 30-46.
13. Myung, I. J., Pitt, M. A., & Kim, W. (2005). Model evaluation, testing and selection. In K. Lamberts & R. Goldstone (Eds.), Handbook of cognition (pp. 422-436). Thousand Oaks, CA: Sage.
14. Opdyke, J. D. (2003). Fast permutation tests that maximize power under conventional Monte Carlo sampling for pairwise and multiple comparisons. Journal of Modern Applied Statistical Methods, 2, 27-49.
15. Perry, C., Ziegler, J. C., & Zorzi, M. (2007). Nested incremental modeling in the development of computational theories: The CDP+ model of reading aloud. Psychological Review, 114, 273-315.
16. Perry, C., Ziegler, J. C., & Zorzi, M. (2010). Beyond single syllables: Large-scale modeling of reading aloud with the Connectionist Dual Process (CDP++) model. Cognitive Psychology. doi: 10. 1016/j. cogpsych. 2010. 04. 001.
17. Pitt, M. A., & Myung, I. J. (2002). When a good fit can be bad. Trends in Cognitive Sciences, 6(10), 421-425.
18. Plaut, D. C., McClelland, J. L., Seidenberg, M. S., & Patterson, K. (1996). Understanding normal and impaired word reading: Computational principles in quasi-regular domains. Psychological Review, 103, 56-115.
19. Rey, A., Courrieu, P., Schmidt-Weigand, F., & Jacobs, A. M. (2009). Item performance in visual word recognition. Psychonomic Bulletin & Review, 16(3), 600-608.
20. Rey, A., Jacobs, A. M., Schmidt-Weigand, F., & Ziegler, J. C. (1998). A phoneme effect in visual word recognition. Cognition, 68, 41-50.
21. Rey, A., & Schiller, N. O. (2005). Graphemic complexity and multiple print-to-sound associations in visual word recognition. Memory & Cognition, 33(1), 76-85.
22. Rissanen, J. (1996). Fisher information and stochastic complexity. IEEE Transactions on Information Theory, 42, 40-47.
23. Schwarz, G. (1978). Estimating the dimension of a model. Annals of Statistics, 6, 461-464.
24. Seidenberg, M. S., & McClelland, J. L. (1989). A distributed, developmental model of word recognition and naming. Psychological Review, 96, 523-568.
25. Seidenberg, M., & Plaut, D. C. (1998). Evaluating word reading models at the item level: Matching the grain of theory and data. Psychological Science, 9, 234-237.
26. Shrout, P. E., & Fleiss, J. L. (1979). Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin, 86(2), 420-428.
27. Spieler, D. H., & Balota, D. A. (1997). Bringing computational models of word naming down to the item level. Psychological Science, 8, 411-416.
28. Webb, N. M., Shavelson, R. J., & Haertel, E. H. (2006). Reliability coefficients and generalizability theory. Handbook of Statistics, 26, 81-124.
29. Yap, M. J., & Balota, D. A. (2009). Visual word recognition of multisyllabic words. Journal of Memory and Language, 60, 502-529.