Cognitive Diagnostic Assessment via Bayesian Evaluation of Informative: Diagnostic hypotheses

Psychological Methods

Volumn 19, Issue 1, 2014, Pages 21-38

  Hoijtink, Herbert ^a,b   Béland, Sébastien ^c   Vermeulen, Jorine A ^a,d  

^a Cito group   (Netherlands)

^b UTRECHT UNIVERSITY   (Netherlands)

^c UNIVERSITÉ DE SHERBROOKE   (Canada)

^d Department of Research Methodology Measurement Methods and Data Analysis ^*  (Netherlands)

Author keywords

Bayes factor; Cognitive diagnostic assessment; Inequality constrained hypothesis; Informative hypothesis; Psychometrics

Indexed keywords

BAYES THEOREM; COGNITION; COGNITION DISORDERS; HUMAN; PSYCHOMETRY;

BAYES THEOREM; COGNITION; COGNITION DISORDERS; HUMANS; PSYCHOMETRICS;

EID: 84897958058     PISSN: 1082989X     EISSN: None     Source Type: Journal    
DOI: 10.1037/a0034176     Document Type: Article

References (73)

1. American Psychiatric Association. (2000). Diagnostic and statistical manual of mental disorders (4th ed., text rev.). Washington, DC: Author.
2. Birnbaum, A. (1968). Some latent trait models and their use in inferring an examinee's ability. In F. M. Lord & M. R. Novick (Eds.), Statistical theories of mental test scores (pp. 395-479). Reading, MA: Addison-Wesley.
3. Boom, J., Hoijtink, H., & Kunnen, S. (2001). Rules in the balance. Classes, strategies or rules for the balance scale task. Cognitive Development, 16, 717-735.
4. Borsboom, D. (2008). Psychometric perspectives on diagnostic systems. Journal of Clinical Psychology, 64, 1089-1108.
5. Borsboom, D., Epskamp, S., Kievit, R. A., Cramer, A. O. J., & Schmittmann, V. D. (2011). Transdiagnostic networks: Commentary on Nolen-Hoeksema and Watkin (2011). Perspectives on Psychological Science, 6, 610-614.
6. Box, G. E. P. (1976). Science and statistics. Journal of the American Statistical Association, 71, 791-802.
7. Carmines, E. G., & Zeller, R. A. (1979). Reliability and validity. London, England: Sage.
8. Chiu, C., Douglas, J., & Li, X. (2009). Cluster analysis to cognitive diagnosis: Theory and applications. Psychometrika, 74, 633-665.
9. Cramer, A. O. J., Waldorp, O. J., van der Maas, H. L., & Borsboom, D./ (2010). Comorbidity: A network perspective. Behavioral and Brain Sciences, 33, 137-150.
10. Dayton, M., & Macready, G. B. (1988). Concomitant-variable latent-class models. Journal of the American Statistical Association, 83, 173-178.
11. De Boeck, P., & Wilson, M. (Eds.). (2004). Explanatory item response models: A generalized linear and nonlinear approach. New York, NY: Springer.
12. DeCarlo, L. T. (2012). Recognizing uncertainty in the Q-matrix via a Bayesian extension of the DINA model. Applied Psychological Measurement, 36, 447-468.
13. De La Torre, J. (2011). The generalized DINA model framework. Psychometrika, 76, 179-199.
14. DiBello, L. V., Roussos, L. A., & Stout, W. (2007). Review of cognitively diagnostic assessment and a summary of psychometric models. In C. R. Rao & S. Sinharay (Eds.), Handbook of statistics: Vol. 26. Psychometrics (pp. 979-1030). Amsterdam, the Netherlands: Elsevier.
15. Doignon, J. P., & Falmagne, J. C. I. (1999). Knowledge spaces. New York, NY: Springer.
16. Embretson, S. E., & Reise, S. P. (2000). Item response theory for psychologists. New York, NY: Erlbaum.
17. Fischer, G. H. (1973). The linear logistic test model as an instrument in educational research. Acta Psychologica, 37, 359-374.
18. Fischer, G. H., & Molenaar, I. W. (1995). Rasch models: Foundations, recent developments and applications. New York, NY: Springer.
19. Fox, J.-P. (2010). Bayesian item response theory modeling: Theory and applications. New York, NY: Springer.
20. Gelman, A., Carlin, J. B., Stern, H. S., & Rubin, D. B. (2004). Bayesian data analysis. Boca Raton, FL: Chapman & Hall/CRC.
21. Glas, C. A. W., & Vos, H. J. (2010). Adaptive mastery testing using a multidimensional IRT model. In W. J. van der Linden & C. A. W. Glas (Eds.), Elements of adaptive testing (pp. 409-432). New York, NY: Springer.
22. Guttman, L. (1944). A basis for scaling qualitative data. American Sociological Review, 9, 139-150.
23. Guttman, L. (1950). The basis for scalogram analysis. In S. A. Stouffer, L. Guttman, E. A. Suchman, P. F. Lazarsfeld, S. A. Star, & J. A. Claussen (Eds.), Measurement and prediction (pp. 60-90). Princeton, NJ: Princeton University Press.
24. Hambleton, R. K., & Swaminathan, H. (1985). Item response theory: Principles and applications. Boston, MA: Kluwer.
25. Hoijtink, H. (2001). Confirmatory latent class analysis: Model selection using Bayes factors and (pseudo) likelihood ratio statistics. Multi variate Behavioral Research, 36, 563-588.
26. Hoijtink, H. (2012). Informative hypotheses: Theory and practice for behavioral and social scientists. Boca Raton, FL: Chapman & Hall/CRC.
27. Hoijtink, H., & Boom, J. (2008). Inequality constrained latent class analysis. In H. Hoijtink, I. Klugkist, & P. A. Boelen (Eds.), Bayesian evaluation of informative hypotheses (pp. 227-246). New York, NY: Springer.
28. Jacob, B. A., & Levitt, S. D. (2003). Rotten apples: An investigation of the prevalence and predictors of teacher cheating. Quarterly Journal of Economics, 118, 843-877.
29. Jaeger, R. M. (1988). Use and effect of caution indices in detecting aberrant patterns of standard-setting judgments. Applied Measurement in Education, 1, 17-31.
30. Janssen, R., Tuerlinckx, F., Meulders, M., & De Boeck, P. (2000). A hierarchical IRT model for criterion-referenced measurement. Journal of Educational and Behavioral Statistics, 25, 285-306.
31. Jeffreys, H. (1961). Theory of probability (3rd ed.). Oxford, England: Oxford University Press.
32. Karabatsos, G. (2003). Comparing the aberrant response detection performance of thirty-six person-fit statistics. Applied Measurement in Education, 16, 277-298.
33. Kass, R. E., & Raftery, A. E. (1995). Bayes factors. Journal of the American Statistical Association, 90, 773-795.
34. Klauer, K. C. (1991). An exact and optimal standardized person test for assessing consistency with the Rasch model. Psychometrika, 56, 213- 228.
35. Klauer, K. C. (1995). The assessment of person fit. In G. H. Fischer & I.W. Molenaar (Eds.), Rasch models (pp. 97-110). New York, NY: Springer.
36. Klugkist, I., Laudy, O., & Hoijtink, H. (2005). Inequality constrained analysis of variance: A Bayesian approach. Psychological Methods, 10, 477-493.
37. Kripke, S. A. (1982). Wittgenstein on rules and language. Cambridge, MA: Harvard University Press.
38. Laudy, O., Boom, J., & Hoijtink, H. (2004). Bayesian computational methods for inequality constrained latent class analysis. In L. A. van der Ark, M. A. Croon, & K. Sijtsma (Eds.), New developments in categorical data analysis for the social and behavioral sciences (pp. 63-82). Mahwah, NJ: Erlbaum.
39. Lavine, M., & Schervish, M. J. (1999). Bayes factors: What they are and what they are not. American Statistician, 53, 119-122.
40. Leighton, J. P., & Gierl, M. J. (Eds.). (2007). Cognitive diagnostic assessment for education: Theory and applications. Cambridge, England: Cambridge University Press.
41. Liu, J., Xu, G., & Ying, Z. (2012). Data driven learning of Q-matrix. Applied Psychological Measurement, 36, 548 -564.
42. Lord, F. M. (1980). Applications of item response theory to practical testing problems. Hillsdale, NJ: Erlbaum.
43. Lynch, S. M. (2007). Introduction to applied Bayesian statistics and estimation for social scientists. New York, NY: Springer.
44. Macready, G. B., & Dayton, M. (1977). The use of probabilistic models in the assessment of mastery. Journal of Educational Statistics, 2, 99-120.
45. Macready, G. B., & Dayton, M. (1980). The nature and use of state mastery models. Applied Psychological Measurement, 4, 493-516.
46. Maris, E. (1999). Estimating multiple classification latent class models. Psychometrika, 64, 187-212.
47. McCutcheon, A. L. (1987). Latent class analysis. Thousand Oaks, CA: Sage.
48. Meehl, P. E., & Rosen, A. (1955). Antecedent probability and the efficiency of psychometric signs, patterns, or cutting scores. Psychological Bulletin, 52, 194-216.
49. Meijer, R. R., Egberink, I. J. L., Emons, W. H. M., & Sijtsma, K. (2008). Detection and validation of unscalable item score patterns using item response theory: An illustration with Harter's Self-Perception Profile for Children. Journal of Personality Assessment, 90, 227-238.
50. Meijer, R. R., & Sijtsma, K. (2001). Methodology review: Evaluating person fit. Applied Psychological Measurement, 25, 107-135.
51. Morey, R. D., & Rouder, J. N. (2011). Bayes factor approaches for testing interval null hypotheses. Psychological Methods, 16, 406-419.
52. Mulder, J., Hoijtink, H., & Klugkist, I. (2010). Equality and inequality constrained multivariate linear models: Objective model selection using constrained posterior priors. Journal of Statistical Planning and Inference, 140, 887-906.
53. Rijkes, C. P. M., & Kelderman, H. (2006). Latent-response Rasch models for strategy shifts in problem-solving processes. In M. von Davier & C. H. Cartstensen (Eds.), Multivariate and mixture distribution Rasch models: Extensions and applications (pp. 311-328). New York, NY: Springer.
54. Rouder, J. N., Speckman, P. L., Sun, D., Morey, R. D., & Iverson, G. (2009). Bayesian t tests for accepting and rejecting the null hypothesis. Psychonomic Bulletin & Review, 16, 225-237.
55. Rupp, A. A., Templin, J., & Henson, R. A. (2010). Diagnostic measurement: Theory, methods, and applications. New York, NY: Guilford Press.
56. Schmittmann, V. D., Cramer, A. O. J., Waldorp, L. J., Epskamp, S., Kievit, R. A., & Borsboom, D. (2013). Deconstructing the construct: A network perspective on psychological phenomena. New Ideas in Psychology, 31, 43-53.
57. Schönemann, P. H. (1997). Some new results on hit rates and base rates in mental testing. Chinese Journal of Psychology, 39, 173-192.
58. Schönemann, P. H. (2005). Psychometrics of intelligence. In K. Kempf-Leonard (Ed.-in-Chief), Encyclopedia of social measurement (Vol. 3, pp. 193-201). Amsterdam, the Netherlands: Elsevier.
59. Schönemann, P. H., & Thompson, W. W. (1996). Hit rate bias in mental testing. Cahiers de Psychologie Cognitive/Current Psychology of Cognition, 15, 3-28.
60. Schrepp, M. (2005). About the connection between knowledge structures and latent class models. Methodology: European Journal of Research Methods for the Behavioral and Social Sciences, 1, 93-103.
61. Siegler, R. S. (1981). Developmental sequences within and between concepts. Monographs of the Society for Research in Child Development, 46(2, Serial No. 189).
62. Sijtsma, K., & Molenaar, I. W. (2002). Introduction to nonparametric item response theory. Thousand Oaks, CA: Sage.
63. Tatsuoka, K. K. (1983). Rule space: An approach for dealing with misconceptions based on item response theory. Journal of Educational Measurement, 20, 345-354.
64. Tatsuoka, K. K. (2009). Cognitive assessment: An introduction to the rule space method. New York, NY: Routledge.
65. Tatsuoka, K. K., & Tatsuoka, M. M. (1997). Computerized cognitive diagnostic adaptive testing: Effects on remedial instruction as empirical validation. Journal of Educational Measurement, 34, 3-20.
66. Taylor, H. C., & Russell, J. T. (1939). The relationship of validity coefficients to the practical effectiveness of tests in selection: Discussion and tables. Journal of Applied Psychology, 23, 565-578.
67. Van de Schoot, R., Hoijtink, H., & Romeijn, J. W. (2011). Moving beyond traditional null hypothesis testing: Evaluating expectations directly. Frontiers in Psychology, 2, Article 24.
68. Vermunt, J. K., & Magidson, J. (2005). Technical guide for Latent GOLD 4.0: Basic and advanced. Belmont, MA: Statistical Innovations.
69. Von Davier, M. (2009). Some notes on the reinvention of latent structure models as diagnostic classification models. Measurement: Interdisciplinary Research and Perspective, 7, 67-74.
70. Von Davier, M. (2011). Equivalence of the DINA model and a constrained general diagnostic model (Research Report No. RR-11-37). Retrieved from the Educational Testing Service website: http://www.ets.org/Media/ Research/pdf/RR-11-37.pdf
71. Vos, H. J., & Glas, C. A. W. (2010). Testlet-based adaptive mastery testing. In W. J. van der Linden & C. A. W. Glas (Eds.), Elements of adaptive testing (pp. 389-408). New York, NY: Springer.
72. Wagenmakers, E.-J. (2007). A practical solution to the pervasive problems of p values. Psychonomic Bulletin & Review, 14, 779-804.
73. Wang, C., Chang, M., & Douglas, J. (2012). Combining CAT with cognitive diagnosis: A weighted item selection approach. Behavior Research Methods, 44, 95-109.