Reliability in content analysis: Some common misconceptions and recommendations

Human Communication Research

Volumn 30, Issue 3, 2004, Pages 411-433

  Krippendorff, Klaus ^a,b  

^a UNIVERSITY OF PENNSYLVANIA   (United States)

^b UNIVERSITY OF PENNSYLVANIA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 4043124032     PISSN: 03603989     EISSN: None     Source Type: Journal    
DOI: 10.1093/hcr/30.3.411     Document Type: Article

References (29)

1. Benini, R. (1901). Principii di Demographia: Manuali Barbera di Scienze Giuridiche Socialie Politiche (No. 29). Firenze, Italy: G. Barbera.
2. Bennett, E. M., Alpert, R., & Goldstein, A. C. (1954). Communications through limited response questioning. Public Opinion Quarterly, 18, 303-308.
3. Brennan, R. L., & Prediger, D. J. (1981). Coefficient kappa: Some uses, misuses, and alternatives. Educational and Psychological Measurement, 41, 687-699.
4. Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20, 37-46.
5. Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrica, 16, 297-334.
6. Craig, R. T. (1981). Generalization of Scott's index of intercoder agreement. Public Opinion Quarterly, 45, 260-264.
7. Fleiss, J. L. (1975). Measuring agreement between two judges on the presence and absence of a trait. Biometrics, 31, 651-659.
8. Fleiss, J. L. (1978). Reply to Klaus Krippendorff's "Reliability of binary attribute data." Biometrics, 34, 144.
9. Fleiss, J. L. (1981). Statistical methods for rates and proportions. New York: Wiley & Sons.
10. Goodman, L. A, & Kruskal, W. H. (1954). Measures of association for cross classifications. Journal of the American Statistical Association, 49, 732-764.
11. Holley, W., & Guilford, J. P. (1964). A note on the G-index of agreement. Educational and Psychological Measurement, 24, 749-754.
12. Holsti, O. R. (1969). Content analysis for the social sciences and humanities. Reading, MA: Addison-Wesley.
13. Hughes, M. A., & Garrett, D. E. (1990). Intercoder reliability estimation - Approaches in marketing: A generalizability theory framework for quantitative data. Journal of Marketing Research, 27, 185-195.
14. Janson, S., & Vegelius, J. (1979). On generalizations of the G index and the phi coefficient to nominal scales. Multivariate Behavioral Research, 14, 255-269.
15. Krippendorff, K. (1970). Bivariate agreement coefficients for reliability data. In E. R. Borgatta & G. W. Bohrnstedt (Eds.), Sociological methodology 1970 (pp. 139-150). San Francisco: Jossey Bass.
16. Krippendorff, K. (1978). Reliability of binary attribute data. Biometrics, 34, 142-144.
17. Krippendorff, K. (2004). Content analysis: An introduction to its methodology (2nd ed.). Thousand Oaks, CA: Sage.
18. Lombard, M., Snyder-Duch, J., & Bracken, C. C. (2002). Content analysis in mass communication research: An assessment and reporting of intercoder reliability. Human Communication Research, 28, 587-604.
19. Lombard, M., Snyder-Duch, J., & Bracken, C. C. (2003). Correction. Human Communication Research, 29, 469-472.
20. Maxwell, A. E. (1970). Comparing the classification of subjects by two independent judges. British Journal of Psychiatry, 116, 651-655.
21. Neuendorf, K. A. (2002). The content analysis guidebook. Thousand Oaks, CA: Sage.
22. Osgood, C. E. (1959). The representational model and relevant research. In I. de Sola Pool (Ed.), Trends in content analysis (pp. 33-88). Urbana: University of Illinois Press.
23. Pasadeos, Y., Huhman, B., Standley, T., & Wilson, G. (1995, May). Applications of content analysis in news research: A critical examination. Paper presented to the annual convention of the Association for Education in Journalism and Mass Communication, Washington, DC. Cited in M. Lombard, J. Snyder-Duch, & C. C. Bracken (2002). Content analysis in mass communication research: An assessment and reporting of intercoder reliability. Human Communication Research, 28, 587-604.
24. Pasadeos, Y., Huhman, B., Standley, T., & Wilson, G. (1995, May). Applications of content analysis in news research: A critical examination. Paper presented to the annual convention of the Association for Education in Journalism and Mass Communication, Washington, DC. Cited in M. Lombard, J. Snyder-Duch, & C. C. Bracken (2002). Content analysis in mass communication research: An assessment and reporting of intercoder reliability. Human Communication Research, 28, 587-604.
25. Perreault, W. D., & Leigh, L. E. (1989). Reliability of nominal data based on qualitative judgments. Journal of Marketing Research, 26, 135-148.
26. Potter, W. J., & Levine-Donnerstein, D. (1999). Rethinking validity and reliability in content analysis. Journal of Applied Communication Research, 27, 258-284.
27. Riffe, D., & Freitag, A. (1996, August). Twenty-five years of content analyses in Journalism & Mass Communication Quarterly. Paper presented to the annual convention of the Association for Education in Journalism and Mass Communication, Anaheim, CA. Cited in D. Riffe, S. Lacy, & F. G. Fico (1998). Analyzing media messages. Mahwah, NJ: Erlbaum.
28. Scott, W. A. (1955). Reliability of content analysis: The case of nominal scale coding. Public Opinion Quarterly, 19, 321-325.
29. Zwick, R. (1988). Another look at interrater agreement. Psychological Bulletin, 103, 347-387.