Reliability for content analysis

Media Messages and Public Health: A Decisions Approach to Content Analysis

Volumn , Issue , 2008, Pages 67-87

  Neuendorf, Kimberly A ^a  

^a CLEVELAND STATE UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 77955101629     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.4324/9780203887349     Document Type: Chapter

References (79)

1. Aiello, A. E., & Larson, E. L. (2001). An analysis of 6 decades of hygiene-related advertising:1940-2000. American Journal of Infection Control 29, 383-388.
2. Allen, A. M. (1995). The new nutrition facts label in the print media: A content analysis. Journal of the American Dietetic Association 95, 348-352.
3. Bakeman, R. (2000). Behavioral observation and coding. In H. T. Reis & C. M. Judge (Eds.), Handbook of research methods in social and personality psychology (pp. 138-159). New York: Cambridge University Press.
4. Bakeman, R. (2007). Computer programs written by Roger Bakeman, GSU. Retrieved on July 14, 2008, from http://www2.gsu.edu/~psyrab/BakemanPrograms.htm
5. Bakeman, R., Quera, V., McArthur, D., & Robinson, B. F. (1997). Detecting sequential patterns and determining their reliability with fallible observers. Psychological Methods 2, 357-370.
6. Banerjee, M., Capozzoli, M., McSweeney, L., & Sinha, D. (1999). Beyond kappa: A review of interrater agreement measures. The Canadian Journal of Statistics 27(1), 3-23.
7. Bartko, J. J., & Carpenter, W. T. Jr. (1976). On the methods and theory of reliability. Journal of Nervous and Mental Disease 163, 307-317.
8. Batchelor, S. A., Kitzinger, J., & Burtney, E. (2004). Representing young people's sexuality in the "youth" media. Health Education Research 19, 669-676.
9. Brennan, R. L., & Prediger, D. J. (1981). Coefficient kappa: Some uses, misuses, and alternatives. Educational and Psychological Measurement 41, 687-699.
10. Carmines, E. G., & Zeller, R. A. (1979). Reliability and validity assessment. Beverly Hills, CA: Sage.
11. Chory-Assad, R. M., & Tamborini, R. (2001). Television doctors: An analysis of physicians in fictional and non-fictional television programs. Journal of Broadcasting & Electronic Media 45, 499-521.
12. Clapp, J. D., Whitney, M., & Shillington, A. M. (2002). The reliability of environmental measures of the college alcohol environment. Journal of Drug Education 32(4), 287-301.
13. Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement 20(1), 37-46.
14. Cohen, J. (1968). Weighted kappa: Nominal scale agreement with provision for scaled disagreement of partial credit. Psychological Bulletin 70(4), 213-220.
15. Commers, M. J., Visser, G., & De Leeuw, E. (2000). Representations of preconditions for and determinants of health in the Dutch press. Health Promotion International 15, 321-332.
16. Cooil, B., & Rust, R. T. (1994). Reliability and expected loss-a unifying principle. Psychometrika 59(2), 203-216.
17. Dominick, J. R. (1999). Who do you think you are? Personal home pages and selfpresentation on the World Wide Web. Journalism & Mass Communication Quarterly 76, 646-658.
18. Fan, X. T., & Chen, M. (2000). Published studies of interrater reliability often overestimate reliability: Computing the correct coefficient. Educational and Psychological Measurement 60, 532-542.
19. Fleiss, J. L. (1971). Measuring nominal scale agreement among many raters. Psychological Bulletin 76, 378-382.
20. Fouts, G., & Burggraf, K. (2000). Television situation comedies: Female weight, male negative comments, and audience reactions. Sex Roles 42, 925-932.
21. Frey, L. R., Botan, C. H., & Kreps, G. L. (2000). Investigating communication: An introduction to research methods (2nd ed.). Boston, MA: Allyn and Bacon.
22. Goetz, C. G., & Stebbins, G. T. (2004). Assuring interrater reliability for the UPDRS motor section: Utility of the UPDRS teaching tape. Movement Disorders 19, 1453-1456.
23. Gustafson, D. H., Van Koningsveld, R., & Peterson, R. W. (1984). Assessment of level of care: Implications of interrater reliability on health policy. Washington, DC: Health Care Financing Administration.
24. Haggard, E. A. (1958). Intraclass correlation and the analysis of variance. New York:The Dryden Press, Inc.
25. Haninger, K., & Thompson, K. M. (2004). Content and ratings of teen-rated video games. JAMA: Journal of the American Medical Association 291, 856-865.
26. Harwell, M. (1999). Evaluating the validity of educational rating data. Educational and Psychological Measurement 59, 25-37.
27. Hayes, A. F. (2007). SPSS and SAS macro for computing Krippendorff's alpha. Retrieved on July 14, 2008, from http://www.comm.ohio-state.edu/ahayes/SPSS%20programs/kalpha.htm
28. Hayes, A. F., & Krippendorff, K. (2007). Answering the call for a standard reliability measure for coding data. Communication Methods and Measures 1(1), 77-89.
29. Holsti, O. R. (1969). Content analysis for the social sciences and humanities. Reading, MA: Addison-Wesley.
30. Hubbell, A. P., & Dearing, J. W. (2003). Local newspapers, community partnerships, and health improvement projects: Their roles in a comprehensive community initiative. Journal of Community Health 28, 363-376.
31. Interrater reliability. (2001). Journal of Consumer Psychology, 10(1&2), 71-73.
32. Kang, N., Kara, A., Laskey, H. A., & Seaton, F. B. (1993). A SAS MACRO for calculating intercoder agreement in content analysis. Journal of Advertising 23(2), 17-28.
33. Kraemer, H. C. (1980). Extension of the kappa coefficient. Biometrics 36, 207-216.
34. Krippendorff, K. (1980). Content analysis: An introduction to its methodology. Beverly Hills, CA: Sage.
35. Krippendorff, K. (2004a). Content analysis: An introduction to its methodology (2nd ed.). Thousand Oaks, CA: Sage.
36. Krippendorff, K. (2004b). Reliability in content analysis: Some common misconceptions and recommendations. Human Communication Research 30, 411-433.
37. Kunkel, D., Cope, K. M., & Biely, E. (1999). Sexual messages on television: Comparing findings from three studies. The Journal of Sex Research 36(3), 230-236.
38. Lacy, S., & Riffe, D. (1996). Sampling error and selecting intercoder reliability samples for nominal content categories: Sins of omission and commission in mass communication quantitative research. Journalism & Mass Communication Quarterly 73, 969-973.
39. Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics 33, 159-174.
40. Lievens, F., Sanchez, J. I., & De Corte, W. (2004). Easing the inferential leaps in competency modeling: The effects of task-related information and subject matter expertise. Personnel Psychology 57, 881-904.
41. Lin, L. I. (1989). A concordance correlation coefficient to evaluate reproducibility. Biometrics 45, 255-268.
42. Lombard, M., Snyder-Duch, J., & Bracken, C. C. (2002). Content analysis in mass communication: Assessment and reporting of intercoder reliability. Human Communication Research 28, 587-604.
43. Lombard, M., Snyder-Duch, J., & Bracken, C. C. (2003). Correction. Human Communication Research 29, 469-472.
44. Lombard, M., Snyder-Duch, J., & Bracken, C. C. (2005, June 13). Practical resources for assessing and reporting intercoder reliability in content analysis research projects. Retrieved on July 14, 2008, from http://www.temple.edu/mmc/reliability
45. Mercado-Martinez, F. J., Robles-Silva, L., Moreno-Leal, N., & Franco-Almazan, C. (2001). Inconsistent journalism: The coverage of chronic diseases in the Mexican press. Journal of Health Communication 6, 235-247.
46. Michelson, J. (1996). Visual imagery in medical journal advertising. M.A. thesis, Cleveland State University, Cleveland, OH.
47. Myhre, S. L., Saphir, M. N., Flora, J. A., Howard, K. A., & Gonzalez, E. M. (2002). Alcohol coverage in California newspapers: Frequency, prominence, and framing. Journal of Public Health Policy 23(2), 172-190.
48. National Center for Education Statistics. (1982). Assessment of intercoder reliability on the classification of secondary school courses. Arlington, VA: Evaluation Technologies, Inc.
49. National Center for the Analysis of Violent Crime. (1990). Criminal investigative analysis:Sexual homicide. Washington, DC: U.S. Department of Justice.
50. National television violence study (Vol. 1). (1997). Thousand Oaks, CA: Sage.
51. Neuendorf, K. A. (2002). The content analysis guidebook. Thousand Oaks, CA: Sage.
52. Neuendorf, K. A. (2006, November 2). Considerations and recommendations for the Annenberg Health Media Coding Project. Retrieved on July 14, 2008, from http://www.youthmediarisk.org
53. Neuendorf, K. A., & Skalski, P. D. (in press). Quantitative content analysis and the measurement of collective identity. In R. Abdelal, Y. M. Herrera, A. I. Johnston, & R. McDermott (Eds.), Identity as a variable. Cambridge, MA: Harvard Identity Project.
54. Nunnally, J. C. (1978). Psychometric theory (2nd ed.). New York: McGraw-Hill.
55. Pasadeos, Y., Huhman, B., Standley, T., & Wilson, G. (1995, May). Applications of content analysis in news research: A critical examination. Paper presented to the Theory and Methodology Division of the Association for Education in Journalism and Mass Communication, Washington, DC.
56. Pedhazur, E., & Schmelkin, L. (1991). Measurement, design, and analysis: An integrated approach. Hillsdale, NJ: Lawrence Erlbaum Associates.
57. Perreault, W. D., & Leigh, L. E. (1989). Reliability of nominal data based on qualitative judgments. Journal of Marketing Research, 26(May), 135-148.
58. Pollock, J. C., & Yulis, S. G. (2004). Nationwide newspaper coverage of physicianassisted suicide: A community structure approach. Journal of Health Communication 9, 281-307.
59. Popping, R. (1984). AGREE, a package for computing nominal scale agreement. Computational Statistics and Data Analysis 2, 182-185.
60. Popping, R. (1988). On agreement indices for nominal data. In W. E. Saris & I. N. Gallhofer (Eds.), Sociometric research (Vol. 1): Data collection and scaling (pp. 90-105). New York: St. Martin's Press.
61. Potter, J., Linz, D., Wilson, B. J., Kunkel, D., Donnerstein, E., Smith, S. L., Blumenthal, E., & Gray, T. (1998). Content analysis of entertainment television: New methodological developments. In J. T. Hamilton (Ed.), Television violence and public policy (pp. 55-103). Ann Arbor: The University of Michigan Press.
62. Potter, W. J., & Levine-Donnerstein, D. (1999). Rethinking validity and reliability in content analysis. Journal of Applied Communication Research 27(3), 258-284.
63. Riffe, D., Lacy, S., & Fico, F. G. (1998). Analyzing media messages: Using quantitative content analysis in research. Mahwah, NJ: Lawrence Erlbaum.
64. Robinson, B. F., & Bakeman, R. (1998). ComKappa: A Windows 95 program for calculating kappa and related statistics. Behavior Research Methods, Instruments, and Computers 30, 731-732.
65. Rosenthal, R. (1987). Judgment studies: Design, analysis, and meta-analysis. New York:Cambridge University Press.
66. Rust, R., & Cooil, B. (1994). Reliability measures for qualitative data: Theory and implications. Journal of Marketing Research, 31(February), 1-14.
67. Salgado, J. F., & Moscoso, S. (1996). Meta-analysis of interrater reliability of job performance ratings in validity studies of personnel selection. Perceptual and Motor Skills 83, 1195-1201.
68. Scott, W. (1955). Reliability of content analysis: The case of nominal scale coding. Public Opinion Quarterly 17, 321-325.
69. Skymeg Software. (2007). Program for Reliability Assessment with Multiple Coders (PRAM). Retrieved on July 14, 2008, from http://www.geocities.com/skymegsoftware/pram.html
70. Sprafkin, J. N., Rubinstein, E. A., & Stone, A. (1977). A content analysis of four television diets. Stony Brook, NY: Brookdale International Institute.
71. Staley, M. R., & Weissmuller, J. J. (1981). Interrater reliability: The development of an automated analysis tool. Brooks Air Force Base, TX: Air Force Human Resources Laboratory.
72. Suen, H. K., & Lee, P. S. C. (1985). Effects of the use of percentage agreement on behavioral observation reliabilities: A reassessment. Journal of Psychopathology and Behavioral Assessment 7, 221-234.
73. Thompson, K. M., & Haninger, K. (2001). Violence in E-rated video games. JAMA:Journal of the American Medical Association 286, 591-598.
74. Tinsley, H. E. A., & Weiss, D. J. (1975). Interrater reliability and agreement of subjective judgements. Journal of Counseling Psychology 22, 358-376.
75. Tinsley, H. E. A., & Weiss, D. J. (2000). Interrater reliability and agreement. In H. E. A. Tinsley & S. D. Brown (Eds.), Handbook of applied multivariate statistics and mathematical modeling (pp. 95-124). San Diego, CA: Academic Press.
76. Traub, R. E. (1994). Reliability for the social sciences: Theory and applications. Thousand Oaks, CA: Sage.
77. Wagner, E. R., & Hansen, E. N. (2002). Methodology for evaluating green advertising of forest products in the United States: A content analysis. Forest Products Journal 52(4), 17-23.
78. Wilson, B. J., Smith, S. L., Potter, W. J., Kunkel, D., Linz, D., Colvin, C. M., & Donnerstein, E. (2002). Violence in children's television programming: Assessing the risks. Journal of Communication 52(1), 5-35.
79. Yeaton, W. H., & Wortman, P. M. (1993). On the reliability of meta-analysis reviews-the role of intercoder agreement. Evaluation Research 17(3), 292-309.