The subjective precision of computers: A methodological comparison with human coding in content analysis

Journalism and Mass Communication Quarterly

Volumn 83, Issue 1, 2006, Pages 186-200

  Conway, Mike ^a  

^a INDIANA UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTER; MASS MEDIA; RELIABILITY ANALYSIS;

EID: 33745309311     PISSN: 10776990     EISSN: None     Source Type: Journal    
DOI: 10.1177/107769900608300112     Document Type: Article

References (55)

1. Daniel Riffe and Alan Freitag, "A Content Analysis of Content Analyses: Twenty-Five Years of Journalism Quarterly," Journalism & Mass Communication Quarterly 74 (winter 1997): 873-82;
2. Craig W. Trumbo, "Research Methods in Mass Communication Research: A Census of Eight Journals 1990-2000," Journalism & Mass Communication Quarterly 81 (summer 2004): 417-36;
3. Doris A. Graber and James M. Smith, "Political Communication Faces the 21st Century," Journal of Communication 55 (September 2005): 479-507.
4. Kimberly A. Neuendorf, The Content Analysis Guidebook (Thousand Oaks, CA: Sage, 2002), 125-26.
5. Harold Lasswell, "Foreword," in The General Inquirer: A Computer Approach to Content Analysis, ed. Philip J. Stone, Dexter C. Dunphy, Marshall S. Smith, and Daniel M. Ogilvie (Cambridge, MA: The MIT Press, 1966), viii.
6. Wayne A. Danielson and James J. Mullen, "A Basic Space Unit for Newspaper Content Analysis," Journalism Quarterly 42 (winter 1965): 108.
7. Ole R. Holsti, "External Conflict and Internal Consensus: The Sino-Soviet Case," in The General Inquirer, ed. Stone et al., 343-58.
8. Marshall S. Smith, Philip J. Stone, and Evelyn N. Glenn, "A Content Analysis of Twenty Presidential Nomination Acceptance Speeches," in The General Inquirer, ed. Stone et al., 359-400.
9. Danielson and Mullen, "A Basic Space Unit for Newspaper Content Analysis."
10. Donald L. Diefenbach, "Historical Foundations of Computer-Assisted Content Analysis," in Theory, Method, and Practice in Computer Content Analysis, ed. Mark D. West (Westport, CT: Ablex Publishing, 2001), 13-41.
11. Roderick P. Hart, "Redeveloping Diction: Theoretical Considerations," in Theory, Method, and Practice, ed. West, 43-60.
12. Roderick P. Hart Campaign Talk: Why Elections Are Good for Us (Princeton NJ: Princeton University Press, 2000)
13. Wayne A. Danielson and Dominic L. Lasorsa, "Perceptions of Social Change: 100 Years of Front-Page Content in The New York Times and The Los Angeles Times," in Text Analysis for the Social Sciences, ed. C.R. Roberts (Mahwah, NJ: Lawrence Erlbaum, 1997)
14. Rebecca A. Lind and Colleen Salo, "The Framing of Feminists and Feminism in News and Public Affairs Programs in U.S. Electronic Media," Journal of Communication 52 (March 2002): 211-28.
15. M. Mark Miller, User's Guide for VBPro: A Program for Analyzing Verbatim Text (Knoxville, TN: Mark Miller, 1995);
16. M. Mark Miller, User's Guide for VBMap: A Program for Multidimensional Scaling of Concepts (Knoxville, TN: Mark Miller, 1995).
17. M. Mark Miller, Julie L. Andsager, and Bonnie P. Riechert, "Framing the Candidates in Presidential Primaries: Issues and Images in Press Releases and News Coverage," Journalism & Mass Communication Quarterly 75 (summer 1998): 312-24.
18. Julie L. Andsager and Angela Powers, "Social or Economic Concerns: How News and Women's Magazines Framed Breast Cancer in the 1990s," Journalism & Mass Communication Quarterly 76 (autumn 1999):531-50;
19. Angela Powers and Julie L. Andsager, "How Newspapers Framed Breast Implants in the 1990s," Journalism & Mass Communication Quarterly 76 (autumn 1999): 551-64.
20. Brigitte L. Nacos, Robert Y. Shapiro, John T. Young, David P. Fan, Torsten Kjellstrand, and Craig McCaa, "Content Analysis of News Reports: Comparing Human Coding and a Computer-Assisted Method," Communication 12 (1991): 111-28.
21. Nacos et al., "Content Analysis," 121.
22. Alf Linderman, "Computer Content Analysis and Manual Coding Techniques: A Comparative Analysis," in Theory, Method, and Practice, ed. West, 97-109.
23. Linderman, "Computer Content Analysis," in Theory, Method, and Practice, ed. West, 107.
24. Issues included any topics that were mentioned in media coverage of the campaign including education, business, the campaign itself, and health care. Attributes involved how the candidates were described in the coverage, including experience, wealth, Hispanic/minority, and leadership.
25. Maxwell McCombs, Esteban Lopez-Escobar, and Juan P. Llamas, "Setting the Agenda Attributes in the 1996 Spanish General Election," Journal of Communication 50 (spring 2000): 77-92.
26. Nacos et al., "Content Analysis," 115-16;
27. Linderman, "Computer Content Analysis," in Theory, Method, and Practice, ed. West, 99-103.
28. Andsager and Powers, "Social or Economic Concerns";
29. M. Mark Miller and Julie Andsager, "Protecting 1st Amendment?: Newspaper Coverage of Hate Speech," Newspaper Research Journal 18 (summer/fall 1997): 2-15;
30. Miller, Andsager, and Riechert, "Framing the Candidates";
31. M. Mark Miller and Bonnie P. Riechert, "The Spiral of Opportunity and Frame Resonance: Mapping the Issue Cycle in News and Public Discourse," in Framing Public Life: Perspectives on Media and Our Understanding of the Social World, ed. Stephen D. Reese, Oscar H. Gandy, Jr., and August E. Grant (Mahwah, NJ: Lawrence Erlbaum, 2001), 107-21;
32. Powers and Andsager, "How Newspapers Framed Breast Implants."
33. When n=7, the rank order correlation would have to be .714 or higher to be considered significant.
34. The word indexes for each attribute were constructed from the list of terms found in the sample. For example, the "campaign" index has 16 words including campaign, election, ballot, and candidate. The "education" index has 15 words including children, education, school, and teacher. The "Hispanic/minority" index has 20 words including minority, Mexican, Hispanic, border, and ethnic.
35. Nacos et al., "Content Analysis," 117-18.
36. Nacos et al., "Content Analysis," 125;
37. Linderman, "Computer Content Analysis," in Theory, Method, and Practice, ed. West, 107.
38. Kathleen A. Hansen, "Using Databases For Content Analysis," in Mass Communication Research and Theory, ed. Guido H. Stempel III, David H. Weaver, and G. Cleveland Wilhoit (Boston: Allyn & Bacon, 2003), 220-30;
39. Philip A. Kaufman, Carol Reese Dykers, and Carole Caldwell, "Why Going Online for Content Analysis Can Reduce Research Reliability," Journalism Quarterly 70 (winter 1993): 824-32;
40. Daniel Riffe, Stephen Lacy, and Frederick G. Fico, Analyzing Media Messages: Using Quantitative Content Analysis in Research, 2d ed. (Mahwah, NJ: Lawrence Erlbaum Associates, 2005), 203-07;
41. Robert L. Stevenson, "In Praise of Dumb Clerks: Computer-Assisted Content Analysis," in Theory, Method, and Practice, ed. West, 11-12.
42. Andsager and Powers, "Social or Economic Concerns."
43. Miller, Andsager, and Riechert, "Framing the Candidates."
44. Paula M. Poindexter and Maxwell E. McCombs, Research in Mass Communication: A Practical Guide (Boston: Bedford/St. Martin's, 2000), 190.
45. Richard W. Budd, Robert K. Thorp, and Lewis Donohew, Content Analysis of Communications (New York: The Macmillan Company, 1967), 66-71;
46. Klaus Krippendorf, Content Analysis: An Introduction To Its Methodology (Newbury Park: Sage Publications, 1980), 129-54;
47. Stephen R. Lacy and Daniel Riffe, "Sins of Omission and Commission in Mass Communication Quantitative Research," Journalism Quarterly 70 (spring 1993), 126-32;
48. Guido H. Stempel III, "Content Analysis," in Mass Communication Research and Theory, ed. Stempel, Weaver, and Wilhoit, 209-19.
49. Krippendorf, Content Analysis, 119-20;
50. Donald G. McTavish, "Pre-Assessment of Scale Reliability: A Computer Content Analysis Approach," in Theory, Method, and Practice, ed. West, 111-24;
51. Marshall S. Smith, "The Development of a Content Analysis Measuring Instrument," in The General Inquirer, ed. Stone et al, 207-32;
52. Mark D. West and Linda K. Fuller, "Toward a Typology and Theoretical Grounding for Computer Content Analysis," in Theory, Method, and Practice, ed. West, 77-94.
53. Smith, "The Development of a Content Analysis Measuring Instrument," in The General Inquirer, ed. Stone et al., 214.
54. Riffe, Lacy, and Fico, Analyzing Media Messages, 155.
55. Riffe, Lacy, and Fico, Analyzing Media Messages, 216-17.