Understanding data quality through reliability: A comparison of data reliability assessment in three international relations datasets

International Studies Review

Volumn 9, Issue 3, 2007, Pages 437-456

  Rothman, Steven B ^a  

^a UNIVERSITY OF OREGON   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

DATA PROCESSING; DATA QUALITY; DATA SET; INTERNATIONAL RELATIONS; RELIABILITY ANALYSIS;

EID: 36549001719     PISSN: 15219488     EISSN: 14682486     Source Type: Journal    
DOI: 10.1111/j.1468-2486.2007.00698.x     Document Type: Article

References (52)

1. Achen, Christopher H. (1983) Toward Theories of Data: The State of Political Methodology. In Political Science: State of the Discipline, edited by Ada W. Finifter. Washington, DC: American Political Science Association.
2. Adcock, Robert, and David Collier. (2001) Measurement Validity: A Shared Standard for Qualitative and Quantitative Research. American Political Science Review 95(3):529-546.
3. Baugh, Frank. (2003) Correcting Effect Sizes for Score Reliability. In Score Reliability: Contemporary Thinking on Reliability Issues, edited by Bruce Thompson. Thousand Oaks, CA: Sage Publications.
4. Braumoeller, Bear F., and Gary Goertz. (2000) The Methodology of Necessary Conditions. American Journal of Political Science 44(4):844-858.
5. Brennan, Robert L. (2001) An Essay on the History and Future of Reliability from the Perspective of Replications. Journal of Education Measurement 38(4):295-317.
6. Campbell, Donald T., and Donald W. Fiske. (1959) Convergent and Discriminant Validation by the Multitrait-Multimethod Matrix. Psychological Bulletin 56(2):81-105.
7. Carmines, Edward G., and Richard A. Zeller. (1979) Reliability and Validity Assessment. Beverly Hills, CA: Sage Publications.
8. Carney, Thomas F. (1972) Content Analysis: A Technique for Systematic Inference from Communications. London: Batsford.
9. Cohen, Jacob. (1960) A Coefficient of Agreement for Nominal Scales. Educational and Psychological Measurement 20(1):37-46.
10. Conca, Ken, Fengshi Wu, and Ciqi Mei. (2006) Global Regime Formation or Complex Institution Building? The Principled Content of International River Agreements International Studies Quarterly 50: 263-285.
11. Correlates of War Project. (2005) State System Membership List (v 2004.1). Available at http://www.correlatesofwar.org/.
12. Cronbach, Lee J. (1951) Coefficient Alpha and the Internal Structure of Tests. Psychometrica 16:297-334.
13. Cronbach, Lee J., and Richard J. Shavelson. (2004) My Current Thoughts on Coefficient Alpha and Successor Procedures. Educational and Psychological Measurement 64(3):391-418.
14. Diehl, Paul F. (2006) Correlates of War. Available at http://www.correlatesofwar.org/.
15. Fan, Xitao, and Bruce Thompson. (2003) Confidence Intervals about Score Reliability Coefficients. In Score Reliability: Contemporary Thinking on Reliability Issues, edited by Bruce Thompson. Thousand Oaks, CA: Sage Publications.
16. Foster, Sharon L., and John D. Cone. (1986) Design and Use of Direct Observation. In Handbook of Behavioral Assessment, edited by A. R. Ciminero, K. S. Calhoun and H. E. Adams. New York: Wiley.
17. Freedom House. (2006) Freedom in the World. Washington, DC: Freedom House.
18. Goertz, Gary. (2005) Social Science Concepts: A User's Guide. Princeton: Princeton University Press.
19. Hensel, Paul R. (1998) Reliability and Validity Issues in the Issue Correlates of War (ICOW) Project. Paper presented at the annual meeting of the International Studies Association, Minneapolis.
20. Hensel, Paul R. (2006) General Codebook: Issue Correlates of War (ICOW) Project. Available at http://www.garnet.acns.fsu.edu/~phensel/icow.html.
21. Hensel, Paul R., and Sara McLaughlin Mitchell. (2006) The Issue Correlates of War (ICOW) Project. Available at http://garnet.acns.fsu.edu/~phensel/icow.html.
22. Holsti, Ole R. (1969) Content Analysis for the Social Sciences and Humanities. Reading, MA: Addison-Wesley.
23. Hovy, Eduard. (2006) Toward a "Science"of Annotation. Coding across the Disciplines. A Project Based Workshop on Manual Text Annotation Techniques, Pittsburgh.
24. Hruschka, Daniel J., Deborah Schwartz, Daphne Cobb St. John, Erin Picone-De Caro, Richard A. Jenkins, and James W. Carey. (2004) Reliability in Coding Open-Minded Data: Lessons Learned from HIV Behavioral Research. Field Methods 16(3):307-331.
25. Jaggers, Keith, and Ted Robert Gurr. (1995) Tracking Democracy's Third Wave with the Polity III Data. Journal of Peace Research 32(4):469-482.
26. John, Oliver P., and Victoria Benet-Martinez. (2000) Measurement: Reliability, Construct Validation, and Scale Construction. In Handbook of Research Methods in Social and Personality Psychology, edited by Harry T. Reis and Charles M. Judd. New York: Cambridge University Press.
27. Jones, Lyle V. (1971) The Nature of Measurement. In Educational Measurement, edited by Robert L. Thorndike. Washington, DC: American Council on Education.
28. King, Gary. (1995) Replication, Replication. PS: Political Science and Politics 28(3):443-449.
29. King, Gary, Robert O. Keohane, and Sidney Verba. (1994) Designing Social Inquiry: Scientific Inference in Qualitative Research. Princeton: Princeton University Press.
30. Kwon, Namhee, Stuart Shulman, and Eduard Hovy. (2006) Multidimensional Text Analysis for eRulemaking. Paper presented at the 7th National Conference on Digital Government Research, San Diego.
31. Kyburg, Henry E. (1984) Theory and Measurement. Cambridge: Cambridge University Press.
32. Lacy, Stephen, and Daniel Riffe. (1996) Sampling Error and Selecting Intercoder Reliability Samples for Nominal Content Categories. Journalism and Mass Communication Quarterly 73(4):963-973.
33. Leeds, Brett Ashley. (2005) Alliance Treaty Obligations and Provisions (ATOP) Codebook. Available at http://www.atop.rice.edu/.
34. Lombard, Mathew, and Jennifer Snyder-Duch. (2002) Content Analysis in Mass Communication: Assessment and Reporting of Intercoder Reliability. Human Communication Research 28(4):587-604.
35. Mitchell, Ronald B. (2003) International Environmental Agreements: A Survey of Their Features, Formation, and Effects. Annual Review of Environmental Resources 28: 429-461.
36. Mitchell, Ronald B., and Steven B. Rothman (2006) Creating Large-N Datasets from Qualitative Information: Lessons from International Environmental Agreements. Paper presented at the annual meeting of the American Political Science Association, Philadelphia.
37. Neuendorf, Kimberly A. (2002) The Content Analysis Guidebook. Thousand Oaks, CA: Sage Publications.
38. Nunnally, Jum C., Jr. (1959) Tests and Measurements: Assessment and Prediction. New York: McGraw-Hill.
39. Oppenheim, Abraham N. (1992) Questionnaire Design, Interviewing, and Attitude Measurement. London: Pinter.
40. Pedhazur, Elazar J., and Liora P. Schmelkin. (1991) Measurement, Design, and Analysis: An Integrated Approach. Hillsdale, NJ: Lawrence Erlbaum.
41. Ryan, Gery W. (1999) Measuring the Typicality of Text: Using Multiple Coders for More than Just Reliability and Validity Checks. Human Organization 58(3):313-322.
42. Sawilowsky, Shlomo S. (2003) Reliability as Psychometrics Versus Datametrics. In Score Reliability: Contemporary Thinking on Reliability Issues, edited by Bruce Thompson. Thousand Oaks, CA: Sage Publications.
43. Shrout, Patrick E. (1998) Measurement Reliability and Agreement in Psychiatry. Statistical Methods in Medical Research 7:301-317.
44. Slavin, Robert E. (1984) Research Methods in Education: A Practical Guide. Englewood Cliffs, NJ: Prentice-Hall.
45. Smith, Charles P. (2000) Content Analysis and Narrative Analysis. New York: Cambridge University Press.
46. Smith, Eric R.A. N., Richard Herrera, and CherylL. Herrera. (1990) The Measurement Characteristics of Congressional Roll-Call Indexes. Legislative Studies Quarterly 15(2):283-295.
47. Stanley, Julian C. (1971) Reliability. In Educational Measurement, edited by Robert L. Thorndike. Washington, DC: American Council on Education.
48. Sullivan, John L., and Stanley Feldman. (1979) Multiple Indicators: An Introduction. Beverly Hills: Sage Publications.
49. Tinsley, Howard E. A., and Steven D. Brown. (2000) Handbook of Applied Multivariate Statistics and Mathematical Modeling. San Diego: Academic Press.
50. Vasquez, John A. (1987) The Steps to War: Toward a Scientific Explanation of Correlates of War Findings. World Politics 40(1):108-145.
51. Wiebe, Janice, Theresa Wilson, and Claire Cardie. (2005) Annotating Expressions of Opinions and Emotions in Language. Computers and the Humanities 39(2-3):165-210.
52. Yang, Hui, and Jamie Callan. (2005) Near-Duplicate Detection for eRulemaking. National Conference on Digital Government Research, Atlanta, ACM International Conference Proceeding Series.