Secondary observer effects: idiosyncratic errors in small-N secondary data analysis

International Journal of Social Research Methodology

Volumn 19, Issue 3, 2016, Pages 301-318

  Breznau, Nate ^a  

^a UNIVERSITY OF BREMEN   (Germany)

Author keywords

macro comparative research; observer effects; replication crisis; secondary data analysis; sensitivity analysis; small N

Indexed keywords

EID: 84956598074     PISSN: 13645579     EISSN: 14645300     Source Type: Journal    
DOI: 10.1080/13645579.2014.1001221     Document Type: Article

References (47)

1. American Sociological Association. (1999). American Sociological Association code of ethics. Washington, DC: Author. Retrieved from http://www.asanet.org/images/asa/docs/pdf/CodeofEthics.pdf
2. Anonymous. (2006). Peer review and fraud. Nature, 444, 971–972. doi:10.1038/444971b
3. B.R.Barnes, (2009). The Hawthorne effect in community trials in developing countries. International Journal of Social Research Methodology, 13, 357–370. doi:10.1080/13645570903269096
4. N.Beck,, & J.N.Katz, (1995). What to do (and not to do) with time-series cross-section data. The American Political Science Review, 89, 634–647. doi:10.2307/2082979
5. E.Bell, (1999). The negotiation of a working role in organizational ethnography. International Journal of Social Research Methodology, 2, 17–37. doi:10.1080/136455799295168
6. K.A.Bollen, (1989). Structural equations with latent variables. New York, NY: Wiley.10.1002/9781118619179
7. K.A.Bollen,, B.Entwisle,, & A.S.Alderson, (1993). Macrocomparative research methods. Annual Review of Sociology, 19, 321–351. doi:10.1146/annurev.so.19.080193.001541
8. D.Bradley,, E.Huber,, S.Moller,, F.Nielsen,, & J.D.Stephens, (2003). Distribution and redistribution in postindustrial democracies. World Politics, 55, 193–228. doi:10.1353/wp.2003.0009 10.1353/wp.2003.0009
9. D.Brady,, J.Beckfield,, & M.Seeleib-Kaiser, (2005). Economic globalization and the welfare state in affluent democracies, 1975–2001. American Sociological Review, 70, 921–948. doi:10.1177/000312240507000603 10.1177/000312240507000603
10. N.Breznau,, V.A.Lykes,, M.D.R.Evans,, & J.Kelley, (2011). IESUD macro-data. Journal for the Scientific Study of Religion, 50. doi:10.1111/j.1468-5906.2011.01605.x/suppinfo
11. C.Brooks,, & J.Manza, (2006a). Social policy responsiveness in developed democracies. American Sociological Review, 71, 474–494. doi:10.1177/000312240607100306
12. C.Brooks,, & J.Manza, (2006b). Reply to Myles: Theory and methods for comparative opinion/social policy research. American Sociological Review, 71, 499–502. doi:10.2307/30039002
13. F.G.Castles, (2002). Developing new measures of welfare state change and reform. European Journal of Political Research, 41, 613–641 doi:10.1111/1475-6765.00024 10.1111/ejpr.2002.41.issue-5
14. F.G.Castles,, & H.Obinger, (2007). Social expenditure and the politics of redistribution. Journal of European Social Policy, 17, 206–222. doi:10.1177/0958928707078364 10.1177/0958928707078364
15. J.Freese, (2007). Replication standards for quantitative social science. Sociological Methods & Research, 36, 153–172. doi:10.1177/0049124107306659
16. D.Heerwegh,, T.Vanhove,, K.Matthijs,, & G.Loosveldt, (2005). The effect of personalization on response rates and data quality in web surveys. International Journal of Social Research Methodology, 8, 85–99. doi:10.1080/1364557042000203107
17. P.S.Herrnson, (1995). Replication, verification, secondary analysis, and data collection in political science. PS: Political Science and Politics, 28, 452–455. doi:10.2307/420302 10.1017/S1049096500057619
18. A.M.Hicks,, & L.Kenworthy, (2003). Varieties of welfare capitalism. Socio-Economic Review, 1, 27–61. doi:10.1093/soceco/1.1.27 10.1093/soceco/1.1.27
19. A.M.Hicks,, & D.H.Swank, (1992). Politics, institutions, and welfare spending in industrialized democracies, 1960–82. The American Political Science Review, 86, 658–674. doi:10.2307/1964129 10.2307/1964129
20. E.Huber,, C.Ragin,, & J.D.Stephens, (1993). Social democracy, christian democracy, constitutional structure, and the welfare state. The American Journal of Sociology, 99, 711–749. URL: http://www.jstor.org/stable/2781288 10.1086/ajs.1993.99.issue-3
21. T.Iversen,, & T.R.Cusack, (2000). The causes of welfare state expansion: Deindustrialization or globalization? World Politics, 52, 313–349. doi:10.1017/S0043887100016567 10.1017/S0043887100016567
22. T.Iversen,, & J.D.Stephens, (2008). Partisan politics, the welfare state and three worlds of human capital formation. Comparative Political Studies, 41, 600–637. doi:10.1177/0010414007313117 10.1177/0010414007313117
23. L.Kenworthy, (1999). Do social-welfare policies reduce poverty? A cross-national assessment. Social Forces, 77, 1119–1139. doi:10.1093/sf/77.3.1119
24. L.Kenworthy, (2007). Toward improved use of regression in macro-comparative analysis. Comparative Social Research, 24, 343–350.10.1016/S0195-6310(06)24010-9
25. L.Kenworthy, (2009). The effect of public opinion on social policy generosity. Socio-Economic Review, 7, 727–740. doi:10.1093/ser/mwp014 10.1093/ser/mwp014
26. B.Kittel, (1999). Sense and sensitivity in pooled analysis of political data. European Journal of Political Research, 35, 225–253. doi:10.1111/1475-6765.00448
27. B.Kittel, (2006). A crazy methodology? On the limits of macro-quantitative social science research. International Sociology, 21, 647–677. doi:10.1177/0268580906067835 10.1177/0268580906067835
28. B.Kittel,, & H.Winner, (2005). How reliable is pooled analysis in political economy? The globalization-welfare state nexus revisited. European Journal of Political Research, 44, 269–293. doi:10.1111/j.1475-6765.2005.00228.x
29. C.Levelt,, C.Noort,, & C.Drenth, (2012). Flawed science: The fraudulent research practices of social psychologist Diederik Stapel. Tilburg: University Joint (C.) Committee's Report. Tilburg University.
30. M.Liu,, & K.Stainback, (2013). Interviewer gender effects on survey responses to marriage-related questions. Public Opinion Quarterly, 77, 606–618. doi:10.1093/poq/nft019
31. G.J.McCall, (1984). Systematic field observation. Annual Review of Sociology, 10, 263–282. doi:10.2307/2083176
32. F.J.MehrtensIII, (2004). Three worlds of public opinion? Values, variation, and the effect on social policy. International Journal of Public Opinion Research, 16, 115–143. doi:10.1093/ijpor/16.2.115 10.1093/ijpor/16.2.115
33. J.Myles, (2006). Welfare states and public opinion: Comment on Brooks and Manza. American Sociological Review, 71, 495–498. doi:10.1177/000312240607100307 10.1177/000312240607100307
34. OECD. (2010). Gross domestic product (expenditure approach), per head, US $, constant prices, OECD base year. Exact download moment of April 19, 2010, 11:33am GMT GMT.
35. OECD. (2014). Gross domestic product (expenditure approach), per head, US $, constant prices, constat PPPs, OECD base year. Exact download moment of May 16, 2014, 10:41am GMT.
36. S.Presser,, & L.Stinson, (1998). Data collection mode and social desirability bias in self-reported religious attendance. American Sociological Review, 63, 137–145. doi:10.2307/2657486
37. F.J.Roethlisberger,, & W.J.Dickson, (2003). Management and the worker. (K.Thompson, Ed.) The early sociology of management and organizations (Reprint of.). New York, NY: Routledge.
38. N.C.Schaeffer, (1980). Evaluating race-of-interviewer effects in a national survey. Sociological Methods & Research, 8, 400–419. doi:10.1177/004912418000800403
39. M.Shalev, (2007). Limits and alternatives to multiple regression in comparative research. Comparative Social Research, 24, 267–314.
40. T.Skocpol,, & E.Amenta, (1986). States and social policies. Annual Review of Sociology, 12, 131–157. doi:10.1146/annurev.so.12.080186.001023 10.1146/annurev.so.12.080186.001023
41. D.J.Stanley,, & J.R.Spence, (2014). Expectations for replications: Are yours realistic? Perspectives on Psychological Science, 9, 305–318. doi:10.1177/1745691614528518
42. T.D.Sterling,, W.L.Rosenbaum,, & J.J.Weinkam, (1995). Publication decisions revisited: The effect of the outcome of statistical tests on the decision to publish and vice versa. The American Statistician, 49, 108–112. doi:10.1080/00031305.1995.10476125
43. R.Svensson,, S.Dahlberg,, S.Kumlin,, & B.Rothstein, (2012). The QoG social policy dataset (version 4A.). University of Gothenburg, The Quality of Government Institue.
44. D.H.Swank, (1998). Funding the welfare state: Globalization and the taxation of business in advanced market economies. Political Studies, 46, 671–692. doi:10.1111/1467-9248.00161
45. T.Van der Meer,, M.Te Grotenhuis,, & B.Pelzer, (2010). Influential cases in multilevel modeling: A methodological comment. American Sociological Review, 75, 173–178. doi:10.1177/0003122409359166
46. F.Van Kolfschooten, (2014). Fresh Misconduct charges hit dutch social psychology. Science, 344, 566–567. doi:10.1126/science.344.6184.566
47. C.Winship,, & L.Radbill, (1994). Sampling weights and regression analysis. Sociological Methods & Research, 23, 230–257. doi:10.1177/0049124194023002004