Improving Survey Response Rates in Online Panels: Effects of Low-Cost Incentives and Cost-Free Text Appeal Interventions

Social Science Computer Review

Volumn 34, Issue 2, 2016, Pages 229-243

  Pedersen, Mogens Jin ^a,b   Nielsen, Christian Videbæk ^c  

^a AARHUS UNIVERSITY   (Denmark)

^b VIVE THE DANISH CENTER FOR SOCIAL SCIENCE RESEARCH   (Denmark)

^c LONDON SCHOOL OF ECONOMICS   (United Kingdom)

Author keywords

incentives; online panel; survey experiment; survey response; text appeal

Indexed keywords

EID: 84975770291     PISSN: 08944393     EISSN: 15528286     Source Type: Journal    
DOI: 10.1177/0894439314563916     Document Type: Article

References (69)

1. J.D.AngristJ.Pischke (2009). Mostly harmless econometrics: An empiricist’s companion. Princeton, NJ: Princeton University Press.
2. D.P.Bachman (1987). Cover letter appeals and sponsorship effects on mail survey response rates. Journal of Marketing Education, 9, 45–51.
3. T.J.BeebeM.E.DavernD.D.McAlpineK.T.CallT.H.Rockwood (2005). Increasing response rates in a survey of medicaid enrollees: The effect of a prepaid monetary incentive and mixed modes (mail and telephone). Medical Care, 43, 411–414.
4. M.BlackwellS.IacusG.KingG.Porro (2009). CEM: Coarsened exact matching in stata. Stata Journal9, 524–546.
5. M.BrennanP.SeymourP.Gendall (1993). The effectiveness of monetary incentives in mail surveys: Further data. Marketing Bulletin, 4, 43–52.
6. E.BrüggeR.WeissJ.KrosnicJ.van den. Braked (2012). Establishing the accuracy of online panel research (Paper presented at the 2012 MESS workshop). Amsterdam, the Netherlands.
7. S.T.CavusgilL.A.Elvey-Kirk (1998). Mail survey response behavior: A conceptualization of motivating factors and an empirical study. European Journal of Marketing, 32, 1165–1192.
8. D.J.ChampionA.M.Sear (1969). Questionnaire response rate: A methodological analysis. Social Forces, 47, 335–339.
9. T.L.ChildersW.M.PrideO.C.Ferrell (1980). A reassessment of the effects of appeals on response to mail surveys. Journal of Marketing Research, 17, 365–370.
10. J.Cohen (1988). Statistical power analysis for the behavioral sciences. Hillsdale, NJ: Lawrence Erlbaum Associates.
11. M.P.Couper (2000). Review: Web surveys: A review of issues and approaches. Public Opinion Quarterly, 64, 464–494.
12. R.CurtinS.PresserE.Singer (2005). Changes in telephone survey nonresponse over the past quarter century. Public Opinion Quarterly, 69, 87–98.
13. C.S.CycyotaD.A.Harrison (2002). Enhancing survey response rates at the executive level: Are employee- or consumer-level techniques effective?Journal of Management, 28, 151–176.
14. E.L.DeciR.M.Ryan (1985). Intrinsic motivation and self-determination in human behavior. New York, NY: Plenum.
15. E.L.DeciR.M.Ryan (2004). Handbook of self-determination research. Rochester, NY: University of Rochester Press.
16. A.DeehanT.TempletonC.TaylorC.DrummondJ.Strang (1997). The effect of cash and other financial inducements on the response rate of general practitioners in a national postal study. British Journal of General Practice, 47, 87–90.
17. E.de LeeuwW.de Heer (2002). Trends in household survey nonresponse: A longitudinal and international comparison. In (Eds.), Survey nonresponse (pp. 41–54). New York, NY: John Wiley.
18. E.DeutskensK.D.RuyterM.WetzelsP.Oosterveld (2004). Response rate and response quality of internet-based surveys: An experimental study. Marketing Letters, 15, 21–36.
19. D.A.DillmanD.K.Bowker (2001). The web questionnaire Challenge to survey methodologists. In (Eds.), Dimensions of internet science (pp. 159–178). Lengerich, Germany: Pabst Science Publishers.
20. D.A.DillmanJ.D.SmythL.M.Christian (2009). Mail and internet surveys: The tailored design method (3rd ed.). New York, NY: John Wiley.
21. D.A.DillmanE.SingerJ.R.ClarkJ.B.Treat (1996). Effects of benefits appeals, mandatory appeals, and variations in statements of confidentiality on completion rates for census questionnaires. Public Opinion Quarterly, 60, 376–389.
22. P.J.EdwardsI.RobertsM.J.ClarkeC.DiGuiseppiS.PratapR.WentzI.Kwan (2002). Increasing response rates to postal questionnaires: Systematic review. British Medical Journal, 324, 1183–1195.
23. P.J.EdwardsI.RobertsM.J.ClarkeC.DiGuiseppiR.WenztI.Kwan…S.Pratap (2009). Methods to increase response to postal and electronic questionnaires. Cochrane Database of Systematic Reviews, 2009, Issue 3. Art. No.: MR000008.
24. W.FanZ.Yan (2010) Factors affecting response rates of the web survey: A systematic review. Computers in Human Behavior, 26, 132–139.
25. A.J.FariaJ.R.Dickinson (1992). Mail survey response, speed, and cost. Industrial Marketing Management, 21, 51–60.
26. B.S.FreyF.Jegen (2001). Motivation crowding theory. Journal of Economic Surveys, 15, 589–611.
27. D.H.FurseD.W.Stewart (1982). Monetary incentives versus promised contribution to charity: New evidence on mail survey response. Journal of Marketing Research, 19, 375–380.
28. M.GagnéE.L.Deci (2005). Self-determination theory and work motivation. Journal of Organizational Behavior, 26, 331–362.
29. M.GattellariJ.E.Ward (2001). Will donations to their learned college increase surgeons’ participation in surveys? A randomized trial. Journal of Clinical Epidemiology, 54, 645–649.
30. A.S.Göritz (2006a). Case lotteries and incentives in online panels. Social Science Computer Review, 24, 445–459.
31. A.S.Göritz (2006b). Incentives in Web studies: Methodological issues and a review. International Journal of Internet Science, 1, 58–70.
32. A.S.GöritzS.C.Luthe (2013a). How do lotteries and study results influence response behavior in online panels?Social Science Computer Review, 31, 371–385.
33. A.S.GöritzS.C.Luthe (2013b). Lotteries and study results in market research online panels. International Journal of Market Research, 55, 611–626.
34. A.S.GöritzS.C.Luthe (2013c). Effects of lotteries on response behavior in online panels. Field Methods, 25, 219–237.
35. R.M.GrovesF.J.FowlerJrM.P.CouperJ.M.LepkowskiE.SingerR.Tourangeau (2009). Survey methodology (2nd ed.). Hoboken, NJ: John Wiley & Sons Inc.
36. K.M.Hansen (2006). The effects of incentives, interview length, and interviewer characteristics on response rates in a CATI-study. International Journal of Public Opinion Research, 19, 112–121.
37. M.J.HoustonJ.R.Nevin (1977). The effect of source and appeal on mail survey response patterns. Journal of Marketing Research, 14, 374–378.
38. R.HubbardE.L.Little (1988). Promised contributions to charity and mail survey responses: Replication with extension. Public Opinion Quarterly, 52, 223–230.
39. S.IacusG.KingG.Porro (2012). Causal inference without balance checking: Coarsened exact matching. Political Analysis, 20, 1–24.
40. J.IlievaS.BaronN.M.Healey (2002). Online surveys in marketing research: Pros and cons. International Journal of Market Research, 44, 361–382.
41. J.JamesR.Bolstein (1990). The effect of monetary incentives and follow-up mailings on the response rate and response quality in mail surveys. Public Opinion Quarterly, 54, 346–361.
42. J.JamesR.Bolstein (1992). Large monetary incentives and their effect on mail survey response rates. Public Opinion Quarterly, 56, 442–453.
43. J.S.KalantarN.J.Talley (1999). The effects of lottery incentive and length of questionnaire on health survey response rates: A randomized study. Journal of Clinical Epidemiology, 52, 1117–1122.
44. F.Keusch (2013). The role of topic interest and topic salience in online panel web surveys. International Journal of Market Research, 55, 58–80.
45. J.R.KofoedD.M.Søndergaard (2013). Mobning gennemtænkt[Rethinking School Bullying]. Copenhagen, Denmark: Hans Reitzels Forlag.
46. M.E.KropfJ.Blair (2005). Eliciting survey cooperation: Incentives, self-interest, and norms of cooperation. Evaluation Review, 29, 559–575.
47. G.M.LeungL.M.HoM.F.ChanJ.M.JohnstonF.K.Wong (2002). The effects of cash and lottery incentives on mailed surveys to physicians: A randomized trial. Journal of Clinical Epidemiology, 55, 801–807.
48. A.S.Linsky (1965). A factorial experiment in inducing responses to a mail questionnaire. Sociology and Social Research, 49, 183–189.
49. S.J.LondonC.J.Dommeyer (1990). Increasing response to industrial mail surveys. Industrial Marketing Management, 19, 235–241.
50. K.Lozar ManfredaM.BosnjakJ.BerzelakI.HaasV.Vehovar (2008). Web surveys versus other survey modes: A meta-analysis comparing response rates. International Journal of Market Research, 50, 79–104.
51. L.D.MarrettN.KreigerL.DoddsS.Hilditch (1992). The effect on response rates of offering a small incentive with a mailed questionnaire. Annals of Epidemology, 2, 745–753.
52. J.G.Nicholls (1984). Achievement motivation: Conceptions of ability, subjective experience, task choice, and performance. Psychological Review, 91, 328–346.
53. J.PetchenikD.J.Watermolen (2011). A cautionary note on using the internet to survey recent hunter education graduates. Human Dimensions of Wildlife, 16, 216–218.
54. R.E.RobertsO.F.McGoryR.N.Forthofer (1978). Further evidence on using a deadline to stimulate responses to a mail survey. Public Opinion Quarterly, 42, 407–410.
55. D.S.RoseS.D.SidleK.H.Griffith (2007). A penny for your thoughts: Monetary incentives improve response rates for company-sponsored employee surveys. Organizational Research Methods, 10, 225–240.
56. R.M.Ryan (1982). Control and information in the intrapersonal sphere: An extension of cognitive evaluation theory. Journal of Personality and Social Psychology, 43, 450–461.
57. E.Singer (2006). Introduction: Nonresponse bias in household surveys. Public Opinion Quarterly, 70, 637–645.
58. E.SingerC.Ye (2013). The use and effects of incentives in surveys. Annals of the American Academy of Political and Social Sciences, 645, 112–141.
59. S.J.SkinnerO.C.FerrellW.M.Pride (1984). Personal and non-personal incentives in mail surveys: Immediate versus delayed inducements. Academy of Marketing Science, 12, 106–114.
60. M.F.TeislB.RoeM.Vayda (2005). Incentive effects on response rates, data quality, and survey administration costs. International Journal of Public Opinion Research, 18, 364–373.
61. P.C.ThistlethwaiteJ.Finlay (1993). The impact of selected mail response enhancement techniques on surveys of the mature market: Some new evidence. Journal of Professional Services Marketing, 8, 269–276.
62. R.TourangeauM.P.CouperD.M.Steiger (2003). Humanizing self-administered surveys: Experiments on social pressure in web and IVR surveys. Computers in Human Behavior, 19, 1–24.
63. R.TourangeauR.M.GrovesC.KennedyT.Yan (2009). The presentation of a web survey, nonresponse and measurement error among members of web panel. Journal of Official Statistics, 25, 299–321.
64. W.Vandenabeele (2007). Towards a theory of public service motivation: An institutional approach. Public Management Review, 9, 545–556.
65. K.WarrinerJ.GoyderH.GjertsenP.HohnerK.McSpurren (1996). Charities, no; lotteries, no; cash, yes: Main effects and interactions in a canadian incentives experiment. Public Opinion Quarterly, 60, 542–562.
66. E.WhiteB.K.ArmstrongR.Saracci (2008). Principles of exposure measurement in epidemiology. New York, NY: Oxford University Press.
67. M.K.WhitemanP.LangenbergK.KjerulffR.McCarterJ.A.Flaws (2003). A randomized trial of incentives to improve response rates to a mailed women’s health questionnaire. Journal of Women’s Health, 12, 821–828.
68. K.B.Wright (2005). Researching internet-based populations: Advantages and disadvantages of online survey research, online questionnaire authoring software packages, and web survey services. Journal of Computer-Mediated Communication, 10. doi:10.1111/j.1083-6101.2005.tb00259.x
69. F.J.YammarinoS.J.SkinnerT.L.Childers (1991). Understanding mail survey response behaviour: A meta-analysis. Public Opinion Quarterly, 55, 613–639.