Response styles in survey research: A literature review of antecedents, consequences, and remedies

International Journal of Public Opinion Research

Volumn 25, Issue 2, 2013, Pages 195-217

  Van Vaerenbergh, Yves ^a,b   Thomas, Troy D ^b,c  

^a HOGESCHOOL UNIVERSITEIT BRUSSEL   (Belgium)

^b GHENT UNIVERSITY   (Belgium)

^c UNIVERSITY OF GUYANA   (Guyana)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84878879852     PISSN: 09542892     EISSN: 14716909     Source Type: Journal    
DOI: 10.1093/ijpor/eds021     Document Type: Review

References (81)

1. Albaum, G., Roster, C., Yu, J. H., & Rogers, R. D. (2007). Simple rating scale formats: Exploring extreme response. International Journal of Market Research, 49, 633-650.
2. Arce-Ferrer, A. J. (2006). An investigation into the factors influencing extremeresponse style. Educational and Psychological Measurement, 66, 374-392. doi: 10.1177/0013164405278575.
3. Austin, E. J., Deary, I. J., & Egan, V. (2006). Individual differences in response scale use: Mixed Rasch modelling of responses to NEO-FFI items. Personality and Individual Differences, 40, 1235-1245. doi:10.1016/j.paid.2005.10.018.
4. Ayidiya, S., & McClendon, M. J. (1990). Response effects in mail surveys. Public Opinion Quarterly, 54, 229-247. doi: 10.1086/269200.
5. Bachman, J. G., & O'Malley, P. M. (1984). Yea-saying, nay-saying, and going to extremes: Black-white differences in response styles. Public Opinion Quarterly, 48, 491-509. doi: 10.1086/268845.
6. Baron-Epel, O., Kaplan, G., Weinstein, R., & Green, M. S. (2010). Extreme and acquiescence bias in a bi-ethnic population. European Journal of Public Health, 20, 543-548. doi: 10.1093/eurpub/ckq052.
7. Baumgartner, H., & Steenkamp, J. B. E. M. (2001). Response styles in marketing research: A cross-national investigation. Journal of Marketing Research, 38, 143-156. doi: 10.1509/jmkr.38.2.143.18840.
8. Bentler, P. M., Jackson, D. N., & Messick, S. (1971). Identification of content and style: A two-dimensional interpretation of acquiescence. Psychological Bulletin, 76, 186-204. doi: 10.1037/h0031474.
9. Billiet, J. B., & Davidov, E. (2008). Testing the stability of an acquiescence style factor behind two interrelated substantive variables in a panel design. Sociological Methods and Research, 36, 543-562. doi: 10.1177/0049124107313901.
10. Billiet, J. B., & McClendon, M. J. (2000). Modeling acquiescence in measurement models for two balanced sets of items. Structural Equation Modeling, 7, 608-628. doi: 10.1207/S15328007SEM0704-5.
11. Bolt, D. M., & Newton, J. R. (2011). Multiscale measurement of extreme response style. Educational and Psychological Measurement, 71, 814-833. doi: 10.1177/0013164410388411.
12. Brengelman, J. (1960). Extreme response set, drive level and abnormality in questionnaire rigidity. Journal of Mental Science, 106, 171-186. doi: 10.1192/bjp.106 .442.171.
13. Cabooter, E. F. K. (2010). The impact of situational and dispositional variables on response styles with respect to attitude measures. Doctoral dissertation, Ghent University, Ghent, Belgium.
14. Chen, C., Lee, S. Y., & Stevenson, H. W. (1995). Response style and cross-cultural comparisons of rating scales among East Asian and North American students. Psychological Science, 6, 170-175. doi: 10.1111/j.1467-9280.1995. tb00327.x.
15. Cheung, G. W., & Rensvold, R. B. (2000). Assessing extreme and acquiescence response sets in cross-cultural research using structural equations modeling. Journal of Cross-Cultural Psychology, 31, 187-212. doi: 10.1177/0022022100031002003.
16. Clarke, I. (2000). Extreme response style in cross-cultural research. Journal of Social Behavior and Personality, 15, 137-152. doi: 10.1177/002202217400500407.
17. Cleary, P. D., Mechanic, D., & Weiss, N. (1981). The effect of interviewer characteristics on responses to a mental health interview. Journal of Health and Social Behavior, 22, 183-193. doi: 10.2307/2136294.
18. Couch, A., & Keniston, K. (1960). Yea-sayers and nay-sayers: Agreeing response set as a personality variable. Journal of Abnormal and Social Psychology, 60, 151-172. doi: 10.1037/h0040372.
19. Crandall, J. (1973). Sex differences in extreme response style: Differences in frequency of use of extreme positive and negative ratings. Journal of Social Psychology, 89, 281-293. doi: 10.1080/00224545.1973.9922601.
20. De Beuckelaer, A., Weijters, B., & Rutten, A. (2010). Using ad hoc measures for response styles: A cautionary note. Quality & Quantity, 44, 761-775. doi: 10.1007/s11135-009-9225-z.
21. De Jong, M. G., Steenkamp, J. B., Fox, J. P., & Baumgartner, H. (2008). Using item response theory to measure extreme response style in marketing research: A global investigation. Journal of Marketing Research, 45, 104-115. doi: 10.1509/jmkr.45.1.104.
22. Eid, M., & Rauber, M. (2000). Detecting measurement invariance in organizational surveys. European Journal of Psychological Assessment, 16, 20-30. doi: 10.1027//1015-5759.16.1.20.
23. Gibbons, J. L., Zelner, J. A., & Reduk, D. J. (1999). Effects of language and meaningfulness on the use of extreme response styles by Spanish-English bilinguals. Cross-Cultural Research, 33, 369-381. doi: 10.1177/106939719903300404.
24. Greenleaf, E. A. (1992a). Improving rating scale measures by detecting and correcting bias components in some response styles. Journal of Marketing Research, 29, 176-188. doi: 10.2307/3172568.
25. Greenleaf, E. A. (1992b). Measuring extreme response style. Public Opinion Quarterly, 56, 328-350. doi: 10.1086/269326.
26. Grimm, D. D., & Church, A. T. (1999). A cross-cultural study of response biases in personality measures. Journal of Research in Personality, 33, 415-441. doi: 10.1006/jrpe.1999.2256.
27. Hamilton, D. (1968). Personality attributes associated with extreme response style. Psychological Bulletin, 69, 192-203. doi: 10.1037/h0025606.
28. Harzing, A.-W. (2006). Response styles in cross-national survey research: A 26-country study. International Journal of Cross-Cultural Management, 6, 243-266. doi: 10.1177/1470595806066332.
29. Heerwegh, D. (2009). Mode differences between face-to-face and web surveys: An experimental investigation of data quality and social desirability effects. International Journal of Public Opinion Research, 21, 111-121. doi: 10.1093/ijpor/edn054.
30. Hofstede, G. H. (2001). Culture's consequences: Comparing values, behaviors, institutions and organizations across nations. (2nd edn). Hillsdale, NJ: Lawrence Erlbaum Associates.
31. Hofstede, G., & McCrae, R. R. (2004). Personality and culture revisited: Linking traits and dimensions of culture. Cross-Cultural Research, 38, 52-88. doi: 10.1177/1069397103259443.
32. House, R. J., Hanges, P. J., Javidan, M., Dorfman, P. W., & Gupta, V. (2004). Culture, leadership, and organizations. The Globe study of 62 Societies. Thousand Oaks, CA: Sage.
33. Hox, J. J., De Leeuw, E., & Kreft, I. G. (1991). The effect of interviewer and respondent characteristics on the quality of survey data: A multilevel model. In P. P. Biemer, R. M. Groves, L. E. Lyberg, N. A. Mathiowetz & S. Sudman (Eds.), Measurement errors in surveys (pp. 439-461). New York: Wiley. doi: 10.1002/9781118150382.ch22.
34. Hurley, J. R. (1998). Timidity as a response style to psychological questionnaires. Journal of Psychology, 132, 201-210. doi: 10.1080/ 00223989809599159.
35. Johnson, T., Kulesa, P., Cho, Y. I., & Shavitt, S. (2005). The relation between culture and response styles: Evidence from 19 countries. Journal of Cross-Cultural Psychology, 36, 264-277. doi: 10.1177/ 0022022104272905.
36. Jordan, L. A., Marcus, A. S., & Reeder, L. G. (1980). Response styles in telephone and household interviewing: A field experiment. Public Opinion Quarterly, 44, 210-222. doi: 10.1086/268585.
37. Kankaras-, M., & Moors, G. (2011). Measurement equivalence and extreme response bias in the comparison of attitudes across Europe: A multigroup latent-class factor approach. Methodology: European Journal of Research Methods for the Behavioral and Social Sciences, 7, 68-80. doi: 10.1027/1614-2241/ a000024.
38. Kieruj, N. D., & Moors, G. (2010). Variations in response style behavior by response scale format in attitude research. International Journal of Public Opinion Research, 22, 320-342. doi: 10.1093/ijpor/edq001.
39. Kieruj, N. D., & Moors, G. (2012). Response style behavior: Question format dependent or personal Style? Quality & Quantity, Advance online publication, doi: 10.1007/s11135-011-9511-4.
40. Knowles, E. S., & Condon, C. A. (1999). Why people say ''yes'': A dual-process theory of acquiescence. Journal of Personality and Social Psychology, 77, 379-386. doi: 10.1037/0022-3514.77.2.379.
41. Krosnick, J. A. (1999). Survey research. Annual Review of Psychology, 50, 537-567. doi: 10.1146/annurev.psych.50.1.537.
42. Lenzner, T., Kaczmirek, L., & Galesic, M. (2011). Seeing through the eyes of the respondent: An eye-tracking study on survey question comprehension. International Journal of Public Opinion Research, 23, 361-373. doi: 10.1093/ijpor/edq053.
43. Light, C. S., Zax, M., & Gardiner, D. H. (1965). Relationship of age, sex, and intelligence level to extreme response style. Journal of Personality and Social Psychology, 2, 907-909. doi: 10.1037/h0022746.
44. Lipps, O., & Pollien, A. (2011). Effects of interviewer experience on components of nonresponse in the European Social Survey. Field Methods, 23, 156-172. doi: 10.1177/1525822X10387770.
45. Marin, G., Gamba, R. J., & Marin, B. V. (1992). Extreme response style and acquiescence among Hispanics. Journal of Cross-Cultural Psychology, 23, 498-509. doi: 10.1177/0022022192234006.
46. Matarazzo, J. D., & Herman, D. O. (1984). Relationship of education and IQ in the WAIS-R standardization sample. Journal of Consulting and Clinical Psychology, 52, 631-634. doi: 10.1037/0022-006X.52.4.631.
47. Maxey, L. M., & Sanford, A. J. (1992). Context effects and the communicative functions of quantifiers: Implications for their use in attitude research. In N. Schwarz & S. Sudman (Eds.), Context effects in social psychology research (pp. 279-296). New York: Springer. doi: 2027.42/64020.
48. Meisenberg, G., & Williams, A. (2008). Are acquiescent and extreme response styles related to low intelligence and education? Personality and Individual Differences, 44, 1539-1550. doi: 10.1016/j.paid.2008.01.010.
49. Moors, G. (2003). Diagnosing response style behavior by means of a latent-class factor approach. Socio-demographic correlates of gender role attitudes and perceptions of ethnic discrimination reexamined. Quality & Quantity, 37, 277-302. doi: 10.1023/A:1024472110002.
50. Moors, G. (2008). Exploring the effect of a middle response category on response style in attitude measurement. Quality & Quantity, 42, 779-794. doi: 10.1007/s11135-006-9067-x.
51. Moors, G. (2010). Ranking the ratings: A latent-class regression model to control for overall agreement in opinion research. International Journal of Public Opinion Research, 22, 93-119. doi: 10.1093/ijpor/edp036.
52. Moors, G. (2012). The effect of response style bias on the measurement of transformational, transaction, and laissez-faire leadership. European Journal of Work and Organizational Psychology, 21, 271-298. doi: 10.1080/1359432X.2010. 550680.
53. Naemi, B. D., Beal, D., & Payne, S. C. (2009). Personality predictors of extreme response style. Journal of Personality, 77, 261-282. doi: 10.1111/j.1467-6494.2008 .00545.x.
54. Olson, K., & Bilgen, I. (2011). The role of interviewer experience on acquiescence. Public Opinion Quarterly, 75, 99-114. doi: 10.1093/poq/nfq067.
55. Oppenheimer, D., Meyvis, T., & Davidenko, N. (2009). Instructional manipulation checks: Detecting satisficing to increase statistical power. Journal of Experimental Social Psychology, 45, 867-872. doi: 10.1016/j.jesp.2009.03.009.
56. Paulhus, D. L. (1991). Measurement and control of response bias. In J. P. Robinson, P. R. Shaver & L. S. Wrightman (Eds.), Measurement of personality and social psychological attitudes (pp. 17-59). San Diego: Academic Press.
57. Preston, C. C., & Colman, A. M. (2000). Optimal number of response categories in rating scales: Reliability, validity, discriminating power and respondent preferences. Acta Psychologica, 104, 1-15. doi: 10.1016/S0001- 6918(99)00050-5.
58. Reynolds, N., & Smith, A. (2010). Assessing the impact of response styles on cross-cultural service quality evaluation: A simplified approach to eliminating the problem. Journal of Service Research, 13, 230-243. doi: 10.1177/1094670509360408.
59. Rorer, L. G. (1965). The great response-style myth. Psychological Bulletin, 63, 129-156. doi: 10.1037/h0021888.
60. Ross, C., & Mirowsky, J. (1984). Socially-desirable response and acquiescence in a cross-cultural survey of mental health. Journal of Health and Social Behavior, 25, 189-197. doi: 10.2307/2136668.
61. Rowland, M. L., Naidoo, S., Abdulkadir, R., Moraru, R., Huang, B., & Pau, A. (2010). Perceptions of intimidation and bullying in dental schools: A multi-national study. International Dental Journal, 60, 106-112. doi: 10.1922/IDJ-2327Rowland07.
62. Saris, W. E., & Aalberts, C. (2003). Different explanations for correlated disturbance terms in MTMM studies. Structural Equation Modeling, 10, 193-213. doi: 10.1207/S15328007SEM1002-2.
63. Saris, W. E., Satorra, A., & Coenders, G. (2004). A new approach to evaluating the quality of measurement instruments. Sociological Methodology, 34, 311-347. doi: 10.1111/j.0081-1750.2004.00155.x.
64. Smith, T. W. (2011). Refining the total survey error perspective. International Journal of Public Opinion Research, 23, 464-484. doi: 10.1093/ijpor/edq052.
65. Stenning, B. W., & Everett, J. E. (1984). Response styles in a cross-cultural managerial study. Journal of Social Psychology, 122, 151-156. doi: 10.1080/00224545.1984.9713475.
66. Stukovsky, R., Palat, M., & Sedlakova, A. (1982). Scoring position styles in the elderly. Studia Psychologica, 24, 145-154.
67. Tourangeau, R., Couper, M., & Conrad, F. (2007). Color, labels, and interpretive heuristics for response scales. Public Opinion Quarterly, 71, 91-112. doi: 10.1093/poq/nfl046.
68. Usunier, J.-C. (2011). Language as a resource to assess cross-cultural equivalence in quantitative management research. Journal of World Business, 46, 314-319. doi: 10.1016/j.jwb.2010.07.002.
69. Van Herk, H., Poortinga, Y. H., & Verhallen, T. M. M. (2004). Response styles in rating scales: Evidence of method bias in data from six EU countries. Journal of Cross-Cultural Psychology, 35, 346-360. doi: 10.1177/0022022104264126.
70. Van Rosmalen, J., Van Herk, H., & Groenen, P. J. F. (2010). Identifying response styles: A latent-class bilinear multinomial logit model. Journal of Marketing Research, 47, 157-172. doi: 10.1509/jmkr.47.1.157.
71. Watkins, D., & Cheung, S. (1995). Culture, gender, and response bias: Analysis of responses to the self-description questionnaire. Journal of Cross-Cultural Psychology, 26, 490-504. doi: 10.1177/0022022195265003.
72. Weijters, B. (2006). Response styles in consumer research Doctoral dissertation, Ghent University, Ghent, Belgium.
73. Weijters, B., Cabooter, E., & Schillewaert, N. (2010a). The effect of rating scale format on response styles: The number of response categories and response category labels. International Journal of Research in Marketing, 27, 236-247. doi: 10.1016/j.ijresmar.2010.02.004,.
74. Weijters, B., Geuens, M., & Schillewaert, N. (2008). Assessing response styles across modes of data collection. Journal of the Academy of Marketing Science, 36, 409-422. doi: 10.1007/s11747-007-0077-6.
75. Weijters, B., Geuens, M., & Schillewaert, N. (2009). The proximity effect: The role of inter-item distance on reverse-item bias. International Journal of Research in Marketing, 26, 2-12. doi: 10.1016/j.ijresmar.2008.09.003.
76. Weijters, B., Geuens, M., & Schillewaert, N. (2010b). The individual consistency of acquiescence and extreme response style in self-report questionnaires. Applied Psychological Measurement, 34, 105-121. doi: 10.1177/0146621609338593.
77. Weijters, B., Geuens, M., & Schillewaert, N. (2010c). The stability of individual response styles. Psychological Methods, 15, 96-110. doi: 10.1037/a0018721.
78. Welkenhuysen-Gybels, J., Billiet, J., & Cambre, B. (2003). Adjustment for acquiescence in the assessment of the construct equivalence of Likert-type score items. Journal of Cross-Cultural Psychology, 34, 702-722. doi: 10.1177/0022022103257070.
79. Weng, L. (2004). Impact of the number of response categories and anchor labels on coefficient alpha and test-retest reliability. Educational and Psychological Measurement, 64, 956-972. doi: 10.1177/0013164404268674.
80. Wong, N., Rindfleisch, A., & Burroughs, J. E. (2003). Do reverse-worded items confound measures in cross-cultural consumer research? The case of the material values scale. Journal of Consumer Research, 30, 72-91. doi: 10.1086/374697.
81. Zuckerman, M., & North, J. (1961). Response set and content factors in the California F-scale and the parental attitude research instrument. Journal of Social Psychology, 53, 199-210. doi: 10.1080/00224545.1961.9922118.