Mode of data elicitati on, acquisition and response to surveys: A systematic review

Health Technology Assessment

Volumn 16, Issue 27, 2012, Pages 1-161

  Hood, K ^a   Robling, M ^a   Ingledew, D ^b   Gillespie, D ^a   Greene, G ^a   Ivins, R ^a   Russell, I ^c   Sayers, A ^d   Shaw, C ^e   Williams, J ^c  

^a CARDIFF UNIVERSITY   (United Kingdom)

^b BANGOR UNIVERSITY   (United Kingdom)

^c SWANSEA UNIVERSITY   (United Kingdom)

^d UNIVERSITY OF BRISTOL   (United Kingdom)

^e UNIVERSITY OF GLAMORGAN   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

ACCURACY; ARTICLE; DATA COLLECTION METHOD; DATA EXTRACTION; HEALTH SURVEY; HUMAN; INFORMATION PROCESSING; INFORMATION RETRIEVAL; INFORMATION TECHNOLOGY; INTERVIEW; MEDICAL RESEARCH; MINNESOTA MULTIPHASIC PERSONALITY INVENTORY; OUTCOME ASSESSMENT; PUBLICATION; RESEARCH SUBJECT; SENSORY STIMULATION; SHORT FORM 36; SYSTEMATIC ERROR; SYSTEMATIC REVIEW; THEORETICAL MODEL; CONFIDENCE INTERVAL; INSTRUMENTATION; METHODOLOGY; REPRODUCIBILITY; REVIEW; SELF REPORT;

CONFIDENCE INTERVALS; DATA COLLECTION; HEALTH SURVEYS; HUMANS; REPRODUCIBILITY OF RESULTS; RESEARCH DESIGN; SELF REPORT;

EID: 84862589387     PISSN: 13665278     EISSN: 20464924     Source Type: Journal    
DOI: 10.3310/hta16270     Document Type: Article

References (150)

1. US Food and Drug Administration (FDA). Guidance for Industry. Patient Reported Outcome Measures: use in medical product development to support labelling claims. 2009.
2. McColl E, Jacoby A, Thomas L, Soutter J, Bamford C, Steen N, et al. Design and use of questionnaires: a review of best practice applicable to surveys of health service staff and patients. Heath Technol Assess 2001;5(31).
3. Edwards PJ, Roberts I, Clarke M, DiGuiseppi C, Wentz R, Kwan I, et al. Methods to increase response to postal and electronic questionnaires. Cochrane Database Syst Rev 2009;3:MR000008.
4. Groves R, Couper M. Nonresponse in household surveys. New York, NY: Wiley; 1998.
5. Groves RM, Singer E, Corning A. Leverage-saliency theory of survey participation. Publ Opin Q 2000;64:299-308.
6. Tourangeau R. Cognitive aspects of survey measurement and mismeasurement. Int J Public Opin Res 2003;15:3-7.
7. Jobe JB. Cognitive psychology and self-reports: models and methods. Qual Life Res 2003;12:219-27.
8. Tourangeau R, Rips LJ, Rasinski K. The psychology of survey response. Cambridge: Cambridge University Press; 2000.
9. Tourangeau R, Smith TW. Asking sensitive questions: the impact of data collection mode, question format, and question context. Publ Opin Q 1996;60:275-304.
10. Honaker LM. The equivalency of computerised and conventional MMPI administration: a critical review. Clin Psychol Rev 1988;8:561-77.
11. Butler C, Kinnersley P, Hood K, Robling M, Prout H, Rollnick S, et al. The natural history of acute upper respiratory tract infection in children: a pragmatic randomised controlled trial cohort. BMJ 2003;327:1088-9.
12. Formica M, Kabbara K, Clark R, McAlindon T. Can clinical trials requiring frequent participant contact be conducted over the internet? Results from an online randomised controlled trial evaluating a topical ointment for herpes labialis. J Med Internet Res 2004;6:e6.
13. McAlindon T, Formica M, Kabbara K, LaValley M, Lehmer M. Conducting clinical trials over the internet: a feasibility study. BMJ 2003;327:484-7.
14. Steeh C, Kirgis N, Cannon B, DeWitt J. Are they really as bad as they seem? Nonresponse rates at the end of the twentieth century. JOS 2001;17:227-47.
15. Bloom DE. Technology, experimentation, and the quality of survey data. Science 1998;280:847-8.
16. Shih T-H, Fan X. Response rates and mode preferences in web-mail mixed-mode surveys: a meta-analysis. Int J Internet Sci 2007;2:59-82.
17. Shih T-H, Fan X. Comparing response rates in e-mail and paper surveys: a meta-analysis. Educ Res Rev 2009;4:26-40.
18. Groves RM, Fowler, Jr, FJ, Couper MP, Lepkowski JM, Singer E, Tourangeau R. Survey methodology. Hoboken, NJ: John Wiley & Sons; 2004.
19. Crow R, Gage H, Hampson S, Hart J, Kimber A, Storey L, et al. The measurement of satisfaction with healthcare: implications for practice from a systematic review of the literature. Health Technol Assess 2002;6(32).
20. Podsakoff PM, MacKenzie SB, Lee J-Y, Podsakoff NP. Common method biases in behavioural research: a critical review of the literature and recommended remedies. J Appl Psychol 2003;88:879-903.
21. Robling MR, Hood K, Greene G, Sayers A, Ingledew DK, Russell IT, et al. Evaluating the impact of data collection mode upon response to subjective surveys: main results from the MODEARTS systematic review. International Society for Quality of Life Research, 14th Annual Conference, Toronto, ON, Canada, 2007.
22. Birnbaum MH. Human research and data collection via the internet. Ann Rev Psychol 2004;55:803-32.
23. Reeve BB, Hays RD, Chang C-H, Perfetto EM. Applying item response theory to enhance health outcomes assessment. Qual Life Res 2007;16:1-3.
24. Walker J, Böhnke JR, Cerny T, Strasser F. Development of symptom assessments utilising item response theory and computer-adaptive testing: a practical method based on a systematic review. Crit Rev Oncol Haematol 2010;73:47-67.
25. Fayers PM. Applying item response theory and computer adaptive testing: the challenges for health outcomes assessment. Qual Life Res 2007;16:187-94.
26. Dillman DA, Christian LM. Survey mode as a source of instability in responses across surveys. Field Methods 2005;17:30-52.
27. Greenberg A, Manfield MN. On the reliability of mail questionnaires in product tests. J Mark 1957;21:342-5.
28. Sorensen S, Rylander R, Berglund K. Interviews and mailed questionnaires for the evaluation of annoyance reactions. Environ Res 1974;8:166-70.
29. Krosnick JA. Response strategies for coping with the cognitive demands of attitude measures in surveys. Appl Cognitive Psychol 1991;5:213-36.
30. Dubrovsky VJ, Kiesler S, Sethna BN. The Equalization Phenomenon: status effects in computer-mediated and face-to-face decision-making groups. HCI 1991;6:119-46.
31. Jones EF, Forrest JD. Underreporting of abortions in surveys of U.S. women: 1976 to 1988. Demography 1992;29:113-26.
32. Booth-Kewley S, Edwards JE, Rosenfeld P. Impression management, social desirability, and computer administration of attitude questionnaires: does the computer make a difference? J Applied Psychol 1992;77:562-6.
33. Turner CF, Ku L, Rogers SM, Lindberg LD, Pleck JH, Sonenstein FL. Adolescent sexual behaviour, drug use, and violence: increased reporting with computer survey technology. Science 1998;280:867-73.
34. Tourangeau R, Couper MP, Steiger DM. Humanising self-administered surveys: experiments on social presence in web and IVR surveys. Comput Hum Behav 2003;19:1-24.
35. Sproull L, Subramani M, Kiesler S, Walker JH, Waters K. When the interface is a face. HCI 1996;11:97-124.
36. Yates BT, Wagner JL, Suprenant LM. Recall of health-risky behaviours for the prior 2 or 4 weeks via computerised versus printed questionnaire. Comput Hum Behav 1997;13:83-110.
37. Smith TW. The impact of the presence of others on a respondent's answers to questions. Int J Publ Opin Res 1997;9:33-47.
38. Dillman DA. Mail and internet surveys. 2nd edn. New York, NY: John Wiley & Sons; 2000.
39. Krosnick JA. Survey research. Annu Rev Psychol 1999;50:537-67.
40. Krosnick JA, Holbrook AL, Berent MK, Carson RT, Hanemann WM, Kopp RJ, et al. The impact of 'no opinion' response options on data quality: non-attitude reduction or an invitation to satisfice? Publ Opin Q 2002;66:371-403.
41. Helgeson JG, Ursic ML. The decision process equivalency of electronic versus pencil-andpaper data collection methods. Soc Sci Comp Rev 1989;7:296-310.
42. Holbrook AL, Green MC, Krosnick JA. Telephone versus face-to-face interviewing of national probability samples with long questionnaires. Publ Opin Q 2003;67:79-125.
43. Kelly D, Harper DJ, Landau B. Questionnaire mode effects in interactive information retrieval experiments. Inf Process Manag 2008;44:122-41.
44. Krosnick JA, Alwin DF. An evaluation of cognitive theory of response-order effects in survey measurement. Public Opin Q 1987;51:201-19.
45. Jäckle A, Roberts C, Lynn P. Telephone versus face-to-face interviewing: mode effects on data quality and likely causes. Report on phase II of the ESS-Gallup Mixed Mode Methodology Project. Colchester: University of Essex; 2006.
46. Sudman S, Bradburn NM. Response effects in surveys. Chicago, IL: Aldine Publishing Company; 1974.
47. Sudman S, Bradburn NM. Asking questions: A practical guide to questionnaire design. San Francisco, CA: Jossey-Bass Publishers; 1982.
48. Paulhus DL. Two-component models of socially desirable responding. J Pers Soc Psychol 1984;46:598-609.
49. de Leeuw ED. Data quality in mail, telephone and face to face surveys. Amsterdam: TT-Publikaties; 1992.
50. Richman WL, Kiesler S, Weisband S, Drasgow F. A meta-analytic study of social desirability distortion in computer-administered questionnaires, traditional questionnaires and interviews. J Appl Psychol 1999;84:754-75.
51. Dwight SA, Feigelson ME. A quantitative review of the effect of computerised testing on the measurement of social desirability. Educ Psychol Meas 2000;60:340-60.
52. Finegan JE, Allen NJ. Computerised and written questionnaires: are they equivalent? Comput Hum Behav 1994;10:483-96.
53. Moher D, Cook DJ, Eastwood S, Olkin I, Rennie D, Stroup DF, et al. Improving the quality of reports of meta-analyses of randomised controlled trials: the QUOROM statement. Lancet 1999;354:1896-900.
54. Whitener EM, Klein HJ. Equivalence of computerised and traditional research methods: the roles of scanning, social environment, and social desirability. Comput Hum Behav 1995;11:65-75.
55. Shaeffer NC, Presser S. The science of asking questions. Annu Rev Sociol 2003;29:65-88.
56. Knowles ES, Condon CA. Why people say 'Yes': a dual-process theory of acquiescence. J Pers Soc Psychol 1999;77:379-86.
57. Jordan LA, Marcus AC, Reeder LG. Response styles in telephone and household interviewing: a field experiment. Publ Opin Q 1980;44:210-22.
58. Day H, Jankey S. Lessons from the literature: towards a holistic model of quality of life. In Renwick R, Brown I, Nagler M, editors. Quality of life in health promotion and rehabilitation. Thousand Oaks, CA: Sage Publications; 1996.
59. Schwartz CE, Rapkin BD. Reconsidering the psychometrics of quality of life assessment in light of response shift and appraisal. Health Qual Life Outcomes 2004;2:16.
60. Cote JA, Buckley R. Estimating trait, method, and error variance: generalising across 70 construct validation studies. J Market Res 1987;24:315-18.
61. Ghanem KG, Hutton HE, Zenilman JM, Zimba R, Erbelding EJ. Audio computer assisted self interview and face to face interview modes in assessing response bias among STD clinic patients. Sex Transm Infect 2004;81:421-5.
62. Hellard ME, Sinclair MI, Forbes AB, Fairley CK. Methods used to maintain a high level of involvement in a clinical trial. J Epidemiol Community Health 2001;55:348-51.
63. Chang B-H, Hendricks AM, Slawsky MT, Locastro JS. Patient recruitment to a randomised clinical trial of behavioural therapy for chronic heart failure. BMC Med Res Methodol 2003;4:8.
64. Hussain-Gambles M. South Asian patients' views and experiences of clinical trial participation. Fam Pract 2004;21:636-42.
65. Hewitt M. Attitudes towards interview mode and comparability of reporting sexual behaviour by personal interview and audio computer-assisted self-interviewing: analyses of the 1995 National Survey of Family Growth. Sociol Methods Res 2002;31:3.
66. Fitzpatrick R, Davey C, Buxton MJ, Jones DR. Evaluating patient-based outcome measures for use in clinical trials. Health Technol Assess 1998;2(14).
67. Dale O, Hagen KB. Despite technical problems personal digital assistants outperform pen and paper when collecting patient diary data. J Clin Epidemiol 2007;60:8-17.
68. Allenby A, Matthews J, Beresford J, McLachlan S. The application of computer touch-screen technology in screening for psychosocial distress in an ambulatory oncology setting. Eur J Cancer Care 2002;11:245-53.
69. Bushnell DM, Reilly MC, Galani C, Martin ML, Ricci J-F, Patrick DL, et al. Validation of electronic data capture of the Irritable Bowel Syndrome-Quality of Life measure, and Activity Impairment Questionnaire for Irritable Bowel Syndrome and the EuroQol. Value Health 2006;9:98-105.
70. Addington-Hall J. Research sensitivities to palliative care patients. Eur J Cancer Care 2002;11:220-4.
71. Marshall S, Haywood K, Fitzpatrick R. Impact of patient-reported outcome measures on routine practice: a structured review. J Eval Clin Pract 2006;12:559-68.
72. Dawson J, Doll H, Fitzpatrick R, Jenkinson C, Carr AJ. Routine use of patient reported outcome measures in healthcare settings. BMJ 2010;340:464-7.
73. Timmins N. Assessing patient care -NHS goes to the PROMS. BMJ 2008;336:1464-5.
74. Greenhalgh J, Meadows K. The effectiveness of the use of patient-based measures of health in routine practice in improving the process and outcomes of patient care: a literature review. J Eval Clin Pract 1999;5:410-16.
75. Espallargues M, Valderas JM. Provision of feedback on perceived health status to health care professionals: a systematic review of its impact. Med Care 2000;38:175-86.
76. Guyatt GH, Estwing Ferrans C, Halyard MY, Revicki DA, Symonds TL, Varricchio CG, et al. Exploration of the value of health-related quality-of-life information from clinical research and into clinical practice. Mayo Clinic Proceedings 2007;82:1229-39.
77. Gutteling JJ, Busschbach JJ, de Man RA, Darlington A-SE. Logistic feasibility of health related quality of life measurement in clinical practice: results of a prospective study in a large population of chronic liver patients. Health Qual Life Outcomes 2008;6:97.
78. Solari A. Role of health-related quality of life measures in the routine care of people with multiple sclerosis. Health Qual Life Outcomes 2005;3:16.
79. Greenhalgh J, Long AF, Flynn R. The use of patient reported outcome measures in routine clinical practice: lack of impact or lack of theory? Soc Sci Med 2005;60:833-43.
80. Department of Health. Guidance on the routine collection of Patient Reported Outcome Measures (PROMs). London: Department of Health; 2008.
81. Varni JW, Burwinkle TM, Lane MM. Health-related quality of life measurement in paediatric clinical practice: an appraisal and precept for future research and application. Health Qual Life Outcomes 2005;3:34.
82. Engelen V, Haverman L, Koopman H, Schouten-van Meeteren N, Meijer-van den Bergh E, Vrijmoet-Wiersma J, et al. Development and implementation of a patient reported outcome intervention (QLIC-ON PROfile) in clinical paediatric oncology practice. Patient Educ Couns 2010:doi:10.1016/j.pec.2010.02.003.
83. Roberts C. Mixing modes of data collection in surveys: a methodological review. Southampton: ESRC National Centre for Research Methods; 2007.
84. Blumberg SJ, Luke JV, Cynamon ML. Telephone coverage and health survey estimates: evaluating the need for concern about wireless substitution. Am J Public Health 2006;96:926-31.
85. Chittleborough CR, Taylor AW, Baum FE, Hiller JE. Non-response to a life course socioeconomic position indicator in surveillance: comparison of telephone and face-to-face modes. BMC Med Res Methodol 2008;8:54.
86. Warner SL. Randomised response: a survey technique for eliminating evasive answer bias. J Am Stat Assoc 1965;60:63-9.
87. Lensvelt-Mulders GJLM, Hox JJ, van der Heijden PGM, Maas CJM. Meta-analysis of randomised response research: thirty-five years of validation. Sociol Methods Res 2005;33:319-48.
88. Rasinski KA, Visser PS, Zagatasky M, Rickett EM. Using implicit goal priming to improve the quality of self-report data. J Exp Soc Psychol 2005;41:321-7.
89. Groves RM, Fowler Jr FJ, Couper MP, Lepkowski JM, Singer E, Tourangeau R. Survey methodology. Wiley Series in Survey Methodology. London: John Wiley & Sons; 2004.
90. Carr E, Worth A. The use of telephone interview for research. Nurs Times Res 2001;6:511-24.
91. Hopewell S, McDonald S, Clarke M, Egger M. Grey literature in meta-analysis of randomised trials in health care interventions. Cochrane Library 2003;1.
92. Khan KS, Ter Riet G, Glanville J, Sowden AJ, Kleijnen J (editors). Undertaking systematic reviews of research on effectiveness. CRD's guidance for carrying out or commissioning reviews. 2nd edn. CRD Report No. 4. York: NHS Centre for Reviews and Dissemination (CRD), University of York; 2001.
93. de Leeuw ED, Hox JJ, Snijkers G. The effect of computer-assisted interviewing on data quality. A review. J Market Res Soc 1995;37:325.
94. Downs SH, Black N. The feasibility of creating a checklist for the assessment of the methodological quality both of randomised and non-randomised studies of health care interventions. J Epidemiol Community Health 1998;52:377-84.
95. Kmet L, Lee R, Cook L. Standard quality assessment criteria for evaluating primary research papers from a variety of fields. Edmonton, AB: Alberta Heritage Foundation for Medical Research; 2004.
96. Group TSo RT. A proposal for structured reporting of randomised controlled trials. The Standards of Reporting Trials Group. JAMA 1994;272:1926-31.
97. Anello C, Fleiss JL. Exploratory or analytic meta-analysis: should we distinguish between them? J Clin Epidemiol 1995;48:109-16.
98. van den Noorgate W, Onghena P. Multi-level meta-analysis: a comparison with traditional meta-analytic procedures. Educ Psychol Meas 2003;63:765-90.
99. Dunn G, Roberts C. Modelling method comparison data. Stat Methods Med Res 1999;8:161-78.
100. Bland JM, Altman DG. Measuring agreement in method comparison studies. Stat Methods Med Res 1999;8:135-60.
101. Altman DG, Bland JM. Measurement in medicine: the analysis of method comparison studies. Statistician 1983;32:307-17.
102. Williamson PR, Lancaster GA, Craig JV, Smuth RL. Meta-analysis of method comparison studies. Stats Med 2002;21:2013-25.
103. Moher D, Liberati A, Tetzlaff J, Altman DG, Group TP. Preferred Reporting Items for Systematic Reviews and Meta-analyses: The PRISMA statement. BMJ 2009;339:b2535. doi: 10.1136/bmj.b2535.
104. Bowling A, Bond M, Jenkinson C, Lamping DL. Short Form 36 (SF-36) Health Survey questionnaire: which normative data should be used? Comparisons between the norms provided by the Omnibus Survey in Britain, the Health Survey for England and the Oxford Healthy Life Survey. J Public Health Med 1999;21:255-70.
105. Berthelsen CL, Stilley KR. Automated personal health inventory for dentistry: a pilot study. JADA 2000;131:59-66.
106. Evans M, Kessler D, Lewis G, Peters TJ, Sharp D. Assessing mental health in primary care research using standardised scales: can it be carried out over the telephone? Psychol Med 2004;34:157-62.
107. Allison T, Ahmad T, Brammah T, Symmons D, Urwin M. Can findings from postal questionnaires be combined with interview results to improve the response rate among ethnic minority populations? Ethn Health 2003;8:63-9.
108. Ooijen MV, Ivens UI, Johansen C, Skov T. Comparison of a self-administered questionnaire and a telephone interview of 146 Danish waste collectors. Am J Ind Med 1997;31:653-8.
109. Cohen J. Statistical power analysis for the behavioural sciences. 2nd edn. Hillsdale, NJ: Lawrence Erlbaum; 1988.
110. Ware J, Kosinski M, Dewey J. How to score version two of the SF-36 Health Survey. Lincoln, RI: QualityMetric Incorporated; 2000.
111. Amodei N, Katerndahl DA, Larme AC, Palmer R. Interview versus self-answer methods of assessing health and emotional functioning in primary care patients. Psychol Rep 2003;92:937-48.
112. Jones D, Kazis L, Lee A, Rogers W, Skinner K, Cassar L, et al. Health status assessments using the Veterans SF-12 and SF-36: methods for evaluating outcomes in the Veterans Health Administration. J Ambul Care Manage 2001;24:68-86.
113. Lyons RA, Wareham K, Lucas M, Price D, Williams J, Hutchings HA. SF-36 scores vary by method of administration: implications for study design. J Public Health Med 1999;21:41-5.
114. Perkins JJ, Sanson-Fisher RW. An examination of self- and telephone-administered modes of administration for the Australian SF-36. J Clin Epidemiol 1998;51:969-73.
115. Saleh KJ, Radosevich DM, Kassim RA, Moussa M, Dykes D, Bottolfson H, et al. Comparison of commonly used orthopaedic outcome measures using palm-top computers and paper surveys. J Orthop Res 2002;20:1146-51.
116. Unruh M, Yan G, Radeva M, Hays RD, Benz R, Athienites NV, et al. Bias in assessment of health-related quality of life in a hemodialysis population: a comparison of selfadministered and interviewer-administered surveys in the HEMO study. J Am Soc Nephrol 2003;14:2132-41.
117. van Campen C, Sixma H, Kerssens JJ, Peters L. Comparisons of the costs and quality of patient data collection by mail versus telephone versus in-person interviews. Eur J Public Health 1998;8:66-70.
118. Weinberger M, Oddone EZ, Samsa GP, Landsman PB. Are health-related quality-of-life measures affected by the mode of administration? J Clin Epidemiol 1996;49:135-40.
119. Abdoh A, Krousel-Wood MA, Re RN. Validity And Reliability Assessment Of An Automated Telephone Survey System (208). Congress Epidemiol Abstr 2001:S87.
120. Bliven BD, Kaufman SE, Spertus JA. Electronic collection of health-related quality of life data: validity, time benefits, and patient preference. Qual Life Res 2001;10:15-22.
121. Caro JJ, Caro I, Caro J, Wouters F, Juniper EF. Does electronic implementation of questionnaires used in asthma alter responses compared to paper implementation? Qual Life Res 2001;10:683-91.
122. Molitor F, Kravitz RL, To Y, Fink A. Methods in survey research: evidence for the reliability of group administration vs personal interviews. Am J Public Health 2001;91:826-7.
123. Revicki DA, Tohen M, Gyulai L, Thompson C, Pike S, Davis-Vogel A, et al. Telephone versus in-person clinical and health status assessment interviews in patients with bipolar disorder. Harvard Rev Psychiatr 1997;5:75-81.
124. Ryan JM, Corry JR, Attewell R, Smithson MJ. A comparison of an electronic version of the SF-36 General Health Questionnaire to the standard paper version. Qual Life Res 2002;11:19-26.
125. Weinberger M, Nagle B, Hanlon JT, Samsa GP, Schmader K, Landsman PB, et al. Assessing health-related quality of life in elderly outpatients: telephone versus face-to-face administration. J Am Geriatr Soc 1994;42:1295-9.
126. Wilson AS, Kitas GD, Carruthers DM, Reay C, Skan J, Harris S, et al. Computerised information-gathering in specialist rheumatology clinics: an initial evaluation of an electronic version of the Short Form 36. Rheumatology 2002;41:268-73.
127. Kazis L. The Veterans SF-36 health status questionnaire: development and application in the Veteran's Health Administration. Med Outcomes Trust Monit 2000;5(1).
128. Graham JR. MMPI-2: Assessing personality and psychopathology. 4th edn. New York, NY: Oxford University Press; 2006.
129. Biskin BH, Kolotkin RL. Effects of computerised administration on scores on the Minnesota Multiphasic Personality Inventory. Appl Psychol Meas 1977;1:543-9.
130. Evan WM, Miller JR. Differential effects on response bias of computer vs. conventional administration of a social science questionnaire: An exploratory methodological experiment. Behav Sci 1969;14:216-27.
131. Hart RR, Goldstein MA. Computer-assisted psychological assessment. Comput Hum Serv 1985;1:69-75.
132. Honaker L, Harrell TH, Buffaloe JD. Equivalency of Microtest computer MMPI administration for standard and special scales. Comput Hum Behav 1988;4:323-37.
133. Lambert ME, Andrews RH, Rylee K, Skinner JR. Equivalence of computerised and traditional MMPI administration with substance abusers. Comput Hum Behav 1987;3:139-43.
134. Locke SD, Gilbert BO. Method of psychological assessment, self-disclosure, and experiential differences: A study of computer, questionnaire, and interview assessment formats. J Soc Behav Pers 1995;10:255-63.
135. White DM, Clements CB, Fowler RD. A comparison of computer administration with standard administration of the MMPI. Comput Hum Behav 1985;1:153-62.
136. Pinsoneault TB. Equivalency of computer-assisted and paper-and-pencil administered versions of the Minnesota Multiphasic Personality Inventory-2. Comput Hum Behav 1996;12:291-300.
137. Schuldberg D. The MMPI is less sensitive to the automated testing format than it is to repeated testing: Item and scale effects. Comput Hum Behav 1988;4:285-98.
138. Coons SJ, Gwaltney CJ, Hayes RD, Lundy JJ, Sloan JA, Revicki DA, et al. Recommendations on evidence needed to support measurement equivalence between electronic and paperbased patient reported outcome (PRO) measures: ISPOR ePRO good research practices task force report. Value Health 2009;12:419-29.
139. Elm EV, Altman DG, Eggar M, Pocock SJ, Gotzsche PC, Vandenbroucke JP, et al. The Strengthening the reporting of observational studies in epidemiology (STROBE) statement: guidelines for reporting of observational studies. J Clin Epidemiol 2008;61:344-9.
140. Shulz KF, Altman DG, Moher D, Group C. CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials. Ann Intern Med 2010;152.
141. Albaum GS, Evangelista F, Medina N. Role of response behaviour theory in survey research: a cross-national study. J Business Res 1998;42:115-25.
142. Willis G, Royston P, Bercini D. The use of verbal report methods in the development and testing of survey questionnaires. Appl Cogn Psychol 1991;5:251-67.
143. Collins D. Pretesting survey instruments: an overview of cognitive methods. Qual Life Res 2003;12:229-38.
144. McColl E, Meadows K, Barofsky I. Cognitive aspects of survey methodology and quality of life assessment. Qual Life Res 2003;12:217-18.
145. Tourangeau R. Survey research and societal change. Ann Rev Psychol 2004;55:775-801.
146. Doll H, McPherson K, Davies J, Flood A, Smith J, Williams G, et al. Reliability of questionnaire responses as compared with interview in the elderly: views of the outcome of transurethral resection of the prostate. Soc Sci Med 1991;33:1303-8.
147. Weir P, Beri S. A comparison and evaluation of two survey data collection methodologies: CATI vs mail. Proc Surv Res Methods Sec Am Stat Assoc 2000:388-93.
148. Edwards P, Roberts I, Clarke M, DiGuiseppi C, Pratap S, Wentz R, et al. Increasing response rates to postal questionnaires: systematic review. BMJ 2002;324:1183-7.
149. Krysan M, Schuman H, Scott LJ, Beatty P. Response rates and response content in mail versus face-to-face surveys. Publ Opin Q 1994;58:381-99.
150. Damiano A, McGrath M, William M, Snyder C, LeWitt P, Reyes P, et al. Evaluation of a measurement strategy for Parkinson's Disease: assessing patient health-related quality of life. Qual Life Res 2000;9:87-100.