Validity evidence based on response processes;Evidencia de validez basada en los procesos de respuesta

Psicothema

Volumn 26, Issue 1, 2014, Pages 136-144

  Padilla, José Luis ^a   Benítez, Isabel ^a  

^a UNIVERSITY OF GRANADA   (Spain)

Author keywords

Cognitive interviewing; Evidence of response processes; Standards; Validity

Indexed keywords

COGNITION; EDUCATION; EPIDEMIOLOGY; EVALUATION STUDY; EYE MOVEMENT; FORECASTING; HUMAN; INFORMATION PROCESSING; PROCEDURES; PSYCHOLOGIC TEST; QUALITATIVE RESEARCH; QUESTIONNAIRE; REACTION TIME; SAMPLE SIZE; STANDARDS; VALIDATION STUDY;

COGNITION; DATA COLLECTION; EDUCATIONAL MEASUREMENT; EVALUATION STUDIES AS TOPIC; EYE MOVEMENTS; FOCUS GROUPS; FORECASTING; HUMANS; INTERVIEW, PSYCHOLOGICAL; QUALITATIVE RESEARCH; QUESTIONNAIRES; REACTION TIME; SAMPLE SIZE; SAMPLING STUDIES; VALIDATION STUDIES AS TOPIC;

EID: 84892888078     PISSN: 02149915     EISSN: 1886144X     Source Type: Journal    
DOI: 10.7334/psicothema2013.259     Document Type: Article

References (51)

1. American Psychological Association, American Educational Research Association, & National Council on Measurement in Education (1985). Standards for educational and psychological testing. Washington, D.C.: American Psychological Association.
2. American Educational Research Association, American Psychological Association, & National Council on Measurement in Education (1999). Standards for educational and psychological testing. Washington, D.C.: American Educational Research Association.
3. Althof, S.E., Perelman, M.A., & Rosen, R.C. (2011). The Subjective Sexual Arousal Scale for Men (SSASM): Preliminary development and psychometric validation of a multidimensional measure of subjective male sexual arousal. The Journal of Sexual Medicine, 8, 2255-2268.
4. Beatty, P., & Willis, G.B. (2007). Research synthesis: The practice of cognitive interviewing. Public Opinion Quarterly, 71, 287-311.
5. Benitez, I., & Padilla, J.L. (2013). Analysis of nonequivalent assessments across different linguistic groups using a mixed methods approach: Understanding the causes of differential item functioning by cognitive interviewing. Journal of Mixed Methods Research, Online first. doi: 10.1177/1558689813488245.
6. Borsboom, D., Mellenbergh, G.J., & Van Heerden, J. (2004). The concept of validity. Psychological Review, 111, 1061-1071.
7. Brod, M., Hammer, M., Christensen, T., Lessard, S., & Bushnell, D.M. (2009). Understanding and assessing the impact of treatment in diabetes: The Treatment-Related Impact Measures for Diabetes and Devices (TRIM-Diabetes and TRIM-Diabetes Device). Health and Quality of Life Outcomes, 7, 83.
8. Castillo, M., Padilla, J.L., Gomez, J., & Andres, A. (2010). A productivity map of cognitive pretest methods for improving survey questions. Psicothema, 22, 475-481.
9. Castillo, M., & Padilla, J.L. (2012). How cognitive interviewing can provide validity evidence of the response processes to scale items. Social Indicators Research. Online first. doi: 10.1007/s11205-012-0184-8.
10. Cepeda, N.J., Blackwell, K.A., & Munakata, Y. (2013). Speed isn't everything: Complex processing speed measures mask individual differences and developmental changes in executive control. Developmental Science, 16, 269-286.
11. Cizek, G.J., Rosenberg, S.L., & Koons, H.H. (2007). Sources of validity evidence for educational and psychological tests. Educational and Psychological Measurement, 68, 397-412.
12. Cizek, G.J. (2012). Defining and distinguishing validity: Interpretations of score meaning and justifications of test use. Psychological Methods, 17, 31-43.
13. Collins, D. (2003). Pretesting survey instruments: An overview of cognitive methods. Quality of Life Research, 12, 229-238.
14. Conrad, F.G.; Blair, J. (2009). Sources of error in cognitive interviews. Public Opinion Quarterly, 73, 32-55.
15. Day, R.F. (2010). Examining the validity of the Needleman-Wunsch algorithm in identifying decision strategy with eye-movement data. Decision Support Systems, 49, 396-403.
16. Deal, L.S., DiBenedetti, D.B., Williams, V.S., & Fehnel, S.E. (2010). The development and validation of the daily electronic Endometriosis Pain and Bleeding Diary. Health and Quality of Life Outcomes, 8, 64.
17. Elling, S., Lentz, L., & de Jong, M. (2012). Combining concurrent think-aloud protocols and eye-tracking observations: An analysis of verbalizations and silences. IEEE Transactions on Professional Communication, 55, 206-220.
18. Embretson, S. (1983). Construct validity: Construct representation versus nomothetic span. Psychological Bulletin, 93, 179-197.
19. Ercikan, K., Arim, R., & Law, D. (2010). Application on think aloud protocols for examining and confirming sources of differential item functioning identified by experts review. Educational Measurement: Issues and Practices, 29, 24-35.
20. Ferdous, A.A., & Plake, B.S. (2005). Understanding factors that influence decisions of panelists in a standard setting study. Applied Measurement in Education, 18, 257-267.
21. Gadermann, A.M., Guhn, M., & Zumbo, B.D. (2011). Investigating the substantive aspect of construct validity for the Satisfaction with Life Scale adapted for children: A focus on cognitive processes. Social Indicator Research, 100, 37-60.
22. Garrett (1937). Statistics in psychology and education. New York: Longmans, Green.
23. Gehlbach, H., & Brinkworth, M.E. (2011). Measure twice, cut down error: A process for enhancing the validity of survey scales. Review of General Psychology, 15, 380-387.
24. Hawthorne, G., Davidson, N., Quinn, K. McCrate, F., Winkler, I., Lucas, R., Kilian, R., & Molzahn, A. (2006). Issues in conducting crosscultural research: implementation of an agreed international protocol designed by the WHOQOL Group for the conduct of focus groups eliciting the quality of life of older adults. Quality of Life Research 15, 1257-1270.
25. Ivie, J.L., & Embretson, S.E. (2010). Cognitive process modeling of spatial ability: The assembling objects task. Intelligence, 38, 324-335.
26. Jabine, T.B., Straf, M.L., Tanur, J.M., & Tourangeau, R. (1984). Cognitive aspects of survey methodology: Building a bridge between disciplines. Washington, DC: National Academy Press.
27. Kane, M.T. (1992). An argument-based approach to validity. Psychological Bulletin, 112, 527-535.
28. Kane, M.T. (2006). Validation. In B.L. Robert (Ed.), Educational Measurement (4th ed., pp. 17-64). Wesport, CT: Praeger.
29. Kane, M.T. (2013). Validation as a pragmatic, scientific activity. Journal of Educational Measurement, 50, 115-122.
30. Krall, J.S., & Lohse, B. (2010). Cognitive testing with female nutrition and education assistance program participants informs validity of the Satter eating competence inventory. Journal of Nutrition Education and Behavior, 42, 277-283.
31. Krosnick, J.A. (1999). Survey research. Annual Review of Psychology, 50, 537-567.
32. Martin, E. (2004). Vignettes and respondent debriefing for questionnaire design and evaluation. En Presser, S., Rothgeb, J.M., Couper, M.P., Lessler, J.T., Martin, E., Martin, J., & Singer, E. (2004). Methods for Testing and Evaluating Survey Questionnaires. (pp. 149-173). New York: Wiley-Interscience.
33. Messick, S. (1989). Validity. In R.L. Linn (Ed.), Educational Measurement 3rd ed. (pp. 13-103). New York: McMillan.
34. Messick, S. (1990). Validity of test interpretation and use. Research Report 90-11. Education Testing Service.
35. Miller, K., Chepp, V., Willson, S., & Padilla, J.L. (2014 in press). Cognitive Interviewing Methodology: A Sociological Approach for Survey Question Evaluation. New York, NJ: John Wiley and Sons.
36. Mislevy, R.J., Steinberg, L.S., & Almond, R.G. (2002). On the role of task model variables in assessment design. In S.H. Irvine & P.C. Kyllonen (Eds.), Item generation for test development (pp. 97-128). Mahwah, NJ: Lawrence Erlbaum.
37. Olt, H., Jirwe, M., Gustavsson, P., & Emami, A. (2010). Psychometric evaluation of the Swedish adaptation of the Inventory for Assessing the Process of Cultural Competence Among Healthcare Professionals-Revised (IAPCC-R). Journal of transcultural nursing: Official journal of the Transcultural Nursing Society / Transcultural Nursing Society, 21, 55-64.
38. Sireci, S.G. (2009). Packing and unpacking sources of validity evidence: History repeats itself again. In R.W. Lissitz (Ed.), The Concept of Validity (pp. 19-39). Charlotte, NC: Information Age Publishing, Inc.
39. Sireci, S.G. (2012). "De-constructing" Test Validation. Paper presented at the annual conference of the National Council on Measurement in Education as part of the symposium "Beyond Consensus: The Changing Face of Validity" (P. Newton, Chair). April 14, 2012, Vancouver, Canada.
40. Sireci, S., & Faulkner-Bond, M. (2014). Validity evidence based on test content. Psicothema, 26, xx-xx???.
41. Sireci, S.G., & Parker, P. (2006). Validity on trial: Psychometric and legal conceptualizations of validity. Educational Measurement, 25, 27-34.
42. Sireci, S.G., Baldwin, P., Martone, A., Zenisky, A.L., Kaira, L., Lam, W., Shea, C.L., Han, K., Deng, N., Delton, J., & Hambleton, R.K. (2008). Massachusetts adult proficiency tests technical manual: Version 2: Amherst, MA: Center for Educational Assessment. Available at http:// www.umass.edu/remp/CEA_TechMan.html.
43. Skorupski, W.P., & Hambleton, R.K. (2005). What are panelists thinking when they participate in standard setting studies? Applied Measurement in Education, 18, 233-256.
44. Tourangeau, R. (1984). Cognitive science and survey methods: A cognitive perspective. En T. Jabine, M. Straf, J. Tanur & R. Tourangeau (Eds.), Cognitive Aspects of Survey Methodology: Building a Bridge Between the Disciplines (pp. 73-100). Washington, DC: National Academy Press.
45. Tourangeau, R., Rips, L.J., & Rasinski, K. (2004). The Psychology of Survey Response. Cambridge: Cambridge University Press.
46. Wang, X., & Sireci, S.G. (2013). Evaluating the cognitive levels measured by test items using item response time. Paper presented at the annual conference of the American Educational Research Association. April, 2013, San Francisco, USA.
47. Webber, M.P., & Huxley, P.J. (2007). Measuring access to social capital: The validity and reliability of the Resource Generator-UK and its association with common mental disorder. Social Science & Medicine (1982), 65, 481-492.
48. Willis, G.B. (2005). Cognitive interviewing. Thousand Oaks: Sage Publications.
49. Willson, S., & Miller, K. (2014 in press). Data collection. In K. Miller, V. Chepp, S. Willson & J.L. Padilla (Eds.), Cognitive Interviewing Methodology: A Sociological Approach for Survey Question Evaluation. New York, NJ: John Wiley and Sons.
50. Zumbo, B.D. (2009). Validity as contextualized and pragmatic explanation, and its implications for validation practice. In R.W. Lissitz (Ed.), The Concept of Validity (pp. 65-83). Charlotte, NC: Information Age Publishing, Inc.
51. Zumbo, B.D., & Shear, B.R. (2011). The concept of validity and some novel validation methods. In Northeastern Educational Research Association (p. 56). Rocky Hill, CT.