Cognitive validity of students' self-reports of classroom mastery goal structure: What students are thinking and why it matters

Contemporary Educational Psychology

Volumn 35, Issue 4, 2010, Pages 254-263

  Koskey, Kristin L K ^a   Karabenick, Stuart A ^b   Woolley, Michael E ^c   Bonney, Christina R ^d   Dever, Bridget V ^b  

^a UNIVERSITY OF AKRON   (United States)

^b UNIVERSITY OF MICHIGAN   (United States)

^c UNIVERSITY OF CHICAGO   (United States)

^d TUFTS UNIVERSITY   (United States)

Author keywords

Classroom mastery goal structure; Cognitive interviewing; Cognitive pretesting; Motivation; PALS

Indexed keywords

EID: 77956340701     PISSN: 0361476X     EISSN: 10902384     Source Type: Journal    
DOI: 10.1016/j.cedpsych.2010.05.004     Document Type: Article

References (65)

1. Alwin D.F. Research on survey quality. Sociological Methods and Research 2001, 20:1-39.
2. Ames, C. (1990). Achievement goals and classroom structure: Developing a learning orientation in students. Paper presented at the annual meeting of the American Research Association, Boston, MA (April).
3. Ames C. Classrooms: Goals, structures, and student motivation. Journal of Educational Psychology 1992, 84:261-271.
4. Ames C., Archer J. Achievement goals in the classroom: Students' learning strategies and motivation processes. Journal of Educational Psychology 1988, 80:260-267.
5. Anderman L.H., Anderman E.M. Social predictors of changes in students' achievement goal orientations. Contemporary Educational Psychology 1999, 25:21-37.
6. Anderman E., Midgley C. Changes in personal achievement goals and the perceived classroom goal structures across the transition to middle level schools. Contemporary Educational Psychology 1997, 22:269-298.
7. Anderman E.M., Wolters C. Goals, values, and affect. Handbook of educational psychology 2006, 369-390. Lawrence Erlbaum, Mahwah, NJ. 2nd ed. P. Alexander, P. Winne (Eds.).
8. Barone D.F., Maddux J.E., Snyder C.R. Social cognitive psychology: History and current domains 1997, Plenum Press, New York.
9. Beatty P., Willis C. Research synthesis: The practice of cognitive interviewing. Public Opinion Quarterly 2007, 71(2):287-311.
10. Conrad, F., Blair, J., & Tracy, E. (1999). Verbal reports are data! A theoretical approach to cognitive interviews. In Proceedings of the Federal Committee on Statistical Methodology Research Conference, Tuesday B Sessions, Arlington, VA (pp. 11-20).
11. Day C., Sammons P., Gu Q. Combining qualitative and quantitative methodologies in research on teachers' lives, work, and effectiveness: From integration to synergy. Educational Researcher 2008, 37:330-342.
12. Dowson M., McInerney D.M. The development and validation of the goal orientation and learning strategies survey (GOALS-S). Educational and Psychological Measurement 2004, 64:290-310.
13. Dowson M., McInerney D.M., Nelson G.F. An investigation of the effects of school context and sex differences on students' motivational goal orientations. Educational Psychology 2006, 26:781-811.
14. Drennan J. Cognitive interviewing: Verbal data in the design and pretesting of questionnaires. Journal of Advanced Nursing 2003, 42(1):57-63.
15. Elliot A.J. A conceptual history of the achievement goal construct. Handbook of competence and motivation 2005, 52-72. The Guilford Press, New York. A.J. Elliot, C.S. Dweck (Eds.).
16. Epstein J.L. Longitudinal effects of family-school-person interactions on students outcomes. Research in sociology of education and socialization: Personal change over the life course 1983, Vol. 4. JAI Press, Greenwich, CT, pp. 101-127.
17. Ericsson K.A., Simon H.A. Verbal reports as data. Psychological Review 1980, 87:215-251.
18. Foddy W. An empirical evaluation of in-depth probes used to pretest survey questions. Sociological Methods & Research 1989, 27:103-133.
19. Fulmer S.M., Frijters J.C. A review of self-report and alternative approaches in the measurement of student motivation. Educational Psychological Review 2009, 21:219-246.
20. Hassin R., Uleman J.S., Bargh J.A. The new unconscious 2005, Oxford University Press, New York.
21. Jobe J.B., Mingay D.J. Cognitive research improves questionnaires. American Journal of Public Health 1989, 79:1053-1055.
22. Kane M. An argument-based approach to validity. Psychological Bulletin 1992, 112:527-535.
23. Kane M. Current concerns in validity theory. Journal of Educational Measurement 2001, 38:319-342.
24. Kaplan A., Middleton M.J., Urdan T., Midgley C. Achievement goals and goal structures. Goals goal structures and patterns of adaptive learning 2002, 21-53. Lawrence Erlbaum, Associates, Mahwah, NJ. C. Midgley (Ed.).
25. Karabenick S.A. Perceived achievement goal structure and college student help seeking. Journal of Educational Psychology 2004, 96:569-581.
26. Karabenick S.A., Woolley M.E., Friedel J.M., Ammon B.V., Blazevski J., Bonney C.R., et al. Cognitive processing of self-report items in educational research: Do they think what we mean?. Educational Psychologist 2007, 42:139-151.
27. Lemos M.S. Students' and teachers' goals in the classroom. Learning and Instruction 1996, 6:151-171.
28. Maehr M.L., Anderman E. Reinventing schools for early adolescents: Emphasizing task goals. Elementary School Journal 1993, 93:593-610.
29. Meece J.L., Anderman E.M., Anderman L.H. Classroom goal structure, student motivation, and academic achievement. Annual review of psychology 2006, Vol. 57:487-503. Annual Reviews, Palo Alto, CA.
30. Messick S. Validity. Educational measurement 1989, 13-104. Macmillan, New York. 3rd ed. R.L. Linn (Ed.).
31. Messick S. Validity of psychological assessment: Validation of inferences from persons' responses and performance as scientific inquiry into scoring meaning. American Psychologist 1995, 9:741-749.
32. Messick S. Standards-based score interpretation: Establishing valid grounds for valid inferences. Proceedings of the joint conference on standard setting for large scale assessments, Sponsored by National Assessment Governing Board and The National Center for Education Statistics 1996, Government Printing Office, Washington, DC.
33. Messick S. Validity of performance assessment. Technical issues in large-scale performance assessment 1996, National Center for Educational Statistics, Washington, DC. G. Philips (Ed.).
34. Meyer D.L., Turner J.C. Discovering emotion in classroom motivation research. Educational Psychologist 2002, 37:107-114.
35. Goals, goal structures and patterns of adaptive learning 2002, Erlbaum, Mahwah, NJ. C. Midgley (Ed.).
36. Midgley C., Anderman E., Hicks L. Differences between elementary and middle school teachers and students: A goal theory approach. Journal of Early Adolescence 1995, 15:90-113.
37. Midgley C., Edelin K. Middle school reform and early adolescent well-being: The good news and the bad. Educational Psychologist 1998, 33:195-206.
38. Midgley C., Maehr M., Hicks L., Roeser R., Urdan T., Anderman E., et al. The patterns of adaptive learning scales (PALS) 1996, The University of Michigan, Ann Arbor, MI.
39. Midgley C., Maehr M.L., Hruda L., Anderman E., Anderman L., Freeman K., et al. Manual for the patterns of adaptive learning scales (PALS) 2000, University of Michigan, Ann Arbor.
40. Nolen S.B., Ward C.J. Sociocultural and situative approaches to studying motivation. Advances in motivation and achievement: Social psychological influences 2008, Vol. 15. Springer, UK.
41. Patrick H. Re-examining classroom mastery goal structure. Advances in motivation: Motivating students, improving schools: The legacy of Carol Midgley 2004, Vol. 13:233-263. Elsevier-JAI, Amsterdam.
42. Patrick H., Anderman L.H., Ryan A.M., Edelin K.C., Midgley C. Teachers' communication of goal orientations in four fifth-grade classrooms. The Elementary School Journal 2001, 102:35-58.
43. Patrick H., Ryan A.M. What do students think about when evaluating their classroom's mastery goal structure? An examination of young adolescents' explanations. The Journal of Experimental Education 2008, 77(2):99-123.
44. Shrout P.E. Measurement reliability and agreement in psychiatry. Statistical Methods in Medical Research 1998, 7:301-317.
45. Shrout P.E., Fleiss J.L. Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin 1979, 86:420-428.
46. Stipek D.J., Salmon J.N., Givvin K.B., Kazemi E., Saxe G., MacGyvers V.L. The value (and convergence) of practices suggested by motivation research and promoted by mathematics education reformers. Journal for Research in Mathematics Education 1998, 29:465-488.
47. Sudman S., Bradburn N., Schwarz N. Thinking about answers: The application of cognitive processes to survey methodology 1996, Jossey-Bass, San Francisco.
48. Tourangeau R. Cognitive aspects of survey measurement and mismeasurement. International Journal of Public Opinion Research 2003, 15:3-7.
49. Tucker, C. (1996). Measurement issues surrounding the use of cognitive methods in survey methods in survey research. Paper presented at the 4th ISA conference on social science methodology, Essex, England (July).
50. Turner J.C., Meyer D.K., Midgley C., Patrick H. Teacher discourse and sixth graders' reported affect and achievement behaviors in two high-mastery/high-performance mathematics classrooms. The Elementary School Journal 2003, 103:357-378.
51. Turner J.C., Midgley C., Meyer D.K., Gheen M., Anderman E.M., Kang Y., et al. The classroom environment and students' reports of avoidance strategies in mathematics: A multimethod study. Journal of Educational Psychology 2002, 94:88-106.
52. Turner J.C., Patrick H. How does motivation develop and why does it change? Reframing motivation research. Educational Psychologist 2008, 119-131.
53. Urdan T. Using multiple methods to assess students' perceptions of classroom goal structures. European Psychologist 2004, 9:222-231.
54. Urdan T. The challenges and promise of research on classroom goal structures. Handbook of Research on Schools, Schooling, and Human development 2010, 92-108. Routledge, Mahwah, NJ. J. Meece, J. Eccles (Eds.).
55. Urdan T., Kniesel L., Mason V. Interpreting messages about motivation in the classroom: Examining the effects of achievement goal structures. Advances in motivation and achievement 1999, Vol. 11:123-158. JAI Press, Greenwich, CT.
56. Urdan T., Midgley C., Anderman E.M. The role of classroom goal structure in students' use of self-handicapping strategies. American Educational Research Journal 1998, 35:101-122.
57. Urdan T., Schoenfelder E. Classroom effects on student motivation: Goal structures, social relationships, and competence beliefs. Journal of School Psychology 2006, 44:331-349.
58. Willis G.B. Cognitive interviewing: A tool for improving questionnaire design 2005, Sage, Thousand Oaks, CA.
59. Wilson T.D. Strangers to ourselves: Discovering the adaptive unconscious 2002, Harvard University Press, Cambridge, MA.
60. Winne P., Jamieson-Noel D., Muis K. Methodological issues and advances in researching tactics, strategies, and self-regulated learning. Advances in motivation and achievement: New directions in measures and methods 2002, Vol. 12:156-221. JAI-Elsevier, Amsterdam.
61. Winne P., Perry N.E. Measuring self-regulated learning. Handbook of self-regulation 2000, 531-566. Academic, San Diego, CA. M. Boekaerts, P. Pintrich, M. Zeidner (Eds.).
62. Wolters C.A. Advancing achievement goal theory: Using goal structures and goal orientations to predict students' motivation, cognition, and achievement. Journal of Educational Psychology 2004, 96:236-250.
63. Wolters C.A., Gonzalez A.L. Classroom climate and motivation: A step toward integration. Advances in motivation and achievement: Social psychological influences 2008, Vol. 15:493-519. Emerald, UK.
64. Woolley M.E., Bowen G.L., Bowen N.K. Cognitive pretesting and the developmental validity of child self-report instruments: Theory and applications. Research on Social Work Practice 2004, 14(3):191-200.
65. Woolley M.E., Bowen G.L., Bowen N.K. The development and evaluation of procedures to assess child self-report item validity. Educational and Psychological Measurement 2006, 66:687-700.