Self-rated competences questionnaires from a design perspective

Educational Research Review

Volumn 7, Issue 1, 2012, Pages 1-18

  Braun, Edith ^a   Woodley, Alan ^b   Richardson, John T E ^b   Leidner, Bernhard ^c  

^a HIS Institut Fur Hochschulforschung   (Germany)

^b OPEN UNIVERSITY   (United Kingdom)

^c UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

Higher education; Questionnaire design; Self rated competences

Indexed keywords

EID: 84856233774     PISSN: 1747938X     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.edurev.2011.11.005     Document Type: Review

References (71)

1. Adelman C. The Bologna club: What US higher education can learn from a decade of European reconstruction? 2008, Institute for Higher Education Policy, Washington, DC, [Retrieved from ].
2. Standards for educational and psychological testing 1999, American Educational Research Association, American Psychological Association, & National Council on Measurement in Education, American Psychological Association, Washington, DC.
3. Angoff W.H. Validity: An evolving concept. Test validity 1988, 9-13. Lawrence Erlbaum, Hillsdale, NJ. H. Wainer, H. Braun (Eds.).
4. A framework for qualifications of the European Higher Education Area 2005, Bologna Working Group on Qualifications Frameworks, Ministry of Science, Technology and Innovation, Copenhagen, [Retrieved from ].
5. Borsboom D., Mellenbergh G.J., van Heerden J. The concept of validity. Psychological Review 2004, 111:1061-1071. 10.1037/0033-295X.111.4.1061.
6. Braun E. Das Berliner Evaluationsinstrument für selbsteingeschätzte studentische Kompetenzen (BEvaKomp) 2007, [The Berlin evaluation tool for students' self-assessed competences (BEvaKomp)], Vandenhoeck & Ruprecht unipress, Göttingen, Germany.
7. Braun E., Hannover B. Zum Zusammenhang zwischen Lehr-Orientierung und Lehrgestaltung von Hochschuldozierenden und subjektivem Kompetenzzuwachs bei Studierenden [On the relationship between teacher orientation and teaching skills in university lecturers and subjective increase in competences in students]. Perspektiven der Didaktik [Perspectives on Teaching] (Zeitschrift für Erziehungswissenschaft 2008, 277-291. [Special Issue No. 9], VS Verlag für Sozialwissenschaften, Wiesbaden. M.A. Meyer, M. Prenzel, S. Hellekamps (Eds.).
8. Braun E., Hannover B. Gelegenheiten zum Kompetenzerwerb in der universitären Lehre: Zusammenhänge zwischen den Einschätzungen Studierender und unabhängigen Beobachtungen relevanter Merkmale universitärer Lehrveranstaltungen [Opportunities for skills acquisition in university teaching: Relationships between students' evaluations and independent observations of relevant characteristics of university courses.]. Zeitschrift für Entwicklungspsychologie und Pädagogische Psychologie 2011, 43:22-28. 10.1026/0049-8637/a000029.
9. Braun E., Leidner B. Academic course evaluation: Theoretical and empirical distinctions between self-rated gain in competences and satisfaction with teaching behavior. European Psychologist 2009, 14:297-306. 10.1027/1016-9040.14.4.297.
10. Cattell R.B. Description and measurement of personality 1946, World Book Company, New York.
11. Centra J. Determining faculty effectiveness: Assessing teaching, research and service for personnel decisions and improvement 1979, Jossey-Bass, San Francisco.
12. Centra J. The development of the Student Instructional Report II 1998, Educational Testing Service, Princeton, NJ, [Retrieved from ].
13. Gaubatz J., Centra N.B. Student perceptions of learning and instructional effectiveness in college courses: A validity study of SIR II 2005, Educational Testing Service, Princeton, NJ, [Retrieved from ].
14. Coates H., Edwards D. Engaging college communities: The impact of residential colleges in Australian higher education (AUSSE Research Briefing No. 4) 2009, Australian Council for Educational Research, Camberwell, Victoria, [Retrieved from ].
15. Cole J.S., Gonyea R.M. Accuracy of self-reported SAT and ACT test scores: Implications for research. Research in Higher Education 2010, 51:305-319. 10.1007/s11162-009-9160-9.
16. Conway M., Ross M. Getting what you want by revising what you had. Journal of Personality and Social Psychology 1984, 47:738-748. 10.1037/0022-3514.47.4.738.
17. Crocker L., Algina J. Introduction to classical and modern test theory 1986, Harcourt Brace, Fort Worth, TX.
18. Cronbach L.J. Coefficient alpha and the internal structure of tests. Psychometrika 1951, 16:297-334. 10.1007/BF02310555.
19. Cronbach L.J. Five perspectives on validity argument. Test validity 1988, 3-17. Lawrence Erlbaum, Hillsdale, NJ. H. Wainer, H. Braun (Eds.).
20. Cronbach L.J., Gleser G.C. Psychological tests and personnel decisions 1965, University of Illinois Press, Urbana, IL.
21. Cronbach L.J., Meehl P.E. Construct validity in psychological tests. Psychological Bulletin 1955, 52:281-302. 10.1037/h0040957.
22. Ebel R.L. Must all tests be valid?. American Psychologist 1961, 16:640-647. 10.1037/h0045478.
23. Edmunds R., Richardson J.T.E. Conceptions of learning, approaches to studying and personal development in UK higher education. British Journal of Educational Psychology 2009, 79:293-309. 10.1348/000709908X368866.
24. Standards and Guidelines for Quality Assurance in the European Higher Education Area 2005, European Association for Quality Assurance in Higher Education, Author, Helsinki, [Retrieved from ].
25. Fan X., Thompson B. Confidence intervals about score reliability coefficients, please: An EPM guidelines editorial. Educational and Psychological Measurement 2001, 61:517-531. 10.1177/0013164401614001.
26. Gonyea R.M., Kish K.A., Kuh G.D., Muthiah R.N., Thomas A.D. College Student Experiences Questionnaire. Norms for the fourth edition 2003, Indiana University Center for Postsecondary Research, Policy and Planning, Bloomington, IN, [Retrieved from ].
27. Gosling S.D., John O.P., Craik K.H., Robins R.W. Do people know how they behave? Self-reported act frequencies compared with on-line codings by observers. Journal of Personality and Social Psychology 1998, 74:1337-1349. 10.1037/0022-3514.74.5.1337.
28. Greenwald A.G., Gillmore G.M. How useful are student ratings?. American Psychologist 1998, 53:1228-1229. 10.1037/0003-066X.53.11.1228.
29. Guion R. Content validity: The source of my discontent. Applied Psychological Measurement 1977, 1:1-10. 10.1177/014662167700100103.
30. Hannover B. Das kontextabhängige Selbst oder warum sich unser Selbst mit dem sozialen Kontext verändert [The context-dependent self, or why our self changes with the social context]. Psychologie des Selbst [Psychology of the self] 2000, 227-238. Psychologie-Verlags-Union, Weinheim. W. Greve (Ed.).
31. Hu S., Kuh G.D. Being (dis)engaged in educationally purposeful activities: The influences of student and institutional characteristics. Research in Higher Education 2002, 43:555-575. 10.1023/A:1020114231387.
32. Joinson A. Self-disclosure in computer-mediated communication: The role of self-awareness and visual anonymity. European Journal of Social Psychology 2001, 31:177-192. 10.1002/ejsp. 36.
33. Kalton G., Collins M., Brook L. Experiments in wording opinion questions. Journal of the Royal Statistical Society (Series C) 1978, 27:149-161. [Retrieved from ].
34. Kane M.T. Current concerns in validity theory. Journal of Educational Measurement 2001, 38:319-342. 10.1111/j.1745-3984.2001.tb01130.x.
35. Kelley T.L. Interpretation of educational measurements 1927, Macmillan, New York.
36. Krosnick J. Survey research. Annual Review of Psychology 1999, 50:537-567. 10.1146/annurev.psych.50.1.537.
37. Kuh G. The National Survey of Student Engagement: Conceptual framework and overview of psychometric properties 2003, Indiana University Center for Postsecondary Research, Bloomington, IN, [Retrieved from ].
38. Kuh G.D. The National Survey of Student Engagement: Conceptual and empirical foundations. New Directions for Institutional Research 2009, 141:5-20. 10.1002/ir.283.
39. Kuh G.D., Cruce T.M., Shoup R., Kinzie J., Gonyea R.M. Unmasking the effects of student engagement on first-year college grades and persistence. Journal of Higher Education 2008, 79:540-563. [Retrieved from ].
40. Lawless C., Richardson J.T.E. Monitoring the experiences of graduates in distance education. Studies in Higher Education 2004, 29:353-374. 10.1080/03075070410001682628.
41. Liu A., Sharkness J., Pryor J.H. Findings from the 2007 administration of Your First College Year (YFCY): National aggregates 2008, University of California, Higher Education Research Institute, Los Angeles, CA, [Retrieved from ].
42. Lord F., Novick M. Statistical theories of mental test scores 1968, Addison-Wesley, Reading, MA.
43. Lucas R., Baird B. Global self assessment. Handbook of psychological measurement: A multimethod perspective 2006, 29-42. American Psychological Association, Washington, DC. M. Eid, E. Diener (Eds.).
44. Mabe P.A., West S.G. Validity of self-evaluation of ability: A review and meta-analysis. Journal of Applied Psychology 1982, 67:280-296. 10.1037/0021-9010.67.3.280.
45. Marsh H.W., Roche L.A. Making students' evaluations of teaching effectiveness effective: The critical issues of validity, bias, and utility. American Psychologist 1997, 52:1187-1197. 10.1037/0003-066X.52.11.1187.
46. Martin E. The effectiveness of different models of work-based university education 1996, Department of Employment, Education, Training and Youth Affairs, Adelaide, [Retrieved from ].
47. Masten A., Coatsworth J.D. The development of competence in favorable and unfavorable environments: Lessons from research on successful children. American Psychologist 1998, 53:205-220. 10.1037/0003-066X.53.2.205.
48. Messick S. Validity. Educational measurement 1989, 13-103. American Council on Education and Macmillan, New York. 3rd ed. R.L. Linn (Ed.).
49. Meyer G.J., Finn S.E., Eyde L.D., Kay G.G., Moreland K.L., Dies R.R., Eisman E.J., Kubiszyn T.W., Read G.M. Psychological testing and psychological assessment: A review of evidence and issues. American Psychologist 2001, 56:128-165. 10.1037/0003-066X.56.2.128.
50. Noftle E., Fleeson W. Age differences in Big Five behavior averages and variabilities across the adult life span: Moving beyond retrospective, global summary accounts of personality. Psychology and Aging 2010, 25:95-107. 10.1037/a0018199.
51. Norenzayan A., Schwarz N. Telling what they want to know: Participants tailor causal attributions to researchers' interests. European Journal of Social Psychology 1999, 29:1011-1020. 10.1002/(SICI)1099-0992(199912)29:8<1011::AID-EJSP974>3.0.CO;2-A.
52. The definition and selection of key competencies: Executive summary 2005, Organisation for Economic Cooperation and Development, OECD, Directorate for Education, Paris, [Retrieved from ].
53. Education at a glance 2009: OECD indicators 2009, Organisation for Economic Cooperation and Development, OECD, Directorate for Education, Paris, [Retrieved from ].
54. Pace R., Stern G. An approach to the measurement of psychological characteristics of college environments. Journal of Educational Psychology 1958, 49:269-277. 10.1037/h0047828.
55. Pike G. The convergent and discriminant validity of NSSE scalelet scores. Journal of College Student Development 2006, 47:551-564. 10.1353/csd.2006.0061.
56. Problem solving for tomorrow's world: First measures of cross-curricular competencies from PISA 2003 2005, Programme for International Student Assessment, Organisation for Economic Cooperation and Development, Paris, [Retrieved from ].
57. Prosser M., Trigwell K. Confirmatory factor analysis of the approaches to teaching inventory. British Journal of Educational Psychology 2006, 76:405-419. 10.1348/000709905X43571.
58. Ramsden P. A performance indicator of teaching quality in higher education: the Course Experience Questionnaire. Studies in Higher Education 1991, 16:129-150. 10.1080/03075079112331382944.
59. Richardson J.T.E. Researching student learning: approaches to studying in campus-based and distance education 2000, SRHE & Open University Press, Buckingham, UK.
60. Richardson J.T.E. Methodological issues in questionnaire-based research on student learning in higher education. Educational Psychology Review 2004, 16:347-358. 10.1007/s10648-004-0004-z.
61. Roche L.A., Marsh H.W. Workload, grades, and students' evaluations of teaching: Clear understanding sometimes requires more patient explanations. American Psychologist 1998, 53:1230-1231. 10.1037/0003-066X.53.11.1230.
62. Ross M. The relation of implicit theories to the construction of personal histories. Psychological Review 1989, 96:341-357. 10.1037/0033-295X.96.2.341.
63. Rychen D.S., Salganik L. Key competencies for a successful life and a well-functioning society 2003, Hogrefe & Huber Publishers, Cambridge, MA.
64. Schuman H., Presser S. Questions and answers in attitude surveys 1981, Academic Press, New York.
65. Schwarz N. Self-reports: How the questions shape the answers. American Psychologist 1999, 54:93-105. 10.1037/0003-066X.54.2.93.
66. Spain J.S., Eaton L.G., Funder D.C. Perspective on personality: The relative accuracy of self versus others for the prediction of emotion and behavior. Journal of Personality 2000, 68:837-867. 10.1111/1467-6494.00118.
67. Spencer S.J., Steele C.M., Quinn D.M. Stereotype threat and women math performance. Journal of Experimental Social Psychology 1999, 35:4-28. 10.1006/jesp. 1998.1373.
68. Steele C.M., Aronson J. Stereotype threat and the intellectual test performance of African Americans. Journal of Personality and Social Psychology 1995, 69:797-811. 10.1037/0022-3514.69.5.797.
69. Strack F., Schwarz N. Communicative influences in standardized question situations: The case of implicit collaboration. Language and social cognition 1992, 173-193. Cambridge University Press, Cambridge, UK. K. Fiedler, G. Semin (Eds.).
70. Weinert F.E. Concept of competence. A conceptual clarification. Defining and selecting key competences 2001, 17-31. Hogrefe, Göttingen, Germany. D. Rychen, L. Salganik (Eds.).
71. Wilson K., Lizzio A., Ramsden P. The development, validation and application of the Course Experience Questionnaire. Studies in Higher Education 1997, 22:33-53. 10.1080/03075079712331381121.