Alternative interpretations of alternative assessments: Some validity issues in educational performance assessments

Educational Measurement: Issues and Practice

Volumn 21, Issue 3, 2002, Pages 5-18

  Bachman, Lyle F ^a  

^a UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

Alternative assessment; Language testing; Performance assessment; Validity

Indexed keywords

EID: 84986196685     PISSN: 07311745     EISSN: 17453992     Source Type: Journal    
DOI: 10.1111/j.1745-3992.2002.tb00095.x     Document Type: Article

References (90)

1. Alderson, J. C. (1981a) Reaction to the Morrow paper. In J. C. Alderson & A. Hughes, (Eds.), Issues in language testing. ELT Documents 111 (pp. 45–54). London: The British Council.
2. Alderson, J. C. (1981b.) Report of the discussion on communicative language testing. In J. C. Alderson & A. Hughes, (Eds.), Issues in language testing. ELT Documents 111. London: British Council.
3. Alderson, J. C. (1983) Response to Harrison: Who needs jam? In A. Hughes & D. Porter, (Eds.), Current developments in language testing (pp. 87–92). London: Academic Press.
4. Alderson, J. C. (2000) Assessing reading ability. Cambridge, UK: Cambridge University Press.
5. Alderson, J. C., &Urquhart, A. H. (1985) The effect of students' academic discipline on their performance on ESP reading tests. Language Testing, 2(2), 192–204.
6. Anderson, N., Bachman, L. F., Cohen, A. D., &Perkins, K. (1991) An exploratory study into the construct validity of a reading comprehension test: Triangulation of data sources. Language Testing, 8(1), 41–66.
7. Archibald, D. A., &Newman, F. M. (1988) Beyond standardized testing: Assessing authentic academic achievement in the secondary school. Reston, VA: National Association of Secondary School Principals.
8. Arnold, J. (Ed.). (1999) Affect in language learning: Cambridge, UK: University Press.
9. Bachman, L. F. (1990) Fundamental considerations in language testing. Oxford, UK: Oxford University Press.
10. Bachman, L. F., Davidson, F., &Milanovic, M. (1996) The use of test methods in the content analysis and design of EFL proficiency tests. Language Testing, 13(2), 125–150.
11. Bachman, L. F., Lynch, B. K., &Mason, M. (1995) Investigating variability in tasks and rater judgments in a performance test of foreign language speaking. Language Testing, 12, 238–257.
12. Bachman, L. F., Lynch, B. K., Mason, M., &Egbert, M. (1992) Education abroad language ability assessment system: Interim, report. Los Angeles, CA: University of California, Los Angeles.
13. Bachman, L. F., &Palmer, A. S. (1996) Language testing in practice: Designing and developing useful language tests. Oxford, UK: Oxford University Press.
14. Baker, E. L. (1992) The role of domain specifications in improving the technical quality of performance assessments. Los Angeles, CA: Center for Research on Evaluation, Standards, and Student Testing.
15. Baker, E. L., O'Neil, H. F., &Linn, R. L. (1993) Policy and validity prospects for performance-based assessment. American Psychologist, 48(12), 1210–1218.
16. Bennett, R. E. (1993) On the meanings of constructed response. In R. E. Bennett & W. C. Ward, Jr., (Eds.), Construction versus choice in cognitive measurement: Issues in constructed response, performance testing and portfolio assessment (pp. 1–27). Hillsdale, NJ: Lawrence Erlbaum.
17. Brennan, R. L., &Johnson, E. G. (1995) Generalizability of performance assessments. Educational Measurement: Issues and Practice, 14(4), 9–12.
18. Burger, S. E., &Burger, D. L. (1994) Determining the validity of performance-based assessment. Educational Measurement: Issues and Practice, 13(1), 9–15.
19. Canale, M. (1983) On some dimensions of language proficiency. In J. W. Oller, (Ed.), issues in language testing research. Rowley, MA Newbury House.
20. Canale, M., &Swain, M. (1981) Theoretical bases of communicative approaches to second language teaching and testing. Applied Linguistics, 1(1), 1–47.
21. Carrell, P. L. (1983) Three components of background knowledge in reading comprehension. Language Learning, 33, 183–207.
22. Carroll, J. B. (1961) Fundamental considerations in language testing, Testing the English proficiency of foreign students (pp. 30–40). Washington, DC: Center for Applied Linguistics.
23. Chapelle, C. (1998) Construct definition and validity inquiry in SLA research. In L. F. Bachman & A. D. Cohen, (Eds.), Interfaces between second language acquisition and language testing research (pp. 32–70). New York: Cambridge University Press.
24. Clapham, C. (1996) The development of IELTS: A study of the effect of background knowledge on reading comprehension. (Vol. 4). Cambridge, UK: University of Cambridge Local Examinations Syndicate/Cambridge University Press.
25. Clark, J. L. D. (1972) Foreign language testing: Theory and practice. Philadelphia, PA: Center for Curriculum Development.
26. Clark, J. L. D. (1975) Theoretical and technical considerations in oral proficiency testing. In R. L. Jones & B. Spolsky, (Eds.), Testing language proficiency (pp. 10–24). Arlington, VA: Center for Applied Linguistics.
27. Clark, J. L. D. (1978) Interview testing research at Educational Testing Service. In J. L. D. Clark, (Ed.), Direct testing of speaking proficiency: Theory and application (pp. 211–228). Princeton, NJ: Educational Testing Service.
28. Cooper, R. L. (1968) An elaborated language testing model. In J. A. Upshur & J. Fata, (Eds.), Problems in foreign language testing: Language Learning Special Issue Number 3 (pp. 57–65). Ann Arbor, MI: Research Club in Language Learning.
29. Darling-Hammond, L., Ancess, J., &Falk, B. (1995) Authentic assessment in an era of restructuring. Alexandria, VA: Association for Supervision and Curriculum Development.
30. Delandshere, G., &Petrosky, A. R. (1998) Assessment of complex performances: Limitations of key measurement assumptions. Educational Researcher, 27(2), 14–24.
31. Douglas, D. (1998) Testing methods in context-based second language research. In L. F. Bachman & A. D. Cohen, (Eds.), Interfaces between second language acquisition and language testing research. New York: Cambridge University Press.
32. Douglas, D. (2000) Assessing language for specific purposes: Theory and practice. Cambridge, UK: Cambridge University Press.
33. Douglas, D., &Selinker, L. (1985) Principles for language tests within the ‘discourse domains’ theory of inter-language: Research, test construction and interpretation. Language Testing, 2, 205–226.
34. Dunbar, S. B., Koretz, D., &Hoover, H. D. (1991) Quality control in the development and use of performance assessments. Applied Measurement in Education, 4, 289–304.
35. Embretson, S. (1983) Construct validity: Construct representation versus nomothetic span. Psychological Bulletin, 93, 179–197.
36. Freedle, R., &Kostin, I. (1993) The prediction of TOEFL reading item difficulty: implications for construct validity. Language Testing, 10, 133–170.
37. Freedle, R., &Kostin, I. (1999) Does the text matter in a multiple-choice test of comprehension? The case for the construct validity of TOEFL's minitalks. Language Testing, 16(1), 2–32.
38. Gardner, R. C. (1985) Social psychology and second language learning: The role of attitudes and motivation. London: Edward Arnold.
39. Gardner, R. C., &Lambert, W. E. (1972) Attitudes and motivation in second language learning. Rowley, MA: Newbury House Publishers.
40. Green, B. F. (1995) Comparability of scores from performance assessments. Educational Measurement: Issues and Practice, 14(4), 13–15.
41. Herman, J. L., Aschbacher, P. R., &Winters, L. (1992) A practical guide to alternative assessment. Alexandria, VA: Association for Supervision and Curriculum Development.
42. Jacoby, S., &McNamara, T. (1999) Locating competence. English for Specific Purposes, 18(3), 213–241.
43. Johnson, P. (1981) Effects on reading comprehension of language complexity and cultural background of a text. TESOL Quarterly, 16, 503–516.
44. Jones, R. L. (1979) Performance testing of second language proficiency. In E. J. Briere & F. R. Hinofotis, (Eds.), Concepts in language testing: Some recent studies (pp. 50–57). Washington, DC: TESOL.
45. Jones, R. L. (1985) Second language performance testing: An overview. In P. C. Hauptman, R. LeBlanc, &M. B. Wesche, (Eds.), Second language performance testing (pp. 15–24). University of Ottawa Press.
46. Kane, M. (1982) A sampling model for validity. Applied Psychological Measurement, 6, 267–292.
47. Kane, M., Crooks, T., &Cohen, A. (1999) Validating measures of performance. Educational Measurement: Issues and practice, 18(2), 5–17.
48. Lado, R. (1961) Language Testing. New York: McGraw-Hill.
49. Leventhal, H., &Scherer, K. (1987) The relationship of emotion and cognition: A functional approach to a semantic controversy. Cognition and emotion, 1(1), 3–28.
50. Lewkowicz, J. A. (1997) Authentic for whom? Does authenticity really matter? In A. Huhta, V. Kohonen, &S. Luoma, (Eds.) Current developments and alternatives in language assessment. Jyvaskyla: University of Jyvaskyla.
51. Lewkowicz, J. A. (2000) Authenticity in language testing: Some outstanding questions. Language Testing, 17(1), 43–64.
52. Linn, R. L. (1994) Performance assessment: Policy promises and technical measurement standards. Educational Researcher, 23(9), 4–14.
53. Linn, R. L., Baker, E. L., &Dunbar, S. B. (1991) Complex, performance-based assessment: Expectations and validation criteria. Educational Researcher, 20(8), 15–21.
54. Linn, R. L., &Burton, E. (1994) Performance-based assessment: Implications of task specificity. Educational Measurement: Issues and Practice, 13(1), 5–8, 15.
55. Lynch, B. K., &McNamara, T. F. (1998) Using g-theory and many-facet Rasch measurement, in the development of performance assessments of the ESL speaking skills of immigrants. Language Testing, 15(2), 158–180.
56. Marcoulides, G. A. (1996) Estimating variance components in generalizability theory: The covariance structure analysis approach. Structural Equation Modeling, 3, 290–299.
57. Marcoulides, G. A., &Drezner, Z. (1997) A method for analyzing performance assessments. In M. Wilson, K. Draney, &G. Engelhard, (Eds.), Objective measurement: Theory into practice (pp. 261–277). Norwood, NJ: Ablex.
58. McNamara, T. F. (1996) Measuring second language performance. London: Longman.
59. Messick, S. (1989) Validity. In R. L. Linn, (Ed.), Educational Measurement (3rd ed., pp. 13–103). New York: American Council on Education, Macmillan Publishing Company.
60. Messick, S. (1994a, October 2526.) Alternative modes of assessment, uniform standards of validity. Paper presented at the Conference on Evaluating Alternatives to Traditional Testing for Selection, Bowling Green State University, October 25–26.
61. Messick, S. (1994b.) The interplay of evidence and consequences in the validation of performance assessments. Educational Researcher, 23(2), 13–23.
62. Messick, S. (1995) Standards of validity and the validity of standards in performance assessment. Educational Measurement: Issues and Practice, 14(4) 5–8.
63. Messick, S. (1996) Validity and washback in language testing. Language Testing, 13(3), 241–256.
64. Mislevy, R. J., Steinberg, L. S., &Almond, R. G. (2001) On the structure of educational assessments. College Park, MD: University of Maryland.
65. Mislevy, R. J., Steinberg, L. S., &Almond, R. G. (2002) Design and analysis in task-based language assessment. Language Testing.
66. Mislevy, R. J., Steinberg, L. S., Breyer, F. J., Almond, R. G., &Johnson, L. (in press). Making sense of data from complex assessment. Applied Measurement in Education.
67. Morrow, K. (1979) Communicative language testing: Revolution or evolution? In C. J. Brumfit & K. Johnson, (Eds.), The communicative approach to language teaching (pp. 9–25). Oxford, UK: Oxford University Press.
68. Moss, P. A. (1992) Shifting conceptions of validity in educational measurement: Implications for performance assessment. Review of Educalional Research, 62, 229–258.
69. Moss, P. A. (1995) Themes and variations in validity theory. Educational Measurement: Issues and Practice, 14(2), 5–13.
70. Newman, F., Brandt, R., &Wiggins, G. (1998) An exchange of views on “semantics, psychometrics, and assessment reform: a close look at ‘authentic’ assessments”. Educational Researcher, 27(6), 19–22.
71. Newman, F. M., Marks, H., &Gamaron, A. (1996) Authentic pedagogy and student performance. American Journal of Education, 104(4), 280–312.
72. Nichols, P., &Sugrue, B. (1999) The lack of fidelity between cognitively complex constructs and conventional test development practice. Educational Measurement: Issues and Practice, 18(2), 18–29.
73. Oller, J. W. (1979) Language tests at school. London: Longman.
74. Schumann, J. H. (1997) The neurology of affect in language. Malden, MA: Blackwell.
75. Shavelson, R. J., Baxter, G. P., &Pine, J. (1992) Performance assessments: Political rhetoric and measurement reality. Educational Researcher, 21(4), 22–27.
76. Shepard, L. A. (1993) Evaluating test validity. In L. Darling-Hammond, (Ed.), Review of research in education (Vol. 19, pp. 405–450). Washington, DC: American Educational Research Association.
77. Skehan, P. (1984) Issues in the testing of English for specific purposes. Language Testing, 1(2), 202–220.
78. Skehan, P. (1998) A cognitive approach to language learning. Oxford, UK: Oxford University Press.
79. Snow, R. E. (1993) Construct validity and constructed response tests. In R. E. Bennett & R. C. Ward, Jr., (Eds.), Construction versus choice in cognitive measurement: Issues in constructed response, performance testing and portfolio assessment (pp. 45–60). Hillsdale, NJ: Lawrence Erlbaum.
80. Solano-Flores, G., &Shavelson, R. (1997) Development of performance assessments in science: Conceptual, practical, and logistic issues. Educational Measurement: Issues and Practice, 16(3), 16–25.
81. Spolsky, B. (1986) A multiple-choice for language testers. Language Testing, 3(2), 31–40.
82. Spolsky, B. (1995) Measured Words. Oxford, UK: Oxford University Press.
83. Stevenson, D. K. (1985) Authenticity, validity and a tea party. Language Testing, 2(1), 41–47.
84. Sugrue, B. (1995) A theory-based framework for assessing domain-specific problem-solving ability. Educational Measurement: Issues and Practice, 14(3), 29–35.
85. Swanson, D. B., Norman, G. R., &Linn, R. L. (1995) Performance-based assessment: Lessons from the health professions. Educational Researcher, 24(5), 5–11.
86. Terwilliger, J. (1997) Semantics, psychometrics and assessment reform: A close look at “authentic” assessments. Educational Researcher, 26(8), 24–27.
87. Upshur, J. A. (1979) Functional proficiency theory and a research role for language tests. In E. Briere & F. B. Hinofotis, (Eds.), Concepts in language testing: Some recent studies (pp. 75–100). Washington, DC: TESOL.
88. Wesche, M. B. (1987) Second language performance testing: The Ontario Test of ESL as an example. Language Testing, 4(1), 28–47.
89. Wiggins, G. (1989) A true test: Toward more authentic and equitable assessment. Phi Delta Kappan, 1989 (May), 703–713.
90. Wiggins, G. (1993) Assessment: Authenticity, context and validity. Phi Delta Kappan, 1993 (November), 200–214.