Contextualizing the interpretation of reliability data

Educational Measurement: Issues and Practice

Volumn 17, Issue 3, 1998, Pages 24-36

  Nichols, Paul D ^a   Smith, Philip L ^a  

^a UNIVERSITY OF WISCONSIN MILWAUKEE   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 52849136287     PISSN: 07311745     EISSN: 17453992     Source Type: Journal    
DOI: 10.1111/j.1745-3992.1998.tb00625.x     Document Type: Article

References (49)

1. American College Testing. (1988). ACT Assessment Program technical manual. Iowa City, IA Author.
2. American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (1985). Standards for educational and psychological testing. Washington, DC: Author.
3. Bereiter, C., & Scardamalia, M. (1987). The psychology of written composition. Hillsdale, NJ: Erlbaum.
4. Bereiter, C., Scardamalia, M., & Turkish, L. (1980, April). The child as discourse grammarian. Paper presented at the Annual Meeting of the American Educational Research Association, Boston.
5. Brennan, R. L. (1992). Elements of generalizability theory (2nd ed.). Iowa City, IA American College Testing.
6. Brennan, R. L., & Johnson, E. G. (1995). Generalizability of performance assessments. Educational Measurement: Issues and Practice, 14(4), 9–12, 27.
7. Cannell, C. F., Miller, P. V., & Oksenberg, L. (1981). Research on interviewing techniques. In S. Leinhart (Ed.), Sociological methodology (pp. 389–437). San Francisco: Jossey Bass.
8. Crocker, L. M., & Algina, J. (1986). Introduction to classical and modern test theory. New York: Holt, Rinehart, & Winston.
9. Cronbach, L. J. (1988). Five perspectives on validation argument. In H. Wainer & H. Braun (Eds.), Test validity (pp. 3–17). Hillsdale, NJ: Erlbaum.
10. Cronbach, L. J. (1989). Construct validation after thirty years. In R. L. Linn (Ed.), Intelligence, measurement theory and public policy (pp. 147–171 Chicago: University of Illinois Press).
11. Cronbach, L. J., Gleser, G. C., Nanda, H., & Rajaratnam, N. (1972). The dependability of behavioral measurements: Theory of generalizability for scores and profiles. New York: Wiley.
12. Dunbar, S. B., Koretz, D. M., & Hoover, H. D. (1991). Quality control in the development and use of performance assessments. Applied Measurement in Education, 4, 289–303.
13. Educational Testing Service & National Center for Education Statistics. (1994). NAEP 1992 technical report. Washington, DC: U.S. Department of Education, Office of Educational Research and Improvement.
14. Feldt, L. S., & Brennan, R. L. (1989). Reliability. In R. L. Linn (Ed.), Educational measurement (3rd ed., pp. 105–146). New York: Macmillan.
15. Flower, L. (1994). The construction of negotiated meaning: A social cognitive theory of writing. Carbondale, IL: Southern Illinois University Press.
16. Frederiksen, J. R., & Collins, A. (1989). A systems approach to educational testing. Educational Researcher, 18(9), 27–32.
17. Goldstein, H. (1994). Recontextualizing mental measurement. Educational Measurement: Issues and Practice, 13(1), 16–19.
18. Graesser, A. C., & Murachver, T. (1985). Symbolic procedures of question answering. In A. C. Graesser & J. B. Black (Eds.), The psychology of questions (pp. 15–88). Hillsdale, NJ Erlbaum.
19. Hermann, J. L. (1991). Research in cognition and learning: Implications for achievement testing practice. In M. C. Wittrock & E. L. Baker (Eds.), Testing and cognition (pp. 154–166). Englewood Cliffs, NJ: Prentice-Hall.
20. Hermann, J. L., & Golan, S. (1993). The effects of standardized testing on teaching and schools. Educational Measurement: Issues and Practice. 12(4), 20–25.
21. Hieronymus, A. N., & Hoover, H. D. (1987). Iowa tests of basic skills: Writing supplement teacher’s guide. Chicago: Riverside.
22. Lane, S., Liu, M., Ankenmann, R. D., & Stone, C. A. (1996). Generalizability and validity of a mathematics performance assessment. Journal of Educational Measurement, 33, 71–92.
23. Linn, R. L., Baker, E. L., & Dunbar, S. B. (1991). Complex, performance-based assessment: Expectations and validation criteria. Educational Researcher, 20(8), 5–21.
24. Marshall, S. P. (1993). Assessing schema knowledge. In N. Fredriksen, R. J. Mislevy, & I. Bejar (Eds.), Test theory for a new generation of tests (pp. 155–180). Hillsdale, NJ: Erlbaum.
25. Messick, S. (1989). Validity. In R. L. Linn (Ed.), Educational measurement (3rd ed., pp. 13–103). Washington, DC: American Council on Education & National Council on Measurement in Education.
26. Messick, S. (1994). The interplay of evidence and consequences in the validation of performance assessments. Educational Researcher, 23(2), 13–23.
27. Messick, S. (1995). Validity of psychological assessment: Validation of inferences from persons’ responses and performances as scientific inquiry into score meaning. American Psychologist, 50, 741–749.
28. Mislevy, R. J. (1995). Probability-based inference in cognitive diagnosis. In P. D. Nichols, S. F. Chipman, & R. L. Brennan (Eds.), Cognitively diagnostic assessment (pp. 43–72). Hillsdale, NJ Erlbaum.
29. Moss, P. A. (1994). Can there be validity without reliability? Educational Researcher, 23(2), 5–12.
30. Moss, P. A. (1995). Themes and variations in validity theory. Educational Measurement: Issues and Practice, 14(2), 5–13.
31. National Council of Teachers of Mathematics. (1989). Curriculum and evaluation standards for school mathematics. Reston, VA Author.
32. National Research Council. (1996). National science education standards Washington, DC: National Academy Press.
33. Nichols, P. D. (1994). A framework for developing cognitively diagnostic assessments. Review of Educational Research, 64, 575–603.
34. Nichols, P. D., & Hatfield, J. (1998). Evaluation of measurement precision within a construct-centered framework. Paper submitted for publication.
35. Nichols, P. D., & Kuehl, B. J. (in press). Prophesying the reliability of cognitively complex assessments. Applied Measurement in Education.
36. Nichols, P. D., & Smith, P. L. (1996, April). Error is in the eye of the beholder. Paper presented at the Annual Meeting of the National Council on Measurement in Education, New York.
37. Nichols, P. D., & Sugrue, B. (1997, August). NAEP redesign: Short forms and construct-centered test development. Paper presented at the National Center for Education Statistics, Washington, DC.
38. Nichols, P. D., & Sugrue, B. (1998). The lack of fidelity between cognitively complex constructs and conventional test development practice. Manuscript submitted for publication.
39. Otter, M. E., Mellenbergh, G. J., & de Glopper, K. (1995). The relation between information-processing variables and test-retest stability for questionnaire items. Journal of Educational Measurement, 32, 199–216.
40. Romberg, T. A., & Wilson, L. D. (1995). In T. A. Romberg (Ed.), Mathematics assessment and evaluation: Imperatives for mathematics educators (pp. 12–28). Albany, NY. State University of New York Press.
41. Ruiz-Primo, M. A., Baxter, G. P., & Shavelson, R. J. (1993). On the stability of performance assessments. Journal of Educational Measurement, 30, 41–53.
42. Shavelson, R. J., Baxter, G. P., & Gao, X. (1993). Sampling variability of performance assessments. Journal of Educational Measurement, 4.
43. Shavelson, R. J., Gao, X., & Baxter, G. P. (1996). On the content validity of performance assessments: Centrality of domain specifications. In M. Birenbaum & F. J. R. C. Dochy (Eds.), Alternatives in assessment of achievements, learning processes, and prior knowledge (pp. 131–142). Norwell, MA: Kluwer Academic Publishers.
44. Shavelson, R. J., & Webb, N. M. (1991). Generalizability theory: A primer. Newbury Park, CA: Sage Publications.
45. Siegler, R. S. (1994). Cognitive variability: A key to understanding cognitive development. Current Directions in Psychological Science, 3(1), 1–5.
46. Tourangeau, R., & Rasinski, K. A. (1988). Cognitive processes underlying context effects in attitude measurement. Psychologica1 Bulletin, 103, 299–314.
47. Traub, R. E. (1994). Reliability for the social sciences: Theory and applications. Thousands Oaks, CA Sage I Publications.
48. Wiggins, G. P. (1993). Assessing student performance: Exploring the purpose and limits of testing. San Francisco: Jossey-Bass.
49. Wolfe, E. W. (1995). A study of expertise in essay scoring. Unpublished doctoral dissertation, University of California, Berkley.