Assessment issues in the testing of children at school entry

Future of Children

Volumn 15, Issue 1, 2005, Pages 15-34

  Rock, Donald A ^a   Stenner, A Jackson ^b  

^a EDUCATIONAL TESTING SERVICE   (United States)

^b MetaMetrics Inc   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EDUCATION; ETHNIC DIFFERENCE; HUMAN; HYPOTHESIS; INCOME; INTELLIGENCE TEST; KINDERGARTEN; LONGITUDINAL STUDY; RACE DIFFERENCE; REVIEW; SCHOOL; SCORING SYSTEM;

BIAS (EPIDEMIOLOGY); CHILD BEHAVIOR; CHILD, PRESCHOOL; CONTINENTAL POPULATION GROUPS; ETHNIC GROUPS; HUMANS; INTELLIGENCE TESTS; LONGITUDINAL STUDIES; NEEDS ASSESSMENT; SCHOOL ADMISSION CRITERIA; SCHOOLS; STUDENTS;

EID: 28044471577     PISSN: 10548289     EISSN: 15501558     Source Type: Journal    
DOI: 10.1353/foc.2005.0009     Document Type: Review

References (44)

1. Lori Shepherd, Sharon Lynn Kagan, and Emily Wurtz, eds., Principles and Recommendations for Early Childhood Assessments (Washington: National Education Goals Panel, February 1998).
2. Samuel J. Meisels, "Can Head Start Pass the Test?" Education Week 22, no. 27 (March 19, 2003): 44;
3. Anthony D. Pellegrini and Carl D. Glickman, "Measuring Kindergarteners' Social Competence," Young Children (May 1990): 40-44.
4. Sally Atkins-Burnett, Brian Rowan, and Richard Correnti, "Administering Standardized Achievement Tests to Young Children: How Mode of Administration Affects the Reliability of Standardized Measures of Student Achievement in Kindergarten and First Grade," paper presented at the annual meeting of the American Educational Research Association, April 2001 (available at www.sii.soe.umich.edu/papers.html).
5. Although the studies reviewed in this issue use the PPVT-R, the test has recently been revised, and studies now in the field use the PPVT-III. For discussion, see Jeanne Brooks-Gunn, Pamela K. Klebanov, Judith Smith, Greg J. Duncan, and Kyunghee Lee, "The Black-White Test Score Gap in Young Children: Contributions of Test and Family," Applied Developmental Science 7, no. 4 (2003): 239-52.
6. A. Jackson Stenner, Malbert Smith, and Donald S. Burdick, "Toward a Theory of Construct Definition," Journal of Educational Measurement 20, no. 4 (1983): 304-15.
7. George A. Miller and Patricia M. Gildea, "How Children Learn Words," Scientific American 257, no. 3 (1987): 94-99.
8. Elizabeth P. Hagen, Elizabeth A. Delaney and Thomas F. Hopkins, Stanford-Binet Intelligence Scale-Examiner's Handbook: An Expanded Guide for Fourth Edition Users (Chicago: Riverside Publishing Company, 1987).
9. For the genesis of the Behavioral Problems Index, see James L. Peterson and Nicholas Zill, "Marital Disruption, Parent-Child Relationships, and Behavioral Problems in Children," Journal of Marriage and the Family 48, no. 2 (May 1986)
10. For a discussion of how the BPI is used in the NLS, see Center for Human Resource Research, NLSY79 Child and Young Adult Data Users Guide (Ohio State University, December 2002), especially pp. 91-94.
11. The article by Jeanne Brooks-Gunn and Lisa Markman in this issue describes in more detail how this approach was used in one study of 2,000 three-year-olds, with data from the Early Head Start Research and Evaluation Project (EHS). These data are also discussed further at the end of the present article.
12. National Center for Education Statistics, U.S. Dept. of Education, ECLS-K Base Year Data Files and Electronic Codebook (2001).
13. Susan M. Benner, Assessing Young Children with Special Needs: An Ecological Perspective (New York: Longman, 1992);
14. Everett Waters and Alan L. Sroufe, "Social Competence as a Developmental Construct," Developmental Review 3 (1983): 79-97;
15. Anthony Pellegrini, Lee Galda, and Donald L. Rubin, "Context in Text: The Development of Oral and Written Language in Two Genres," Child Development 55 (1984): 1549-55.
16. Frederick M. Lord and Melvin R. Novick, Statistical Theories of Mental Test Scores, with Contributions by Alan Birnbaum (Reading, Mass.: Addison-Wesley 1968);
17. Frederick M. Lord, Applications of Item Response Theory to Practical Testing Problems (Hillsdale, N.J.: Lawrence Erlbaum Associates, 1980).
18. See also Benner, Assessing Young Children (see note 11).
19. Donald T. Campbell and Donald W. Fiske, "Convergent and Discriminant Validation by the Multi-Trait Multi-Method Matrix," Psychological Bulletin 56 (1959): 81-105.
20. Pellegrini, Galda, and Rubin, "Context in Text" (see note 11).
21. A. Jackson Stenner and Benjamin D. Wright, "Readability, Reading Ability, and Comprehension" (paper presented at the Association of Test Publishers Hall of Fame induction for Benjamin D. Wright, San Diego, 2002), in Making Measures, edited by Benjamin D. Wright and Mark H. Stone (Chicago: Phaneron Press, 2004).
22. The mathematical formula for calculating standard deviation works like this: (1) calculate the average of the scores; (2) calculate the difference between each individual score and the average; (3) square these differences from the average, and then add them up; (4) take the square root of the total. This calculation will give the number of points that are equal to 1 standard deviation for this group of scores.
23. Space does not permit a full treatment of the soundness of all assumptions underlying the standard deviation as a common unit of effect. However, we did compare the standard deviations for five well-known reading tests that were linked to a common scale and found they ranged from a low of .94 to a high of 1.13. This modest variability across grades and tests provides a context for evaluating the variability in estimates of the black-white achievement gap across various studies and instruments reported in this volume.
24. For purposes of this discussion we made the usual simplifying assumptions of bivariate normality, homogeneity of variance, and equal sample sizes in the two groups. Furthermore, we assume that the 1 standard deviation difference is in construct measures, not test score performances, which are uncorrected for measurement error.
25. Using data from the NCES-NAEP website, we estimate that 1.0 standard deviation on NAEP is equivalent to 220L (220 Lexiles). A back check on this number is to average four norm-referenced achievement test (NRT) standard deviations. The RMSA standard deviation for the four NRTs is 229L. Comprehension rate is modeled as the difference between reader ability and text readability. A difference of 225L between a targeted reader (75 percent comprehension rate) at fourth grade and the average black fourth grader implies a 53 percent comprehension rate for a "book bag" of fourth grade textbooks. See Lexile.com, the Lexile Calculator.
26. For background on construct validity, see Stenner, Smith, and Burdick, "Toward a Theory of Construct Definition" (see note 5).
27. For discussion of the evidence, see Richard E. Nisbett, "Race, Genetics, and IQ," in The Black-White Test Score Gap, edited by Christopher Jencks and Meredith Phillips (Brookings, 1998).
28. The psychometric literature has largely given up on the term "bias" in favor of the less emotionally charged terms "differential item functioning" and "differential instrument functioning."
29. See Thomas J. Kane, "Racial and Ethnic Preferences in College Admissions,"
30. Frederick E. Vars and William G. Bowen, "Scholastic Aptitude Test Scores, Race, and Academic Performance in Selective Colleges and Universities,"
31. and William R. Johnson and Derek Neal, "Basic Skills and the Black-White Earning Gap," in The Black-White Test Score Gap, edited by Jencks and Phillips (see note 20).
32. Claude M. Steele, "Race and the Schooling of Black America," Atlantic Monthly (April 1992): 68-78;
33. Claude M. Steele, "A Threat in the Air: How Stereotypes Shape the Intellectual Identities and Performance of Women and African Americans," American Psychologist (June 1997): 613-29;
34. Claude M. Steele and Joshua Aronson, "Stereotype Threat and the Intellectual Test Performance of African Americans," Journal of Personality and Social Psychology 69, no. 5 (1995): 797-811.
35. Arthur R. Jensen, Bias in Mental Testing (New York: Free Press, 1980), especially p. 146-147;
36. Arthur R. Jensen, "Spearman's Hypothesis Tested with Chronometric Information-Processing Tasks," Intelligence 17 (1993): 47-77.
37. Arthur R. Jensen, "Psychometric g and Achievement," in Policy and Perspectives on Educational Testing, edited by Bernard R. Gifford (Boston: Kluwer Academic, 1993), pp. 117-227.
38. Betty Hart and Todd R. Risley, The Social World of Children Learning to Talk (Baltimore: Brooks, 1999).
39. See Stenner, Smith, and Burdick, "Toward a Theory of Construct Definition" (see note 5),
40. and Miller and Gildea, "How Children Learn Words" (see note 6),
41. for introductions to exposure theory. See Jensen, Bias in Mental Testing (see note 23) for an introduction to education theory.
42. Meredith Phillips, James Crouse, and John Ralph, "Does the Black-White Test Score Gap Widen after Children Enter School?" in Jencks and Phillips, The Black-White Test Score Gap (see note 20), pp. 229-71.
43. Jensen, Bias in Mental Testing (see note 23).
44. See also William Hick, "On the Rate of Information," Quarterly Journal of Experimental Psychology 4 (1952): 11-26.