The bookmark standard-setting method: A literature review

Educational Measurement: Issues and Practice

Volumn 25, Issue 1, 2006, Pages 4-12

  Karantonis, Ana ^a   Sireci, Stephen G ^a,b  

^a UNIVERSITY OF MASSACHUSETTS   (United States)

^b Department of Education ^*  (United States)

Author keywords

Bookmark method; Cut scores; Standard setting; Validity

Indexed keywords

EID: 33644929989     PISSN: 07311745     EISSN: 17453992     Source Type: Journal    
DOI: 10.1111/j.1745-3992.2006.00047.x     Document Type: Review

References (33)

1. Beretvas, N. S. (2004). Comparison of Bookmark difficulty locations under different item response models. Applied Psychology cal Measurement, 28(1), 25-47.
2. Buckendahl, C. W., Smith, R. W., Impara, J. C., & Plake, B. S. (2002). A comparison of Angoff and Bookmark standard setting methods. Journal of Educational Measurement, 39(3), 253-263.
3. Cizek, G. J., Bunch, M. B., & Koons, H. (2004). Setting performance standards: Contemporary methods. Educational Measurement: Issues and Practice, 23(4), 31-50.
4. Dawber, T., Lewis, D. M., & Rogers, W. T. (2002). The cognitive experience of Bookmark standard setting participants. Paper presented at the annual meeting of the American Educational Research Association, New Orleans, LA.
5. Egan, K. L. (2001). Validity and defensibility of cut scores established by the Bookmark Standard Setting Method. Paper presented at the 2001 Council of Chief State School Officers Conference on Large-Scale Assessment, Houston, TX.
6. Ferdous, A. A. (2005, April). Use of an item selection strategy to estimate passing scores in an Angoff standard setting study. Paper presented at the annual meeting of the National Council on Measurement in Education, Montreal, Canada.
7. Ferdous, A. A., & Plake, B. S. (2005). Understanding factors that influence decisions of panelists in a standard setting study. Applied Measurement in Education, 18, 257-267.
8. Ferrara, S., Perie, M., & Johnson, E. (2002, April). Matching the judgmental task with, standard setting panelist expertise: The item-descriptor (ID) matching procedure. Paper presented at the annual meeting of the National Council on Measurement in Education, New Orleans, LA.
9. Green, D. R., Trimble, C. S., & Lewis, D. M. (2003). Interpreting the results of three different standard setting procedures. Educational Measurement: Issues and Practice, 22(1), 22-32.
10. Hambleton, R. K., Jaeger, R. M., Plake, B. S., & Mills, C. N. (2000). Setting performance standards on complex educational assessments. Applied Psychological Measurement, 24, 355-366.
11. Huynh, H. (1998). On score locations of binary and partial credit items and their applications to item mapping and criterion-referenced interpretation. Journal of Educational and Behavioral Statistics, 23, 35-56.
12. Huynh, H. (2000, April). On item mappings and statistical rules for selecting binary items for criterion-referenced interpretation and Bookmark standard settings. Paper presented at the annual meeting of the National Council on Measurement in Education, New Orleans, LA.
13. Kane, M. T. (1994). Validating the performance standards associated with passing scores. Review of Educational Research, 64(3), 425-461.
14. Kane, M. T. (2001). So much remains the same: Conception and status of validation in setting standards. In G. Cizek (Ed.), Setting performance standards: Concepts, methods and perspectives (pp. 53-88). Mahwah, NJ: Erlbaum.
15. Kolstad, A., Cohen, J., Baldi, S., Chan, T., De-Fur, E., & Angeles, J. (1998, May). The response probability convention used in reporting data from IRT assessment scales: Should NCES adopt a standard? Washington, DC: American Institutes for Research.
16. Lee, G., & Lewis, D. M. (2001). A generalizability theory approach toward estimating standard errors of cut scores set using the Bookmark standard setting procedure. Paper presented at the annual meeting of the National Council on Measurement in Education, Seattle, WA.
17. Lewis, D. (2005, April). Comments on Standard Setting Methods: Studying sources of complexity (Symposium on audiotape). Discussant remarks given at the annual meeting of the National Council on Measurement in Education, Montreal, Canada.
18. Lewis, D. M., Green, D. R., Mitzel, H. C., Baum, K., & Patz, R. J. (1998, April). The Bookmark procedure: Methodology and recent implementations. Paper presented at the annual meeting of the National Council on Measurement in Education, San Diego, CA.
19. Lewis, D. M., Mitzel, H. C., & Green, D. R. (1996, June). Standard setting: A Bookmark approach. In D. R. Green (Chair), IRT-based standard setting procedures utilizing behavioral anchoring. Symposium conducted at the Council of Chief State School Officers National Conference on Large-Scale Assessment, Phoenix, AZ.
20. Lewis, D. M., Mitzel, H. C., Green, D. R., & Patz, R. J. (1999). The Bookmark standard setting procedure. Monterey, CA: McGraw-Hill.
21. Mitzel, H. C., Lewis, D. M., Patz, R. J., & Green, D. R. (2001). The Bookmark procedure: Psychological perspectives. In G. Cizek (Ed.), Setting performance standards: Concepts, methods and perspectives (pp. 249-281). Mahwah, NJ: Erlbaum.
22. National Academies of Sciences (2005). Measuring literacy: Performance levels for adults, interim report. Retrieved July 23, 2005 from http://www.nap.edu/books/0309096529/html/.
23. Nellhaus, J. M. (2000). States with NAEP-Like Performance Standards. In M. L. Bourque & S. Byrd. Student performance standards on the National Educational, Assessment of Educational Progress: Affirmation and improvements (pp. 99-130). Washington, DC: National Assessment Governing Board. Retrieved July 23, 2005 from http://www.nagb.org/pubs/student perfstandard.pdf.
24. Perie, M. (2005, April). Angoff and Bookmark methods. Workshop presented at the annual meeting of the National Council on Measurement in Education, Montreal, Canada.
25. Reckase, M. D. (2000). A survey and evaluation of recently developed procedures for setting standards on educational tests. In M. L. Bourque & S. Byrd (Eds.), Student performance standards on the National Assessment of Educational Progress. Washington, DC: National Assessment Governing Board.
26. Reckase, M. D. (2005, April). A theoretical evaluation of an item rating method and a bookmark method for setting standards. Paper presented at the annual meeting of the National Council on Measurement in Education, Montreal, Quebec.
27. Schulz, E. M., Lee, W., & Mullen, K. (2005). A domain-level approach to describing growth in achievement. Journal of Educational Measurement, 42, 1-26.
28. Sireci, S.G., Patelis, T., Rizavi, S., Dillingham, A., & Rodriguez, G. (2000, April). Setting standards on a computerized-adaptive placement examination. Paper presented at the annual meeting of the National Council on Measurement in Education, New Orleans, LA.
29. Skaggs, G., & Tessema, A. (2001, April). Item disordinality with the Bookmark standard setting procedure. Paper presented at the annual meeting of the National Council on Measurement in Education, Seattle, WA.
30. Skorupski, W. P., & Hambleton, R. K. (2005). What are panelists thinking when they participate in standard setting studies? Applied Measurement in Education, 18, 233-256.
31. Wang, N. (2003). Use of the Rasch IRT model in standard setting; An item mapping method. Journal of Educational Measurement, 40, 231-253.
32. Williams, N. J., & Schulz, E. M. (2005, April). An investigation of response probability (RP) values used in standard setting. Paper presented at the annual meeting of the National Council on Measurement in Education, Montreal, Canada.
33. Yin, P., & Schulz, E. M. (2005, April). A comparison of cut scores and cut score variability from Angoff-based and Bookmark-based procedures in standard setting. Paper presented at the annual meeting of the National Council on Measurement in Education, Montreal, Canada.