Effect of written instructions on assessor agreement

SIGIR'12 - Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval

Volumn , Issue , 2012, Pages 1053-1054

  Webber, William ^a   Toth, Bryan ^b   Desamito, Marjorie ^b  

^a UNIVERSITY OF MARYLAND   (United States)

^b Eleanor Roosevelt High School   (United States)

Author keywords

e discovery; evaluation; retrieval experiment

Indexed keywords

E-DISCOVERY; EVALUATION;

INFORMATION RETRIEVAL;

EID: 84866634816     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2348283.2348465     Document Type: Conference Paper

References (6)

1. Jacob Cohen. A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20(1):37-46, 1960.
2. Maura R. Grossman and Gordon V. Cormack. Technology-assisted review in e-discovery can be more effective and more efficient than exhaustive manual review. Richmond Journal of Law and Technology, 17(3):11:1-48, 2011.
3. Bruce Hedin, Stephen Tomlinson, Jason R. Baron, and Douglas W. Oard. Overview of the TREC 2009 legal track. In Ellen Voorhees and Lori P. Buckland, editors, Proc. 18th Text REtrieval Conference, pages 1:4:1-40, Gaithersburg, Maryland, USA, November 2009. NIST Special Publication 500-278.
4. Herbert L. Roitblat, Anne Kershaw, and Patrick Oot. Document categorization in legal electronic discovery: computer classification vs. manual review. Journal of the American Society for Information Science and Technology, 61(1):70-80, 2010.
5. Ellen Voorhees. Variations in relevance judgments and the measurement of retrieval effectiveness. Information Processing & Management, 36(5): 697-716, September 2000.
6. WilliamWebber. Re-examining the effectiveness of manual review. In Proc. SIGIR Information Retrieval for E-Discovery Workshop, pages 2:1-8, Beijing, China, July 2011.