Redundant documents and search effectiveness

International Conference on Information and Knowledge Management, Proceedings

Volumn , Issue , 2005, Pages 736-743

  Bernstein, Yaniv ^a   Zobel, Justin ^a  

^a RMIT UNIVERSITY   (Australia)

Author keywords

Duplicate detection; Novelty; Search effectiveness

Indexed keywords

CHARACTERIZATION; EVALUATION; INFORMATION ANALYSIS; KNOWLEDGE ACQUISITION; REDUNDANCY;

DUPLICATE DETECTION; NOVELTY; REDUNDANT DOCUMENTS; SEARCH EFFECTIVENESS;

WORLD WIDE WEB;

EID: 33745784229     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1099554.1099733     Document Type: Conference Paper

References (26)

1. Allan, J., Wade, C. & Bolivar, A. (2003), Retrieval and novelty detection at the sentence level,20in 'Proc. ACM SIGIR conference', ACM Press, pp. 314-321.
2. Bernstein, Y. & Zobel, J. (2004), A scalable system for identifying co-derivative documents, in 'Proc. String Processing and Information Retrieval Symposium (SPIRE)', Springer, pp. 55-67.
3. Brin, S., Davis, J. & García-Molina, H. (1995), Copy detection mechanisms for digital documents, in 'Proceedings of the ACM SIGMOD Annual Conference', pp. 398-409.
4. Broder, A. Z., Glassman, S. C., Manasse, M. S. & Zweig, G. (1997), 'Syntactic clustering of the Web', Computer Networks and ISDN Systems 29(8-13), 1157-1166.
5. Buckley, C. & Voorhees, E. M. (2000), Evaluating evaluation measure stability, in 'Proc. ACM SIGIR conference', ACM Press, pp. 33-40.
6. Cho, J., Shivakumar, N. & Garcia-Molina, H. (2000), Finding Replicated Web Collections, in 'Proc. ACM SIGMOD Conference', pp. 355-366.
7. Chowdhury, A., Frieder, O., Grossman, D. & McCabe, M. C. (2002), 'Collection statistics for fast duplicate document detection', ACM Transactions on Information Systems (TOIS) 20(2), 171-191.
8. Clarke, C., Craswell, N. 26amp; Soboroff, I. (2004), Overview of the TREC 2004 Terabyte Track, in 'Proceedings of the 13th Text REtrieval Conference (TREC 2004)'.
9. Fetterly, D., Manasse, M. & Najork, M. (2003), On the Evolution of Clusters of Near-Duplicate Web Pages, in 'Proceedings of the 1st Latin American Web Congress', IEEE, pp. 37-45.
10. Garcia, S., Williams, H. E. & Cannane, A. (2004), Access-ordered indexes, in 'Proc, 27th conference on Australasian computer science', pp. 7-14.
11. Harman, D. (2002), Overview of the TREC 2002 Novelty Track, in 'The Eleventh Text REtrieval Conference (TREC 2002).
12. Hearst, M. A. & Pedersen, J. O. (1996), Reexamining the cluster hypothesis: scatter/gather on retrieval results, in 'Proc. ACM SIGIR conference', ACM Press, pp. 76-84.
13. Heintze, N. (1996), Scalable Document Fingerprinting, in '1996 USENIX Workshop on Electronic Commerce'.
14. Hoad, T. C. & Zobel, J. (2003), 'Methods for Identifying Versioned and Plagiarised Documents', Journal of the American Society for Information Science and Technology 54(3), 203-215.
15. Manber, U. (1994), Finding Similar Files in a Large File System, in 'Proceedings of the USENIX Winter 1994 Technical Conference', pp. 1-10.
16. Rivest, R. (1992), 'The MD5 Message-Digest Algorithm'. RFC 1321.
17. Sanderson, M. & Zobel, J. (2005), Information Retrieval System Evaluation: Effort, Sensitivity, and Reliability, in 'Proc. ACM SIGIR conference', pp. 162-169.
18. Schleimer, S., Wilkerson, D. S. & Aiken, A. (2003), Winnowing: local algorithms for document fingerprinting, in 'Proc. ACM SIGMOD conference', ACM Press, pp. 76-85.
19. Shivakumar, N. & García-Molina, H. (1995), SCAM: A Copy Detection Mechanism for Digital Documents, in 'Proceedings of the Second Annual Conference on the Theory and Practice of Digital Libraries'.
20. Shivakumar, N. & García-Molina, H. (1999), Finding Near-Replicas of Documents on the Web, in 'WEBDB: International Workshop on the World Wide Web and Databases, WebDB', Springer-Verlag.
21. Soboroff, I. & Harman, D. (2003), Overview of the TREC 2003 Novelty Track, in 'The Twelfth Text REtrieval Conference (TREC 2003)', pp. 38-53.
22. van Rijsbergen, C. J. (1979), Information Retrieval, Butterworth- Heinemann.
23. Voorhees, E. M. & Buckley, C. (2002), The effect of topic set size on retrieval experiment error, in 'Proc. ACM SIGIR conference', ACM Press, pp. 316-323.
24. Witten, I. H., Moffat, A. & Bell, T. C. (1999), Managing Gigabytes; Compressing and Indexing Documents and Images, Morgan Kauffman.
25. Zhai, C. X., Cohen, W. W. & Lafferty, J. (2003), Beyond independent relevance: methods and evaluation metrics for subtopic retrieval, in 'Proc. ACM SIGIR conference', ACM Press, pp. 10-17.
26. Zhang, Y., Callan, J. & Minka, T. (2002), Novelty and redundancy detection in adaptive filtering, in 'Proc. ACM SIGIR conference', ACM Press, pp. 81-88.