Collection statistics for fast duplicate document detection

ACM Transactions on Information Systems

Volumn 20, Issue 2, 2002, Pages 171-191

  Chowdhury, Abdur ^a   Frieder, Ophir ^a   Grossman, David ^a   McCabe, Mary Catherine ^a  

^a ILLINOIS INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COLLECTION STATISTICS; DATA COLLECTION; WEB DOCUMENTS;

DATA RECORDING; INFORMATION TECHNOLOGY; MAGNETIC DISK STORAGE; STATISTICS; WORLD WIDE WEB;

INFORMATION RETRIEVAL SYSTEMS;

EID: 0013206133     PISSN: 10468188     EISSN: 10468188     Source Type: Journal    
DOI: 10.1145/506309.506311     Document Type: Article

References (24)

1. BAEZA-YATES, R. AND RIDEIRO-NETO, B. 1999. Modern Information Retrieval. Addison Wesley.
2. BRIN, S., DAVIS, J., AND GARCIA-MOLINA, H. 1995. Copy Detection Mechanisms for Digital Documents. In Proceeding of the Special Interest Group on Management of Data (SIGMOD'95) (San Francisco, CA., May). 298-409.
3. BRODER, A., GLASSMAN, S., MANASSE, S., AND ZWEIG, G. 1997. Syntactic clustering of the web. In Proceedings of the Sixth International World Wide Web Conference (WWW6'97) (Santa Clara, CA., April). 391-104.
4. BUCKLEY, C., CARDIE, C., MARDIS, S., MITRA, M., PIERCE, D., WAGSTAFF, K., AND WALZ, J. 1999. The Smart/Empire TIPSTER IR System. In Proceedings of TIPSTER Phase III (San Francisco, CA.). 107-121.
5. CHOWDHURY, A., HOLMES, D., MCCABE, M. C., GROSSMAN, D., AND FRIEDER, O. 2000. The use of fusion with AIRE at TREC-9. In Proceedings of the Ninth Text Retrieval Conference (TREC-9, Gathersburg, MD, November).
6. FRIEDER, O., GROSSMAN, D., CHOWDHURY, A., AND FRIEDER, G. 2000. Efficiency considerations in very large information retrieval servers. J. Dig. Inf. 1, 5 (Apr).
7. GILES, L. AND LAWRENCE, S. 1999. Accessibility and distribution of information on the web. Nature 400, 107-109.
8. GROSSMAN, D., HOLMES, D., AND FRIEDER, O. 1993. A DBMS Approach to IR in TREC-4. In Proceedings of the Fourth Text Retrieval Conference (TREC-4) (Gaithersburg, Maryland, November).
9. HEINTZE, N. 1996. Scalable document fingerprinting. In Proceedings of the Second USENIX Electronic Commerce Workshop (Oakland, CA., November). 191-200.
10. KJELL, B., WOODS, W., AND FRIEDER, O. 1994. Discrimination of authorship using visualization. Information Processing and Management. Pergamon Press 30, 1 (Jan), 141-150.
11. KWOK, K. L. 1996. A new method of weighting query terms for Ad-hoc retrieval. In Proceedings of the 19th Annual International. ACM Special Interest Group on Information Retrieval (SIGIR '96) (Zurich, Switzerland, August), pp.187-195.
12. LAWRENCE, S. AND GILES, C. L. 1998. Searching the World Wide Web. Science. 280, 5360, 98-100.
13. NIH. 2000. http://nccam.nih.gov/, The National Institutes of Health (NIH), National Center for Complementary and Alternative Medicine (NCCAM), April 12, 2000.
14. NIST. 1995. Secure Hash Standard, U.S. Department of Commerce/National Institute of Standards and Technology, FIPS PUB 180-1 (April 17).
15. PORTER, M. 1980. An algorithm for suffix stripping. Program 14, 3, 130-137.
16. ROBERTSON, S. WALKER, S., AND BEAULIEU, M. 1999. Okapi at TREC-7: Automatic Ad Hoc, Filtering, VLC and interactive, Proceedings of the 7th Text Retrieval Conference (TREC-7'99) (July). 253-264.
17. ROCCHIO, J. 1971. Relevance Feedback in Information Retrieval. In The Smart System - experiments in automatic document processing, pages 313-323. Prentice Hall, Englewood Cliffs, NJ.
18. SALTON, G., YANG, C. S., AND WONG, A. 1975. A vector-space model for information retrieval. C. ACM. ASIS. 18, 11, 613-620.
19. SANDERSON, M. 1997. Duplicate detection in the Reuters collection. Technical Report (TR-1997-5) of the Department of Computing Science at the University of Glasgow, Glasgow G12 8QQ, UK.
20. SCOTTI, R. AND LILLY, C. 1999. George Washington University Declassification Productivity Research Center, http://dprc.seas.gwu.edu. July 31.
21. SHIVAKUMAR, N. AND GARICA-MOLINA, H. 1998. Finding near-replicas of documents on the web. In Proceedings of Workshop on Web Databases (WebDB'98) (Valencia, Spain, March). 204-212.
22. SINGHAL, A., BUCKLEY, C., AND MITRA, M. 1996. Pivoted Document Length Normalization, Proceedings of the Nineteenth Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR'96) (Zurich, Switzerland, August). 21-29.
23. SMART FTP site: 2000. ftp://ftp.cs.cornell.edu/pub/smart/. January 19.
24. SMEATON, A., KELLEDY, F., AND QUINN, G. 1997. Ad-hoc retrieval using thresholds, WSTs for French monolingual retrieval, Document-at-a-Glance for high precision and triphone windows for spoken documents. In Proceedings of the Sixth Text Retrieval Conference (TREC-6, Gathersburg, Maryland).