Efficient similarity estimation for systems exploiting data redundancy

Proceedings - IEEE INFOCOM

Volumn , Issue , 2010, Pages

  Tangwongsan, Kanat ^a   Pucha, Himabindu ^b   Andersen, David G ^a   Kaminsky, Michael ^c  

^a CARNEGIE MELLON UNIVERSITY   (United States)

^b IBM RESEARCH   (United States)

^c INTEL CORPORATION   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

CRITICAL PARAMETER; DATA ITEMS; DATA REDUNDANCY; EXISTING SYSTEMS; FILE TRANSFERS; MULTI-RESOLUTIONS; PEER TO PEER; SIMILARITY DETECTION; SIMILARITY ESTIMATION; SIZE SELECTION; STORAGE SYSTEMS;

QUALITY ASSURANCE; REDUNDANCY;

PEER TO PEER NETWORKS;

EID: 77953300991     PISSN: 0743166X     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/INFCOM.2010.5461965     Document Type: Conference Paper

References (25)

1. S. Annapureddy, M. J. Freedman, and D. Mazìeres. Shark: Scaling file servers via cooperative caching. In Proc. 2nd USENIX NSDI, May 2005.
2. A. Broder. On the resemblance and containment of documents. In SEQUENCES '97: Proceedings of the Compression and Complexity of Sequences 1997, page 21. IEEE Computer Society, 1997. ISBN 0-8186-8132-2.
3. A. Broder, S. Glassman, M. Manasse, and G. Zweig. Syntactic clustering of the web. In Proceedings of the 6th International WWW Conference, pages 1157-1166, Apr. 1997.
4. B. Cohen. Incentives build robustness in BitTorrent. In Workshop on Economics of Peer-to-Peer Systems, June 2003.
5. L. P. Cox, C. D. Murray, and B. D. Noble. Pastiche: Making backup cheap and easy. In Proc. 5th USENIX OSDI, Dec. 2002.
6. F. Dabek, M. F. Kaashoek, D. Karger, R. Morris, and I. Stoica. Wide-area cooperative storage with CFS. In Proc. 18th ACM Symposium on Operating Systems Principles (SOSP), Oct. 2001.
7. T. E. Denehy and W. W. Hsu. Duplicate management for reference data. Research Report RJ10305, IBM, Oct. 2003.
8. F. Douglis and A. Iyengar. Application-specific delta-encoding via resemblance detection. In Proceedings of the USENIX Annual Technical Conference, June 2003.
9. EMC Centera Content Addressed Storage System. EMC Corporation, 2003. http://www.emc.com/.
10. S. Janson. Large deviation inequalities for sums of indicator variables. Technical report, Uppsala University, 1994.
11. P. Kulkarni, F. Douglis, J. LaVoie, and J. M. Tracey. Redundancy elimination within large collections of files. In Proc. USENIX Annual Technical Conference, June 2004.
12. U. Manber. Finding similar files in a large file system. In Proc. Winter USENIX Conference, pages 1-10, Jan. 1994.
13. A. Muthitacharoen, B. Chen, and D. Mazieres. A low-bandwidth network file system. In Proc. 18th ACM Symposium on Operating Systems Principles (SOSP), Oct. 2001.
14. K. Park, S. Ihm, M. Bowman, and V. S. Pai. Supporting practical content-addressable caching with CZIP compression. In Proceedings of the USENIX Annual Technical Conference, June 2007.
15. C. Policroniades and I. Pratt. Alternatives for detecting redundancy in storage systems data. In Proc. USENIX Annual Technical Conference, June 2004.
16. H. Pucha, D. G. Andersen, and M. Kaminsky. Exploiting similarity for multi-source downloads using file handprints. In Proc. 4th USENIX NSDI, Apr. 2007.
17. H. Pucha, M. Kaminsky, D. G. Andersen, and M. A. Kozuch. Adaptive file transfers for diverse environments. In Proc. USENIX Annual Technical Conference, June 2008.
18. S. Quinlan and S. Dorward. Venti: A new approach to archival storage. In Proc. USENIX Conference on File and Storage Technologies (FAST), pages 89-101, Jan. 2002.
19. M. O. Rabin. Fingerprinting by random polynomials. Technical Report TR-15-81, Center for Research in Computing Technology, Harvard University, 1981.
20. S. C. Rhea, K. Liang, and E. Brewer. Value-based web caching. In Proc. Twelfth International World Wide Web Conference, May 2003.
21. Riverbed. Riverbed. http://www.riverbed.com/.
22. N. Spring and D. Wetherall. A protocol-independent technique for eliminating redundant network traffic. In Proc. ACM SIGCOMM, Sept. 2000.
23. D. Teodosiu, N. Bjorner, Y. Gurevich, M. Manasse, and J. Porkka. Optimizing file replication over limited bandwidth networks using remote differential compression. Technical Report MSR-TR-2006-157, Microsoft Research, Nov. 2006.
24. N. Tolia, M. Kaminsky, D. G. Andersen, and S. Patil. An architecture for Internet data transfer. In Proc. 3rd Symposium on Networked Systems Design and Implementation (NSDI), May 2006.
25. A. Tridgell and P. Mackerras. The rsync algorithm. Technical Report TR-CS-96-05, Department of Computer Science, The Australian National University, Canberra, Australia, 1996.