Randomized protocols for duplicate elimination in peer-to-peer storage systems

IEEE Transactions on Parallel and Distributed Systems

Volumn 18, Issue 5, 2007, Pages 686-696

  Ferreira, Ronaldo A ^a   Ramanathan, Murali K ^a   Grama, Ananth ^a   Jagannathan, Suresh ^a  

^a Purdue University   (United States)

Author keywords

Duplicate elimination; Peer to peer; Randomized algorithms; Unstructured networks

Indexed keywords

COMPUTER SIMULATION; DISTRIBUTED COMPUTER SYSTEMS; NETWORK PROTOCOLS; PROBABILITY; RANDOM PROCESSES;

DUPLICATE ELIMINATION; PEER-TO-PEER NETWORKS; RANDOMIZED ALGORITHMS; UNSTRUCTURED NETWORKS;

CLIENT SERVER COMPUTER SYSTEMS;

EID: 34247558495     PISSN: 10459219     EISSN: None     Source Type: Journal    
DOI: 10.1109/TPDS.2007.351710     Document Type: Article

References (31)

1. A. Adya, W.J. Bolosky, M. Castro, G. Cermak, R. Chaiken, J.R. Douceur, J. Howell, J.R. Lorch, M. Theimer, and R.P. Wattenhofer, "FARSITE: Federated, Available, and Reliable Storage for an Incompletely Trusted Environment," Proc. Symp. Operating Systems Design and Implementation (OSDI '02), Dec. 2002.
2. A. Awan, R. Ferreira, A. Grama, and S. Jagannathan, "Distributed Uniform Sampling in Large Real-World Networks," technical report, Dept. of Computer Sciences, Purdue Univ., Oct. 2004.
3. M. Bawa, H. Garcia-Molina, A. Gionis, and R. Motwani, "Estimating Aggregates on a Peer-to-Peer Network," technical report, Computer Science Dept., Stanford Univ., 2003.
4. W.J. Bolosky, S. Corbin, D. Goebel, and J.R. Douceur, "Single Instance Storage in Windows 2000," Proc. Fourth USENIX Windows Systems Symp., Aug. 2000.
5. S. Boyd, P. Diaconis, and L. Xiao, "Fastest Mixing Markov Chain on a Graph," SIAM Rev., Problems and Techniques Section, vol. 46, no. 4, pp. 667-689, Dec. 2004.
6. L. Cox, C. Murray, and B. Noble, "Pastiche: Making Backup Cheap and Easy," Proc. Symp. Operating Systems Design and Implementation (OSDI '02), Dec. 2002.
7. L. Cox and B. Noble, "Samsara: Honor among Thieves in Peer-to-Peer Storage," Proc. ACM Symp. Operating Systems Principles (SOSP '03), Oct. 2003.
8. J. Douceur, A. Adya, W. Bolosky, D. Simon, and M. Theimer, "Reclaiming Space from Duplicate Files in a Serverless Distributed File System," Proc. IEEE Int'l Conf. Distributed Computing Systems (ICDCS '02), July 2002.
9. J.R. Douceur and W.J. Bolosky, "A Large-Scale Study of File-System Contents," Proc. SIGMETRICS '99, pp. 59-70, 1999.
10. R. Ferreira, M.K. Ramanathan, A. Awan, A. Grama, and S. Jagannathan, "Search with Probabilistic Guarantees in Unstructured Peer-to-Peer Networks," Proc. IEEE Int'l Conf. Peer-to-Peer Computing (P2P '05), Aug. 2005.
11. Freenet, http://freenet.sourceforge.net/, 2007.
12. C. Gkantsidis, M. Mihail, and A. Saberi, "Random Walks in Peer-to-Peer Networks," Proc. IEEE INFOCOM '04, Mar. 2004.
13. Gnutella, http://gnutella.wego.com/, 2007.
14. A.G. Goldberg and P.N. Yianilos, "Towards an Archival Intermemory," Proc. IEEE Advances in Digital Libraries (ADL), 1998.
15. W. Hastings, "Monte Carlo Sampling Methods Using Markov Chains and Their Applications," Biometrika, vol. 57, no. 1, pp. 97-109, 1970.
16. K. Horowitz and D. Malkhi, "Estimating Network Size from Local Information," Information Processing Letters, vol. 88, no. 5, pp. 237-243, Dec. 2003.
17. P. Kulkarni, F. Douglis, J. LaVoie, and J.M. Tracey, "Redundancy Elimination within Large Collection of Files," Proc. USENIX Ann. Technical Conf., pp. 59-72, June 2004.
18. M. Lillibridge, S. Elnikety, A. Birrel, and M. Burrows, "A Cooperative Internet Backup Scheme," Proc. Usenix Technical Conf., June 2003.
19. N.A. Lynch, Distributed Algorithms. Morgan Kaufmann Publishers, 1996.
20. D. Malkhi, M. Reiter, and R. Wright, "Probabilistic Quorum Systems," Proc. 16th Ann. ACM Symp. Principles of Distributed Computing (PODC '97), pp. 267-273, Aug. 1997.
21. N. Metropolis, A. Rosenbluth, M. Rosenbluth, A. Teller, and E. Teller, "Equations of State Calculations by Fast Computing Machines," J. Chemical Physics, vol. 21, pp. 1087-1091, 1953.
22. R. Motwani and P. Raghavan, Randomized Algorithms. Cambridge Univ. Press, 1995.
23. PlanetLab, http://www.planet-lab.org/, 2007.
24. C. Policroniades and I. Pratt, "Alternatives for Detecting Redundancy in Storage Systems Data," Proc. USENIX Ann. Technical Conf., pp. 73-86, June 2004.
25. S. Ratnasamy, P. Francis, M. Handley, R. Karp, and S. Shenker, "A Scalable Content-Addressable Network," Proc. ACM SIGCOMM '01, pp. 247-254, Aug. 2001.
26. A. Rowstron and P. Druschel, "Pastry: Scalable, Decentralized Object Location and Routing for Large-Scale Peer-to-Peer Systems," Proc. ACM SIGCOMM '01, pp. 247-254, Aug. 2001.
27. Y. Saito, C. Karamanolis, M. Karlsson, and M. Mahalingam, "Taming Aggressive Replication in the Pangaea Wide-Area File System," Proc. Symp. Operating Systems Design and Implementation (OSDI '02), Dec. 2002.
28. S. Saroiu, P.K. Gummadi, and S.D. Gribble, "A Measurement Study of Peer-to-Peer File Sharing Systems," Proc. SPIE/ACM Conf. Multimedia Computing and Networking (MMCN '02), Jan. 2002.
29. N. Sarshar, P. Boykin, and V. Roychowdhury, "Percolation Search in Power-Law Networks: Making Unstructured Peer-to-Peer Networks Scalable," Proc. Fourth IEEE Int'l Conf. Peer-to-Peer Computing (P2P '04), Aug. 2004.
30. I. Stoica R. Morris, D. Karger, F. Kaashoek, and H. Balakrishnan, "Chord: A Scalable Peer-To-Peer Lookup Service for Internet Applications," Proc. ACM SIGCOMM '01, pp. 149-160, Aug. 2001.
31. B.Y. Zhao, J. Kubiatowicz, and A.D. Joseph, "Tapestry: An Infrastructure for Fault-Tolerant Wide-Area Location and Routing," Technical Report UCB/CSD-0101141, Computer Science Division, Univ. of California Berkeley, Apr. 2001.