A novel optimization method to improve de-duplication storage system performance

Proceedings of the International Conference on Parallel and Distributed Systems - ICPADS

Volumn , Issue , 2009, Pages 228-235

  Liu, Chuanyi ^a,b   Xue, Yibo ^b   Ju, Dapeng ^b   Wang, Dongsheng ^a,b  

^a TSINGHUA UNIVERSITY   (China)

^b TSINGHUA NATIONAL LABORATORY FOR INFORMATION SCIENCE AND TECHNOLOGY   (China)

Author keywords

[No Author keywords available]

Indexed keywords

COMMODITY COMPONENTS; CONTENT DISTRIBUTION ALGORITHM; CONTENT-AWARE; CPU-INTENSIVE; DATA DUPLICATION; FUNCTION COMPONENTS; MAIN COMPONENT; NETWORK COMMUNICATIONS; OPTIMIZATION METHOD; PARALLEL CALCULATION; PERFORMANCE OPTIMIZATIONS; PIPELINE DESIGN; POTENTIAL SYSTEMS; STORAGE PARADIGM; STORAGE SYSTEMS; SYSTEM THROUGHPUT; TASK PARTITION;

COMPUTER SYSTEMS;

EID: 77949620656     PISSN: 15219097     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICPADS.2009.103     Document Type: Conference Paper

References (28)

1. M. O. Rabin. Fingerprinting by random polynomials. Technical Report TR-15-81, Center for Research in Computing Technology, Harvard University, 1981.
2. Chuanyi Liu, Yingping Lu, David Du and Dongsheng Wang. ADMAD: Application-Driven Metadata Aware Deduplication Archival Storage System. International Workshop on Storage Network Architecture and Parallel I/Os (SNAPI 2008) held in conjunction with the 25th IEEE Conference on Mass Storage Systems and Technologies (MSST 2008)
3. Storage Networking Solutions. Object Storage Architecture: Defining a new generation of storage systems built on distributed, intelligent storage devices. http://www.snseurope.com/featuresfull.php?id=2193. 2004, 9
4. Storage Networking Solutions - Europe. Object Storage Architecture: Defining a new generation of storage systems built on distributed, intelligent storage devices. October 2004. Available at: http://www.snseurope.com/ featuresfull.php?id=2193.
5. S. Ghemawat, H. Gobioff, and S.-T. Leung. The Google File System. Proc. SOSP'03, 2003.
6. Sage Weil, Scott A. Brandt, Ethan L. Miller, Darrell D. E. Long, and Carlos Maltzahn, Ceph: A Scalable, High-Performance Distributed File System, Proceedings of the 7th Conference on Operating Systems Design and Implementation (OSDI '06), November 2006.
7. Noga Alon, Haim Kaplan, Michael Krivelevich, Dahlia Malkhi, and Julien P. Stern. Scalable secure storage when half the system is faulty. Proc. 27th International Colloquium on Automata, Languages and Programming (ICALP), volume 1853 of Lecture Notes in Computer Science, pages576-587. Springer, 2000.
8. RFC 1321. The MD5 Message-Digest Algorithm. USA: Network Working Group, 1992.
9. RFC 3174. US Secure Hash Algorithm 1 (SHA1). USA: Network Working Group, 2001.
10. Benjamin Zhu; Kai Li; Hugo Patterson. Avoiding the Disk Bottleneck in the Data Domain Deduplication File System. Proceedings of the 6th USENIX Conference on File and Storage Technologies (FAST '08), February 2008, Pp. 269-282
11. Andrew Leung, Minglong Shao, Timothy Bisson, Shankar Pasupathy, Ethan L. Miller. Spyglass: Fast, Scalable Metadata Search for Large-Scale Storage Systems. Proceedings of the 7th USENIX Conference on File and Storage Technologies (FAST '09), February 2009
12. Intel Server Architecture Lab. Iometer: The I/O Performance Analysis Tool for Servers. accessed from http://www.intel.com/design/servers/devtools/ iometer/index.htm.
13. R. Coker. Bonnie++ Benchmark Tool. accessed from http://www.coker.com.au/ bonnie++/, 2004.
14. Mark Lillibridge, Kave Eshghi, Deepavali Bhagwat, Vinay Deolalikar, Greg Trezise, and Peter Campbell. Sparse Indexing, Large Scale, Inline Deduplication Using Sampling and Locality. In Proceedings of the Seventh USENIX Conference on File and Storage Technologies (FAST) 2009, San Francisco, CA
15. Data Domain. http://www.datadomain.com.
16. Quantum Dxi-Series, http://www.quantum.com/Products/Disk-BasedBackup/ Index.aspx
17. EMC Centera. Content Addressed Storage, http://www.emc.com/pdf/products/ centera/centera guide.pdf. 2002
18. Symantec Veritas NetBackup PureDisk Product Overview, http://www.symantec.com/business/products/overview.jsp?pcid=2244
19. W. J. Bolosky, J. R. Douceur, D. Ely, and M. Theimer. Feasibility of a serverless distributed file system deployed on an existing set of desktop pcs. SIGMETRICS Perform. Eval. Rev., 28(1):34-43, 2000.
20. N Tolia, M Kozuch, and M Satyanarayanan, et al. Opportunistic Use of Content Addressable Storage for Distributed File Systems. In Proc. of Usenix 2003 Annual Technical Conference, San Antonio, TX, USA
21. S Quinlan and S Dorward. Venti: A New Approach to Archival Storage. In Proceedings of Conference on File and Storage Technologies (2002).
22. Athicha Muthitacharoen, Benjie Chen, David Mazières, and Abhishek Rawat, A low bandwidth network file system. SOSP 2001.
23. M. Ajtai, R. Burns, R. Fagin, D. D. E. Long, and L. Stockmeyer, Compactly encoding unstructured inputs with differential compression. Journal of the ACM, 49(3):318-367, May 2002.
24. Lawrence L. You and Christos Karamanolis, Evaluation of Efficient Archival Storage Techniques. 12th NASA Goddard, 21st IEEE Conference on Mass Storage Systems and Technologies. April 13-16, 2004, College Park, Maryland, USA
25. J F Gantz, et al. The Expanding Digital Universe: A Forecast of Worldwide Information Growth through 2010. IDC, March 2007.
26. Red-black Tree Introduction From Wikipedia, accessed from http://en.wikipedia.org/wiki/Red-black-tree
27. Cryptographic hash function From Wikipedia, accessed http://en.wikipedia. org/wiki/Cryptographic-hash-function
28. Rosenblum, M., and Ousterhout, J.K. The design and implementation of a log-structured file system. ACM Transactions on Computer Systems (TOCS), 10(1). February, 1992.