File recipe compression in data deduplication systems

Proceedings of the 11th USENIX Conference on File and Storage Technologies, FAST 2013

Volumn , Issue , 2013, Pages 175-182

  Meister, Dirk ^a   Brinkmann, André ^a   Süß, Tim ^a  

^a JOHANNES GUTENBERG UNIVERSITY   (Germany)

Author keywords

[No Author keywords available]

Indexed keywords

DATA COMPRESSION; REDUNDANCY;

DATA BLOCKS; DATA DE DUPLICATIONS; DE DUPLICATIONS; DISK SPACE; FILE CONTENTS; SCALABLE COMPRESSION; TRACE-BASED SIMULATION;

DIGITAL STORAGE;

EID: 85062210551     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (31)

1. ABADI, D., MADDEN, S., AND FERREIRA, M. Integrating Compression and Execution in Column-Oriented Database Systems. In Proceedings of the 2006 ACM SIGMOD International Conference on Management of Data (2006), pp. 671-682.
2. ALVAREZ, C. NetApp Deduplication for FAS and V-Series Deployment and Implementation Guide. Tech. rep., NetApp, 2009.
3. BALACHANDRAN, S., AND CONSTANTINESCU, C. Sequence of Hashes Compression in Data De-duplication. In Proceedings of the 18thData Compression Conference (DCC) (2008), pp. 671-682.
4. BHAGWAT, D., ESHGHI, K., LONG, D., AND LILLIBRIDGE, M. Extreme Binning: Scalable, Parallel Deduplication for Chunkbased File Backup. In Proceedings of the 17th IEEE International Symposium on Modeling, Analysis, and Simulation (MASCOTS) (2009), pp. 1-9.
5. CONSTANTINESCU, C., PIEPER, J., AND LI, T. Block Size Optimization in Deduplication Systems. In Proceedings of the 19thData Compression Conference (DCC) (2009), p. 442.
6. ESHGHI, K., LILLIBRIDGE, M., WILCOCK, L., BELROSE, G., AND HAWKES, R. Jumbo Store: Providing Efficient Incremental Upload and Versioning for a Utility Rendering Service. In Proceedings of the 5th USENIX Conference on File and Storage Technologies (FAST) (2007), pp. 123-138.
7. HOLLOWAY, A. L., RAMAN, V., SWART, G., AND DEWITT, D. J. How to Barter Bits for Chronons: Compression and Bandwidth Trade Offs for Database Scans. In Proceedings of the 2007 ACM SIGMOD International Conference on Management of Data (2007), pp. 389-400.
8. HONG, B., AND LONG, D. D. E. Duplicate Data Elimination in a SAN File System. In Proceedings of the 21st IEEE / 12th NASA Goddard Conference on Mass Storage Systems and Technologies (MSST) (2004), pp. 301-314.
9. HUFFMAN, D. A method for the construction of minimumredundancy codes. Proceedings of the IRE 40, 9 (1952), 1098-1101.
10. JIN, K., AND MILLER, E. L. The Effectiveness of Deduplication on Virtual Machine Disk Images. In Proceedings of the 2nd Israeli Experimental Systems Conference (SYSTOR) (2009), p. 7.
11. KAISER, J., MEISTER, D., BRINKMANN, A., AND EFFERT, S. Design of an Exact Data Deduplication Cluster. In Proceedings of the 28th IEEE Conference on Mass Storage Systems and Technologies (MSST) (2012), pp. 1-12.
12. LILLIBRIDGE, M., ESHGHI, K., BHAGWAT, D., DEOLALIKAR, V., TREZISE, G., AND CAMBLE, P. Sparse Indexing: Large Scale, Inline Deduplication using Sampling and Locality. In Proceedings of the 7th USENIX Conference on File and Storage Technologies (FAST) (2009), pp. 111-123.
13. LOKESHWARI, Y. V., PRABAVATHY, B., AND BABU, C. Optimized Cloud Storage with High Throughput Deduplication Approach. In Proceedings of the International Conference on Emerging Technology Trends (2011), pp. 32-37.
14. MANBER, U., ET AL. Finding similar files in a large file system. In Proceedings of the USENIXWinter 1994 Technical Conference (1994), San Fransisco, CA, USA, pp. 1-10.
15. MANNING, C. D., RAGHAVAN, P., AND SCHÜTZE, H. Introduction to Information Retrieval. Cambridge University Press, 2009.
16. MEISTER, D., AND BRINKMANN, A. Multi-level Comparison of Data Deduplication in a Backup Scenario. In Proceedings of the 2nd Israeli Experimental Systems Conference (SYSTOR) (2009), pp. 8:1-8:12.
17. MEISTER, D., AND BRINKMANN, A. dedupv1: Improving Deduplication Throughput using Solid State Drives. In Proceedings of the 26th IEEE Conference on Mass Storage Systems and Technologies (MSST) (2010), pp. 1-6.
18. MEYER, D. T., AND BOLOSKY, W. J. A Study of Practical Deduplication. ACM Transactions on Storage 7, 4 (2012), 14:1-14:20.
19. MISRA, J., AND GRIES, D. Finding Repeated Elements. Tech. rep., Cornell University, Ithaca, NY, USA, 1982.
20. PATTERSON, H. Data Storage using Identifiers. US Patent 7,143,251, November 2006.
21. QUINLAN, S., AND DORWARD, S. Venti: A new Approach to Archival Storage. In Proceedings of the 1st USENIX Conference on File and Storage Technologies (FAST) (2002).
22. RAAB, M., AND STEGER, A. "Balls into Bins"-A Simple and Tight Analysis. In Proceedings of the 2nd InternationalWorkshop on Randomization and Approximation Techniques in Computer Science (RANDOM) (1998), pp. 159-170.
23. SHILANE, P., WALLACE, G., HUANG, M., AND HSU, W. Delta compressed and deduplicated storage using stream-informed locality. In Proceedings of the 4th USENIX Conference on Hot Topics in Storage and File Systems (2012), pp. 10-15.
24. STONEBRAKER, M., HELD, G., WONG, E., AND KREPS, P. The Design and Implementation of INGRES. ACM Transactions on Database Systems 1, 3 (1976), 189-222.
25. TAN, Y., JIANG, H., FENG, D., TIAN, L., AND YAN, Z. CABdedupe: A Causality-Based Deduplication Performance Booster for Cloud Backup Services. In Proceedings of the 25th IEEE International Parallel and Distributed Processing Symposium (IPDPS) (2011), pp. 1266-1277.
26. TOLIA, N., KOZUCH, M., SATYANARAYANAN, M., KARP, B., BRESSOUD, T., AND PERRIG, A. Opportunistic use of Content Addressable Storage for Distributed File Systems. In Proceedings of the 2003 USENIX Annual Technical Conference (2003), pp. 127-140.
27. UNGUREANU, C., ATKIN, B., ARANYA, A., GOKHALE, S., RAGO, S., CALKOWSKI, G., DUBNICKI, C., AND BOHRA, A. HydraFS: A High-throughput File System for the HYDRAstor Content-addressable Storage System. In Proceedings of the 8th USENIX Conference on File and Storage Technologies (FAST) (2010), pp. 225-238.
28. WALLACE, G., DOUGLIS, F., QIAN, H., SHILANE, P., AND SMALDONE, S. Characteristics of BackupWorkloads in Production Systems. In Proceedings of the 12th USENIX Conference on File and Storage Technologies (FAST) (2012), pp. 1-16.
29. WEI, J., JIANG, H., ZHOU, K., AND FENG, D. MAD2: A Scalable High-throughput Exact Deduplication Approach for Network Backup Services. In Proceedings of the 26th IEEE Conference on Mass Storage Systems and Technologies (MSST) (2010), pp. 1-14.
30. WITTEN, I. H., MOFFAT, A., AND BELL, T. C. Managing Gigabytes. Morgan Kaufmann, 1999.
31. ZHU, B., LI, K., AND PATTERSON, H. Avoiding the Disk Bottleneck in the Data Domain Deduplication File System. In Proceedings of the 6th USENIX Conference on File and Storage Technologies (FAST) (2008), pp. 18:1-18:14.