To zip or not to zip: Effective resource usage for real-time compression

Proceedings of the 11th USENIX Conference on File and Storage Technologies, FAST 2013

Volumn , Issue , 2013, Pages 229-241

  Harnik, Danny ^a   Kat, Ronen ^a   Margalit, Oded ^a   Sotnikov, Dmitry ^a   Traeger, Avishay ^a  

^a IBM HAIFA RESEARCH LAB   (Israel)

Author keywords

[No Author keywords available]

Indexed keywords

COMPRESSIBILITY; DIGITAL STORAGE;

CAPACITY PLANNING; FILTERING SCHEMES; MEMORY RESOURCES; POTENTIAL CUSTOMERS; PRIMARY STORAGES; REAL-TIME COMPRESSION; RESOURCE USAGE; STORAGE SYSTEMS;

DATA COMPRESSION;

EID: 85109922112     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (25)

1. E. Bertrand. Playing with page compression-for real. http://sqlblog.com/blogs/ aaron_bertrand/archive/2009/12/29/ playing-with-page-compression-for-real. aspx.
2. Michael Burrows, Charles Jerian, Butler W. Lampson, and Timothy Mann. On-line data compression in a log-structured file system. In Barry Flahive and Richard L. Wexelblat, editors, ASPLOS, pages 2-9. ACM Press, 1992.
3. C. Constantinescu and M. Lu. Quick Estimation of Data Compression and De-duplication for Large Storage Systems. In Proceedings of the 2011 First International Conference on Data Compression, Communications and Processing, pages 98-102. IEEE, 2011.
4. W. Culhane. Statistical Measures as Predictors of Compression Savings. The Ohio State University, Department of Computer Science and Engineering, Honors Thesis, May 2008.
5. P. Deutsch and J. L. Gailly. Zlib Compressed Data Format Specification version 3.3. Technical Report RFC 1950, Network Working Group, May 1996.
6. EMC. EMC Data Compression: A Detailed Review. Technical Report h8045.1, EMC, September 2010.
7. J. L. Gailly and M. Adler. The Zlib home page. www.gzip.org/zlib, 1998.
8. J. Gantz and D. Reinsel. 2009 Digital Universe Study: As the Economy Contracts, the Digital Universe Expands. Technical Report White paper, International Data Corporation, May 2009.
9. J. Gantz and D. Reinsel. 2011 Digital Universe Study: Extracting Value from Chaos. Technical Report White paper, International Data Corporation, June 2011.
10. D. Harnik, O. Margalit, D. Naor, D. Sotnikov, and G. Vernik. Estimation of Deduplication Ratios in Large Data Sets. In Proceedings of the 18th International IEEE Symposium on Mass Storage Systems and Technologies (MSST), pages 1-11. IEEE, 2012.
11. Wassily Hoeffding. Probability inequalities for sums of bounded random variables. Journal of the American Statistical Association, 301(58):13-30, 1963.
12. D. A. Huffman. A Method for the Construction of Minimum-Redundancy Codes. Proceedings of the Institute of Radio Engineers, 40(9):1098-1101, September 1952.
13. IBM. IBM Comprestimator Utility V1.2.0.2-United States. http://www-01.ibm. com/support/docview.wss?uid= ssg1S4001012.
14. A. Kattan and R. Poli. Genetic-Programming Based Prediction of Data Compression Saving. Lecture Notes in Computer Science, 5975/2010:182-193, 2010.
15. D. Kay. Oracle Solaris ZFS Storage Management. Technical Report 507914, Oracle Corporation, November 2011.
16. M. Lu, D. Chambliss, J. Glider, and C. Constantinescu. Insights for Data Reduction in Primary Storage: a Practical Analysis. In Proceedings of the 5th Annual International Systems and Storage Conference (SYSTOR 2012). ACM, 2012.
17. Microsoft. sp estimate data compression savings (Transact-SQL). http://msdn.microsoft. com/en-us/library/cc280574.aspx.
18. S. Moulton and C. Alvarez. NetApp Data Compression and Deduplication Deployment and Implementation Guide: Data ONTAP Operating in Cluster-Mode. Technical Report TR-3966, NetApp, June 2012.
19. C. E. Shannon. A Mathematical Theory of Communication. Bell System Technical Journal, 27:379-423, 1948.
20. K. Srinivasan, T. Bisson, G. Goodson, and K. Voruganti. iDedup: Latency-aware, inline data deduplication for primary storage. In Proceedings of the 10th USENIX conference on File and Storage Technologies (FAST 2012). USENIX Association, 2012.
21. Nimble Storage. Nimble Storage: Engineered for Efficiency. Technical Report WP-EFE-0812, Nimble Storage, 2012.
22. J. Tate, B. Tuv-El, J. Quintal, E. Traitel, and B. Whyte. Real-time Compression in SAN Volume Controller and Storwize V7000. Technical Report REDP-4859-00, IBM, August 2012.
23. R. Tretau, M. Miletic, S. Pemberton, T. Provost, and T. Setiawan. Introduction to IBM Real-time Compression Appliances. Technical Report SG24-7953-01, IBM, January 2012.
24. J. Ziv and A. Lempel. A Universal Algorithm for Sequential Data Compression. IEEE Transactions on Information Theory, 23(2):337-343, May 1977.
25. J. Ziv and A. Lempel. Compression of Individual Sequences via Variable-Rate Coding. IEEE Transactions on Information Theory, 24(5):530-536, September 1978.