Migratory compression: Coarse-grained data reordering to improve compressibility

Proceedings of the 12th USENIX Conference on File and Storage Technologies, FAST 2014

Volumn , Issue , 2014, Pages 257-271

  Lin, Xing ^a   Lu, Guanlin ^b   Douglis, Fred ^b   Shilane, Philip ^b   Wallace, Grant ^b  

^a UNIVERSITY OF UTAH   (United States)

^b GNSS Technology for Rail Transportation

Author keywords

[No Author keywords available]

Indexed keywords

COMPRESSORS; FILE ORGANIZATION; METADATA;

COARSE-GRAINED; COMPRESSION FACTOR; DATA REORDERING; DATA TRANSFORMATION; DELTA COMPRESSION; PROTOTYPE IMPLEMENTATIONS; RUN-TIME PERFORMANCE; STORAGE SYSTEMS;

DIGITAL STORAGE;

EID: 85077051296     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (30)

1. 7-zip. http://www.7-zip.org/. Retrieved Sep. 7, 2013.
2. ARONOVICH, L., ASHER, R., BACHMAT, E., BIT-NER, H., HIRSCH, M., AND KLEIN, S. T. The design of a similarity based deduplication system. In Proceedings of SYSTOR 2009: The Israeli Experimental Systems Conference (2009).
3. BRODER, A. Z. On the resemblance and containment of documents. In In Compression and Complexity of Sequences (SEQUENCES97) (1997), IEEE Computer Society.
4. BURROWS, M., JERIAN, C., LAMPSON, B., AND MANN, T. On-line data compression in a log-structured file system. In Proceedings of the fifth international conference on Architectural support for programming languages and operating systems (1992), ASPLOS V.
5. BURROWS, M., AND WHEELER, D. J. A block-sorting lossless data compression algorithm. Tech. Rep. SRC-RR-124, Digital Equipment Corporation, 1994.
6. DEUTSCH, P. DEFLATE Compressed Data Format Specification version 1.3. RFC 1951 (Informational), May 1996.
7. FIALA, E. R., AND GREENE, D. H. Data compression with finite windows. Communications of the ACM 32, 4 (Apr. 1989), 490–505.
8. GILCHRIST, J. Parallel data compression with bzip2. In Proceedings of the 16th IASTED International Conference on Parallel and Distributed Computing and Systems (2004), vol. 16, pp. 559–564.
9. HUNT, J. J., VO, K.-P., AND TICHY, W. F. Delta algorithms: an empirical analysis. ACM Trans. Softw. Eng. Methodol. 7 (April 1998), 192–214.
10. JAIN, N., DAHLIN, M., AND TEWARI, R. Taper: Tiered approach for eliminating redundancy in replica synchronization. In 4th USENIX Conference on File and Storage Technologies (2005).
11. KORN, D. G., AND VO, K.-P. Engineering a differencing and compression data format. In USENIX Annual Technical Conference (2002).
12. KULKARNI, P., DOUGLIS, F., LAVOIE, J., AND TRACEY, J. M. Redundancy elimination within large collections of files. In USENIX 2004 Annual Technical Conference (June 2004).
13. LILLIBRIDGE, M., ESHGHI, K., AND BHAGWAT, D. Improving restore speed for backup systems that use inline chunk-based deduplication. In 11th USENIX Conference on File and Storage Technologies (Feb 2013).
14. MACDONALD, J. File system support for delta compression. Masters thesis. Department of Electrical Engineering and Computer Science, University of California at Berkeley, 2000.
15. MAKATOS, T., KLONATOS, Y., MARAZAKIS, M., FLOURIS, M. D., AND BILAS, A. Using transparent compression to improve SSD-based I/O caches. In Proceedings of the 5th European Conference on Computer Systems (2010), EuroSys’10.
16. MOGUL, J. C., DOUGLIS, F., FELDMANN, A., AND KRISHNAMURTHY, B. Potential benefits of delta encoding and data compression for http. In Proceedings of the ACM SIGCOMM’97 conference on Applications, technologies, architectures, and protocols for computer communication (1997), SIGCOMM’97.
17. MUTHITACHAROEN, A., CHEN, B., AND MAZIÈRES, D. A low-bandwidth network file system. In Proceedings of the eighteenth ACM symposium on Operating systems principles (2001), SOSP’01.
18. PARK, K., IHM, S., BOWMAN, M., AND PAI, V. S. Supporting practical content-addressable caching with CZIP compression. In USENIX ATC (2007).
19. RIVERBED TECHNOLOGY. Wan Optimization (Steelhead). http://www.riverbed.com/productssolutions/products/wan-optimization-steelhead/, 2014. Retrieved Jan. 13, 2014.
20. SHILANE, P., WALLACE, G., HUANG, M., AND HSU, W. Delta compressed and deduplicated storage using stream-informed locality. In Proceedings of the 4th USENIX conference on Hot Topics in Storage and File Systems (June 2012), USENIX Association.
21. SMALDONE, S., WALLACE, G., AND HSU, W. Efficiently storing virtual machine backups. In Proceedings of the 5th USENIX conference on Hot Topics in Storage and File Systems (June 2013), USENIX Association.
22. SPRING, N. T., AND WETHERALL, D. A protocol-independent technique for eliminating redundant network traffic. In ACM SIGCOMM (2000).
23. TRIDGELL, A. Efficient algorithms for sorting and synchronization. PhD thesis, Australian National University Canberra, 1999.
24. TUDUCE, I. C., AND GROSS, T. Adaptive main memory compression. In USENIX 2005 Annual Technical Conference (April 2005).
25. VARIA, J., AND MATHEW, S. Overview of amazon web services, 2012.
26. WALLACE, G., DOUGLIS, F., QIAN, H., SHILANE, P., SMALDONE, S., CHAMNESS, M., AND HSU, W. Characteristics of backup workloads in production systems. In FAST’12: Proceedings of the 10th Conference on File and Storage Technologies (2012).
27. xz. http://tukaani.org/xz/. Retrieved Sep. 25, 2013.
28. ZHU, B., LI, K., AND PATTERSON, H. Avoiding the disk bottleneck in the data domain deduplication file system. In 6th USENIX Conference on File and Storage Technologies (Feb 2008).
29. ZIV, J., AND LEMPEL, A. A universal algorithm for sequential data compression. IEEE Transactions on Information Theory 23, 3 (May 1977), 337–343.
30. ZIV, J., AND LEMPEL, A. Compression of individual sequences via variable-rate coding. Information Theory, IEEE Transactions on 24, 5 (1978), 530–536.