Design tradeoffs for data deduplication performance in backup workloads

Proceedings of the 13th USENIX Conference on File and Storage Technologies, FAST 2015

Volumn , Issue , 2015, Pages 331-344

  Fu, Min ^a   Feng, Dan ^a   Hua, Yu ^a   He, Xubin ^b   Chen, Zuoning ^c   Xia, Wen ^a   Zhang, Yucheng ^a   Tan, Yujuan ^d  

^a HUAZHONG UNIVERSITY OF SCIENCE AND TECHNOLOGY   (China)

^b VIRGINIA COMMONWEALTH UNIVERSITY   (United States)

^c National Parallel Computing Engineering Research Center   (China)

^d CHONGQING UNIVERSITY   (China)

Author keywords

[No Author keywords available]

Indexed keywords

RESTORATION;

BACK-UP SYSTEMS; DATA DE DUPLICATIONS; DIMENSIONAL PARAMETERS; EFFICIENT DESIGNS; GENERAL PURPOSE FRAMEWORK; MEMORY FOOTPRINT; PARAMETER SETTING; STANDARD COMPONENTS;

DIGITAL STORAGE;

EID: 85077062946     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (39)

1. Amazon. http://www.amazon.com, 2014.
2. Berkeley DB. http://www.oracle.com/us/products/database/berkeley-db/overview/index.htm, 2014.
3. The digital universe of opportunities. http://www.emc.com/leadership/digital-universe/2014iview/ executive-summary.htm, 2014.
4. Intel solid-state drive dc s3700 series. http://www.intel.com/content/www/us/en/solid-state-drives/ solid-state-drives-dc-s3700-series.html, 2014.
5. Linux kernel. http://www.kernel.org/, 2014.
6. MySQL. http://www.mysql.com/, 2014.
7. Redis. http://redis.io/, 2014.
8. ANAND, A., MUTHUKRISHNAN, C., KAPPES, S., AKELLA, A., AND NATH, S. Cheap and large CAMs for high performance data-intensive networked systems. In Proc. USENIX NSDI, 2010.
9. ARONOVICH, L., ASHER, R., BACHMAT, E., BITNER, H., HIRSCH, M., AND KLEIN, S. T. The design of a similarity based deduplication system. In Proc. ACM SYSTOR, 2009.
10. BHAGWAT, D., ESHGHI, K., LONG, D. D., AND LILLIBRIDGE, M. Extreme binning: Scalable, parallel deduplication for chunk-based file backup. In Proc. IEEE MASCOTS, 2009.
11. BHATOTIA, P., RODRIGUES, R., AND VERMA, A. Shredder: Gpu-accelerated incremental storage and computation. In Proc. USENIX FAST, 2012.
12. BLOOM, B. H. Space/time trade-offs in hash coding with allowable errors. Commun. ACM 13, 7 (July 1970), 422–426.
13. BRODER, A. On the resemblance and containment of documents. In Compression and Complexity of Sequences 1997. Proceedings (Jun 1997), pp. 21–29.
14. DEBNATH, B., SENGUPTA, S., AND LI, J. ChunkStash: speeding up inline storage deduplication using flash memory. In Proc. USENIX ATC, 2010.
15. DEBNATH, B., SENGUPTA, S., AND LI, J. SkimpyStash: Ram space skimpy key-value store on flash-based storage. In Proc. ACM SIGMOD, 2011.
16. DONG, W., DOUGLIS, F., LI, K., PATTERSON, H., REDDY, S., AND SHILANE, P. Tradeoffs in scalable data routing for deduplication clusters. In Proc. USENIX FAST, 2011.
17. FU, M., FENG, D., HUA, Y., HE, X., CHEN, Z., XIA, W., HUANG, F., AND LIU, Q. Accelerating restore and garbage collection in deduplication-based backup systems via exploiting historical information. In Proc. USENIX ATC, 2014.
18. GUO, F., AND EFSTATHOPOULOS, P. Building a high-performance deduplication system. In Proc. USENIX ATC, 2011.
19. HUANG, P., WU, G., HE, X., AND XIAO, W. An aggressive worn-out flash block management scheme to alleviate ssd performance degradation. In Proc. ACM EuroSys, 2014.
20. KACZMARCZYK, M., BARCZYNSKI, M., KILIAN, W., AND DUBNICKI, C. Reducing impact of data fragmentation caused by in-line deduplication. In Proc. ACM SYSTOR, 2012.
21. LILLIBRIDGE, M., ESHGHI, K., AND BHAGWAT, D. Improving restore speed for backup systems that use inline chunk-based deduplication. In Proc. USENIX FAST, 2013.
22. LILLIBRIDGE, M., ESHGHI, K., BHAGWAT, D., DEOLALIKAR, V., TREZISE, G., AND CAMBLE, P. Sparse indexing: large scale, inline deduplication using sampling and locality. In Proc. USENIX FAST, 2009.
23. LIM, H., FAN, B., ANDERSEN, D. G., AND KAMINSKY, M. SILT: A memory-efficient, high-performance key-value store. In Proc. ACM SOSP, 2011.
24. MANDAGERE, N., ZHOU, P., SMITH, M. A., AND UTTAMCHANDANI, S. Demystifying data deduplication. In Proc. ACM/IFIP/USENIX Middleware, 2008.
25. MEISTER, D., KAISER, J., AND BRINKMANN, A. Block locality caching for data deduplication. In Proc. ACM SYSTOR, 2013.
26. MEISTER, D., KAISER, J., BRINKMANN, A., CORTES, T., KUHN, M., AND KUNKEL, J. A study on data deduplication in hpc storage systems. In Proc. IEEE SC, 2012.
27. MEYER, D. T., AND BOLOSKY, W. J. A study of practical deduplication. In Proc. USENIX FAST, 2011.
28. MIN, J., YOON, D., AND WON, Y. Efficient deduplication techniques for modern backup operation. Computers, IEEE Transactions on 60, 6 (June 2011), 824–840.
29. NAM, Y., LU, G., PARK, N., XIAO, W., AND DU, D. H. Chunk fragmentation level: An effective indicator for read performance degradation in deduplication storage. In Proc. IEEE HPCC, 2011.
30. NAM, Y. J., PARK, D., AND DU, D. H. Assuring demanded read performance of data deduplication storage with backup datasets. In Proc. IEEE MASCOTS, 2012.
31. PATTERSON, D. A., GIBSON, G., AND KATZ, R. H. A case for redundant arrays of inexpensive disks (RAID). In Proc. ACM SIGMOD, 1988.
32. PAULO, J. A., AND PEREIRA, J. A survey and classification of storage deduplication systems. ACM Comput. Surv. 47, 1 (June 2014), 11:1–11:30.
33. QUINLAN, S., AND DORWARD, S. Venti: a new approach to archival storage. In Proc. USENIX FAST, 2002.
34. STRZELCZAK, P., ADAMCZYK, E., HERMAN-IZYCKA, U., SAKOWICZ, J., SLUSARCZYK, L., WRONA, J., AND DUBNICKI, C. Concurrent deletion in a distributed content-addressable storage system with global deduplication. In Proc. USENIX FAST, 2013.
35. TARASOV, V., JAIN, D., KUENNING, G., MANDAL, S., PALANISAMI, K., SHILANE, P., TREHAN, S., AND ZADOK, E. Dmdedup: Device mapper target for data deduplication. In 2014 Ottawa Linux Symposium.
36. WALLACE, G., DOUGLIS, F., QIAN, H., SHILANE, P., SMALDONE, S., CHAMNESS, M., AND HSU, W. Characteristics of backup workloads in production systems. In Proc. USENIX FAST, 2012.
37. WILDANI, A., MILLER, E. L., AND RODEH, O. Hands: A heuristically arranged non-backup in-line deduplication system. In Proc. IEEE ICDE, 2013.
38. XIA, W., JIANG, H., FENG, D., AND HUA, Y. Silo: a similarity-locality based near-exact deduplication scheme with low ram overhead and high throughput. In Proc. USENIX ATC, 2011.
39. ZHU, B., LI, K., AND PATTERSON, H. Avoiding the disk bottleneck in the data domain deduplication file system. In Proc. USENIX FAST, 2008.