Primary data deduplication - Large scale study and system design

Proceedings of the 2012 USENIX Annual Technical Conference, USENIX ATC 2012

Volumn , Issue , 2019, Pages 285-296

  El Shimi, Ahmed ^a   Kalach, Ran ^a   Kumar, Ankit ^a   Oltean, Adi ^a   Li, Jin ^a   Sengupta, Sudipta ^a  

^a MICROSOFT   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

DATA HANDLING; RANDOM ACCESS STORAGE;

COMPRESSION APPROACH; DATA PARTITIONING; DE DUPLICATIONS; DISTRIBUTED FILE SERVER; LARGE-SCALE STUDIES; MULTI-NATIONAL CORPORATIONS; PRIMARY DATA; WINDOWS SERVERS;

WINDOWS OPERATING SYSTEM;

EID: 85050373541     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (26)

1. Linux SDFS. www.opendedup.org.
2. NetApp Deduplication and Compression. www.netapp.com/us/products/platform-os/dedupe.html.
3. Ocarina Networks. www.ocarinanetworks.com.
4. Permabit Data Optimization. www.permabit.com.
5. Windows Storage Server. technet.microsoft.com/ en- us/library/gg232683(WS.10).aspx.
6. ZFS Deduplication. blogs.oracle.com/bonwick/ entry/zfs_dedup.
7. BOLOSKY, W. J., CORBIN, S., GOEBEL, D., AND DOUCEUR, J. R. Single instance storage in windows 2000. In 4th USENIX Windows Systems Symposium (2000).
8. BRODER, A., AND MITZENMACHER, M. Network Applications of Bloom Filters: A Survey. In Internet Mathematics (2002).
9. CLEMENTS, A., AHMAD, I., VILAYANNUR, M., AND LI, J. Decentralized Deduplication in SAN Cluster File Systems. In USENIX ATC (2009).
10. DEBNATH, B., SENGUPTA, S., AND LI, J. ChunkStash: Speeding Up Inline Storage Deduplication Using Flash Memory. In USENIX ATC (2010).
11. DUBNICKI, C., GRYZ, L., HELDT, L., KACZMARCZYK, M., KILIAN, W., STRZELCZAK, P., SZCZEPKOWSKI, J., UNGUREANU, C.,, AND WELNICKI, M. HYDRAstor: a Scalable Secondary Storage. In FAST (2009).
12. EMC CORPORATION. EMC Centera: Content Addresses Storage System, Data Sheet, April 2002.
13. ESHGHI, K. A framework for analyzing and improving content-based chunking algorithms. HP Labs Technical Report HPL-2005-30 (R.1) (2005).
14. KRUUS, E., UNGUREANU, C., AND DUBNICKI, C. Bimodal Content Defined Chunking for Backup Streams. In FAST (2010).
15. LEUNG, A. W., PASUPATHY, S., GOODSON, G., AND MILLER, E. L. Measurement and analysis of large-scale network file system workloads. In USENIX ATC (2008).
16. LILLIBRIDGE, M., ESHGHI, K., BHAGWAT, D., DEOLALIKAR, V., TREZISE, G., AND CAMBLE, P. Sparse Indexing: Large Scale, Inline Deduplication Using Sampling and Locality. In FAST (2009).
17. MEYER, D. T., AND BOLOSKY, W. J. A study of practical deduplication. In FAST (2011).
18. MUTHITACHAROEN, A., CHEN, B., AND MAZIÈRES, D. A low-bandwidth network file system. In SOSP (2001).
19. NATIONAL INSTITUTE OF STANDARDS AND TECHNOLOGY, FIPS 180-1. Secure Hash Standard. U.S. Department of Commerce, 1995.
20. PAGH, R., AND RODLER, F. F. Cuckoo hashing. Journal of Algorithms 51, 2 (May 2004), 122-144.
21. QUINLAN, S., AND DORWARD, S. Venti: A New Approach to Archival Data Storage. In FAST (2002).
22. RABIN, M. O. Fingerprinting by Random Polynomials. Harvard University Technical Report TR-15-81 (1981).
23. SCHLEIMER, S., WILKERSON, D. S., AND AIKEN, A. Winnowing: Local algorithms for document fingerprinting. In ACM SIGMOD (2003).
24. SRINIVASAN, K., BISSON, T., GOODSON, G., AND VORUGANTI, K. iDedup: Latency-aware, Inline Data Deduplication for Primary Storage. In USENIX FAST (2012).
25. WELCH, T. A technique for high-performance data compression. IEEE Computer (June 1984).
26. ZHU, B., LI, K., AND PATTERSON, H. Avoiding the Disk Bottleneck in the Data Domain Deduplication File System. In FAST (2008).