Concurrent deletion in a distributed content-addressable storage system with global deduplication

Proceedings of the 11th USENIX Conference on File and Storage Technologies, FAST 2013

Volumn , Issue , 2013, Pages 161-174

  Strzelczak, Przemyslaw ^a   Adamczyk, Elzbieta ^a   Herman Izycka, Urszula ^a   Sakowicz, Jakub ^a   Slusarczyk, Lukasz ^a   Wrona, Jaroslaw ^a   Dubnicki, Cezary ^a  

^a Research Affiliates

Author keywords

[No Author keywords available]

Indexed keywords

DISTRIBUTED COMPUTER SYSTEMS; DISTRIBUTED DATABASE SYSTEMS;

DATA DE DUPLICATIONS; DISTRIBUTED SYSTEMS; FAILURE-TOLERANT; HIGH AVAILABILITY; PHYSICAL RESOURCES; REFERENCE COUNTING; RELIABLE DISTRIBUTED SYSTEMS; SYSTEM CONFIGURATIONS;

DIGITAL STORAGE;

EID: 84920191892     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (34)

1. ABD-EL-MALEK, M., II, W. V. C., CRANOR, C., GANGER, G. R., HENDRICKS, J., KLOSTERMAN, A. J., MESNIER, M. P., PRASAD, M., SALMON, B., SAMBASIVAN, R. R., SINNAMOHIDEEN, S., STRUNK, J. D., THERESKA, E., WACHS, M., AND WYLIE, J. J. Ursa Minor: Versatile Cluster-based Storage. In FAST’05: Proceedings of the 4th conference on USENIX Conference on File and Storage Technologies (2005).
2. ABDULLAHI, S. E., AND RINGWOOD, G. A. Garbage Collecting the Internet: A Survey of Distributed Garbage Collection. ACM Comput. Surv. 30, 3 (1998), pp. 330–373.
3. ADYA, A., BOLOSKY, W., CASTRO, M., CHAIKEN, R., CERMAK, G., DOUCEUR, J., HOWELL, J., LORCH, J., THEIMER, M., AND WATTENHOFER, R. FARSITE: Federated, Available, and Reliable Storage for an Incompletely Trusted Environment, 2002.
4. ANDROUTSELLIS-THEOTOKIS, S., AND SPINELLIS, D. A Survey of Peer-to-Peer Content Distribution Technologies. ACM Comput. Surv. 36, 4 (2004), pp. 335–371.
5. BHAGWAT, D., ESHGHI, K., LONG, D. D. E., AND LILLIBRIDGE, M. Extreme Binning: Scalable, Parallel Deduplication for Chunk-based File Backup. In Proceedings of the 17th IEEE International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems (MASCOTS 2009) (Sep 2009).
6. BOLOSKY, W. J., CORBIN, S., GOEBEL, D., AND DOUCEUR, J. R. Single Instance Storage in Windows 2000. In Proceedings of the 4th USENIX Windows Systems Symposium (2000), pp. 13–24.
7. CLEMENTS, A., AHMAD, I., VILAYANNUR, M., AND LI, J. Decentralized Deduplication in SAN Cluster File Systems. In Proceedings of the USENIX Annual Technical Conference (June 2009).
8. DIJKSTRA, E. W., LAMPORT, L., MARTIN, A. J., SCHOLTEN, C. S., AND STEFFENS, E. F. M. On-the-Fly Garbage Collection: An Exercise in Cooperation. Commun. ACM 21, 11 (1978), 966–975.
9. DONG, W., DOUGLIS, F., LI, K., PATTERSON, H., REDDY, S., AND SHILANE, P. Tradeoffs in Scalable Data Routing for Deduplication Clusters. In Proceedings of the 9th USENIX conference on File and Storage Technologies (Berkeley, CA, USA, 2011), FAST’11, USENIX Association, pp. 15–29.
10. DUBNICKI, C., GRYZ, L., HELDT, L., KACZMARCZYK, M., KILIAN, W., STRZELCZAK, P., SZCZEPKOWSKI, J., UNGUREANU, C., AND WELNICKI, M. HYDRAstor: A Scalable Secondary Storage. In FAST’09: Proceedings of the 7th USENIX Conference on File and Storage Technologies (Berkeley, CA, USA, 2009), USENIX Association, pp. 197–210.
11. EFSTATHOPOULOS, P., AND GUO, F. Rethinking Deduplication Scalability. In 2nd USENIX Workshop on Hot Topics in Storage and File Systems (Boston, MA, USA, June 2010).
12. EMC Avamar: Backup and Recovery with Global Deduplication, 2008. http://www.emc.com/avamar.
13. EMC Centera: Content Addressed Storage System, January 2008. http://www.emc.com/centera.
14. EMC Corporation: Data Domain Global Deduplication Array, 2011. http://www.datadomain.com/products/globaldeduplication-array.html.
15. ESHGHI, K., LILLIBRIDGE, M., WILCOCK, L., BELROSE, G., AND HAWKES, R. Jumbo Store: Providing Efficient Incremental Upload and Versioning for a Utility Rendering Service. In FAST ’07: Proceedings of the 5th USENIX conference on File and Storage Technologies (Berkeley, CA, USA, 2007), USENIX Association, p. 22.
16. Exagrid. http://www.exagrid.com.
17. GUNAWI, H. S., AGRAWAL, N., ARPACI-DUSSEAU, A. C., ARPACI-DUSSEAU, R. H., AND SCHINDLER, J. Deconstructing Commodity Storage Clusters. In Proceedings of the 32nd International Symposium on Computer Architecture (Madison, WI, June 2005).
18. GUO, F., AND EFSTATHOPOULOS, P. Building a High-performance Deduplication System. In Proceedings of the 2011 USENIX conference on USENIX annual technical conference (Berkeley, CA, USA, 2011), USENIX ATC’11, USENIX Association, p. 25.
19. HAEBERLEN, A., MISLOVE, A., AND DRUSCHEL, P. Glacier: Highly Durable, Decentralized Storage Despite Massive Correlated Failures. In NSDI’05: Proceedings of the 2nd conference on Symposium on Networked Systems Design & Implementation (Berkeley, CA, USA, 2005), USENIX Association, pp. 143–158.
20. HONG, B., AND LONG, D. D. E. Duplicate Data Elimination in a SAN File System. In Proceedings of the 21st IEEE / 12th NASA Goddard Conference on Mass Storage Systems and Technologies (MSST) (2004), pp. 301–314.
21. IBM. IBM ProtecTIER Deduplication Solution http://www-03.ibm.com/systems/storage/tape/protectier/
22. LILLIBRIDGE, M., ESHGHI, K., BHAGWAT, D., DEOLALIKAR, V., TREZIS, G., AND CAMBLE, P. Sparse Indexing: Large Scale, Inline Deduplication Using Sampling and Locality. In FAST’09: Proceedings of the 7th USENIX Conference on File and Storage Technologies (2009), pp. 111–123.
23. MEISTER, D., AND BRINKMANN, A. dedupv1: Improving Deduplication Throughput Using Solid State Drives (SSD). In Proceedings of the 26th IEEE Symposium on Massive Storage Systems and Technologies (MSST) (May 2010).
24. MEISTER, D., AND BRINKMANN, A. dedupv1: Improving Deduplication Throughput Using Solid State Drives (Technical Report). Tech. rep., April 2010.
25. PATTERSON, R. H. Incremental Garbage Collection of Data in a Secondary Storage. Patent, 11 2008. US 7451168.
26. PATTERSON, R. H. Probabilistic Summary Data Structure Based Encoding for Garbage Collection. Patent, 04 2010. US 7707166.
27. QUINLAN, S., AND DORWARD, S. Venti: A New Approach to Archival Storage. In FAST’02: Proceedings of the Conference on File and Storage Technologies (Berkeley, CA, USA, 2002), USENIX Association, pp. 89–101.
28. SEPATON Scalable Data Deduplication Solutions. http://sepaton.com/solutions/data-deduplication.
29. TODD, S. J., KILIAN, M., TEUGELS, T., MARIVOET, K., AND MATTHYS, F. Methods and Apparatus for Managing Deletion of Data. Patent, 04 2010. US 7698516.
30. UNGUREANU, C., ARANYA, A., GOKHALE, S., RAGO, S., ATKIN, B., BOHRA, A., DUBNICKI, C., AND CALKOWSKI, G. HydraFS: A High-throughput File System for the HYDRAstor Content-addressable Storage System. In FAST’10 (2010), pp. 225–239.
31. WEI, J., JIANG, H., ZHOU, K., AND FENG, D. Mad2: A Scalable High-throughput Exact Deduplication Approach for Network Backup Services. In Proceedings of the 26th IEEE Symposium on Massive Storage Systems and Technologies (MSST) (May 2010).
32. WILSON, P. R. Uniprocessor Garbage Collection Techniques. In IWMM’92: Proceedings of the International Workshop on Memory Management (London, UK, 1992), Springer-Verlag, pp. 1–42.
33. YOU, L. L., POLLACK, K. T., AND LONG, D. D. E. Deep Store: An Archival Storage System Architecture. In ICDE’05: Proceedings of the 21st International Conference on Data Engineering (Washington, DC, USA, 2005), IEEE Computer Society, pp. 804–815.
34. ZHU, B., LI, K., AND PATTERSON, H. Avoiding the Disk Bottleneck in the Data Domain Deduplication File System. In FAST’08: Proceedings of the 6th USENIX Conference on File and Storage Technologies (Berkeley, CA, USA, 2008), USENIX Association, pp. 1–14.