Characterizing datasets for data deduplication in backup applications

IEEE International Symposium on Workload Characterization, IISWC'10

Volumn , Issue , 2010, Pages

  Park, Nohhyun ^a   Lilja, David J ^a  

^a University of Minnesota   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ARRIVAL RATES; COMPRESSION RATIOS; DATA SETS; DEDUPLICATION; DUPLICATE ELIMINATION; EVALUATION METRICS; INDEX CACHING; INDEX TABLE; STORAGE SYSTEMS; THROUGHPUT PERFORMANCE;

COMPRESSION RATIO (MACHINERY);

DATA COMPRESSION RATIO;

EID: 78751526844     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/IISWC.2010.5650369     Document Type: Conference Paper

References (32)

1. L. Aronovich, R. Asher, E. Bachmat, H. Bitner, M. Hirsch, and S. T. Klein, "The design of a similarity based deduplication system," in Proceedings of SYSTOR 2009: The Israeli Experimental Systems Conference, ser. SYSTOR '09. New York, NY, USA: ACM, 2009.
2. B. Zhu, K. Li, and H. Patterson, "Avoiding the disk bottleneck in the data domain deduplication file system," in Proceedings of the 6th USENIX Conference on File and Storage Technologies. Berkeley, CA, USA: USENIX Association, 2008.
3. D. Bobbarjung, S. Jagannathan, and C. Dubnicki, "Improving duplicate elimination in storage systems," ACM Transactions on Storage (TOS), vol. 2, no. 4, p. 448, 2006.
4. A. Muthitacharoen, B. Chen, and D. Mazières, "A low-bandwidth network file system," SIGOPS Oper. Syst. Rev., vol. 35, pp. 174-187, 2001.
5. D. Meister and A. Brinkmann, "Multi-level comparison of data deduplication in a backup scenario," in Proceedings of SYSTOR 2009: The Israeli Experimental Systems Conference, ser. SYSTOR '09. New York, NY, USA: ACM, 2009.
6. M. Lillibridge, K. Eshghi, D. Bhagwat, V. Deolalikar, G. Trezise, and P. Camble, "Sparse indexing: large scale, inline deduplication using sampling and locality," in Proceedings of the USENIX conference on File and storage technologies. Berkeley, CA, USA: USENIX Association, 2009.
7. L. You, K. Pollack, and D. Long, "Deep Store: An archival storage system architecture," in Data Engineering, 2005. ICDE 2005. Proceedings. 21st International Conference on, 2005, pp. 804-815.
8. M. Smith, J. Pieper, D. Gruhl, and L. Real, "IZO: applications of largewindow compression to virtual machine management," in Proceedings of the 22nd conference on Large installation system administration conference. USENIX Association, 2008, pp. 121-132.
9. K. Jin and E. L. Miller, "The effectiveness of deduplication on virtual machine disk images," in Proceedings of SYSTOR 2009: The Israeli Experimental Systems Conference, ser. SYSTOR '09. New York, NY, USA: ACM, 2009.
10. A. Clements, I. Ahmad, M. Vilayannur, J. Li, I. VMware, and M. CSAIL, "Decentralized deduplication in san cluster file systems," in Proceedings of the USENIX Annual Technical Conference, 2009.
11. E. Kruus, C. Ungureanu, and C. Dubnicki, "Bimodal Content Defined Chunking for Backup Streams," in Proceedings of the Eighth USENIX Conference on File and Storage Technologies, 2010.
12. K. Eshghi and H. Tang, "A framework for analyzing and improving content-based chunking algorithms," Hewlett-Packard Labs Technical Report TR, vol. 30, 2005.
13. N. Mandagere, P. Zhou, M. A. Smith, and S. Uttamchandani, "Demystifying data deduplication," in Proceedings of the ACM/IFIP/USENIX Middleware '08 Conference Companion, ser. Companion '08. New York, NY, USA: ACM, 2008.
14. D. Bhagwat, K. Eshghi, D. Long, and M. Lillibridge, "Extreme binning: Scalable, parallel deduplication for chunk-based file backup," in Proceedings of the IEEE International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems (MASCOTS), 2009.
15. B. H. Bloom, "Space/time trade-offs in hash coding with allowable errors," Commun. ACM, vol. 13, pp. 422-426, 1970.
16. R. Koller and R. Rangaswami, "I/o deduplication: Utilizing content similarity to improve i/o performance," in Proceedings of the 8th conference on file and storage technologies (FAST '10), 2010.
17. S. Rhea, R. Cox, and A. Pesterev, "Fast, inexpensive content-addressed storage in foundation," in Proceedings of the USENIX Annual Technical Conference, 2008.
18. J. Bonwick, M. Ahrens, V. Henson, M. Maybee, and M. Shellenbaum, "The Zettabyte File System," in FAST 2003: 2nd Usenix Conference on File and Storage Technologies, 2003.
19. M. Dillon, "The Hammer Filesystem," 2008.
20. A. Riska and E. Riedel, "Evaluation of disk-level workloads at different time scales," SIGMETRICS Perform. Eval. Rev., vol. 37, pp. 67-68, 2009.
21. A. W. Leung, S. Pasupathy, G. Goodson, and E. L. Miller, "Measurement and analysis of large-scale network file system workloads," in USENIX 2008 Annual Technical Conference on Annual Technical Conference. Berkeley, CA, USA: USENIX Association, 2008.
22. G. Soundararajan, V. Prabhakaran, M. Balakrishnan, and T. Wobber, "Extending SSD Lifetimes with Disk-Based Write Caches," in Proceedings of the USENIX Conference on File and Storage Technologies. FAST, 2010.
23. S. Lee, K. Ha, K. Zhang, J. Kim, and J. Kim, "FlexFS: A Flexible Flash File System for MLC NAND Flash Memory," in Proceedings of the USENIX Annual Technical Conference, San Diego, CA, 2009.
24. W. Wang, Y. Zhao, and R. Bunt, "HyLog: A High Performance Approach to Managing Disk Layout," in Proceedings of the 3rd USENIX Conference on File and Storage Technologies. USENIX Association, 2004, pp. 144-158.
25. U. Manber et al., "Finding similar files in a large file system," in Proceedings of the USENIX winter technical conference, 1994.
26. M. W. Storer, K. Greenan, D. D. Long, and E. L. Miller, "Secure data deduplication," in Proceedings of the 4th ACM international workshop on Storage security and survivability, ser. StorageSS '08. New York, NY, USA: ACM, 2008.
27. L. Eeckhout, H. Vandierendonck, and K. D. Bosschere, "Quantifying the impact of input data sets on program behavior and its applications," Journal of Instruction-Level Parallelism, vol. 5, pp. 1-33, 2003.
28. J. J. Yi, D. J. Lilja, and D. M. Hawkins, "A statistically rigorous approach for improving simulation methodology," in in Proceedings of the Ninth International Symposium on High Performance Computer Architecture (HPCA-9), 2002.
29. W. C. Hsu, H. Chen, P. C. Yew, and D.-Y. Chen, "On the predictability of program behavior using different input data sets," in Proceedings of the Sixth Annual Workshop on Interaction between Compilers and Computer Architectures, ser. INTERACT '02. Washington, DC, USA: IEEE Computer Society, 2002.
30. J. Douceur and W. Bolosky, "A large-scale study of file-system contents," ACM SIGMETRICS Performance Evaluation Review, vol. 27, no. 1, p. 70, 1999.
31. J. R. Douceur and W. J. Bolosky, "A large-scale study of file-system contents," SIGMETRICS Perform. Eval. Rev., vol. 27, pp. 59-70, 1999.
32. N. Agrawal, A. C. Arpaci-Dusseau, and R. H. Arpaci-Dusseau, "Generating realistic impressions for file-system benchmarking," Trans. Storage, vol. 5, pp. 16:1-16:30, December 2009.