Generating realistic datasets for deduplication analysis

Proceedings of the 2012 USENIX Annual Technical Conference, USENIX ATC 2012

Volumn , Issue , 2019, Pages 261-272

  Tarasov, Vasily ^a   Mudrankit, Amar ^a   Buik, Will ^b   Shilane, Philip ^c   Kuenning, Geoff ^b   Zadok, Erez ^a  

^a STONY BROOK UNIVERSITY   (United States)

^b HARVEY MUDD COLLEGE   (United States)

^c GNSS Technology for Rail Transportation

Author keywords

[No Author keywords available]

Indexed keywords

FILE ORGANIZATION; METADATA;

ACCESS PATTERNS; COMPETING ALGORITHMS; DE DUPLICATIONS; GENERIC FRAMEWORKS; GENERIC MODELING; REALISTIC EVALUATIONS; RELATIVE ERRORS; STORAGE SYSTEMS;

DIGITAL STORAGE;

EID: 85077129387     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (25)

1. N. Agrawal, A. C. Arpaci-Dusseau, and R.H. Arpaci-Dusseau. Generating realistic impressions for file-system benchmarking. In Proceedings of the Seventh USENIX Conference on File and Storage Technologies (FAST'09), 2009.
2. N. Agrawal, W. J. Bolosky, J. R. Douceur, and J. R. Lorch. A five-year study of file-system metadata. In Proceedings of the Fifth USENIX Conference on File and Storage Technologies (FAST'07), 2007.
3. J. M. Bennett, M. A. Bauer, and D. Kinchlea. Characteristics of files in NFS environments. ACM SIGSMALL/PC Notes, 18(3-4):18-25, 1992.
4. M. Chamness. Capacity forecasting in a backup storage environment. In Proceedings of USENIX Large Installation System Administration Conference (LISA), 2011.
5. EMC Corporation. EMC Centra: content addressed storage systems. Product description guide, 2004.
6. W. Dong, F. Douglis, K. Li, H. Patterson, S. Reddy, and P. Shilane. Tradeoffs in scalable data routing for deduplication clusters. In Proceedings of the Nineth USENIX Conference on File and Storage Technologies (FAST'11), 2011.
7. A. B. Downey. The structural cause of file size distributions. In Proceedings of IEEE/ACM International Symposium on Modeling, Analysis and Simulation of Computer Telecommunications Systems (MASCOTS), 2001.
8. Filebench. http://filebench.sourceforge.net.
9. Advanced Storage Products Group. Identifying the hidden risk of data deduplication: how the HYDRAstor solution proactively solves the problem. Technical Report WP103-3 0709, NEC Corporation of America, 2009.
10. N. Jain, M. Dahlin, and R. Tewari. TAPER: tiered approach for eliminating redundancy in replica synchronization. In Proceedings of the Fourth USENIX Conference on File and Storage Technologies (FAST'05), 2005.
11. K. Jin and E. Miller. The effectiveness of deduplication on virtual machine disk images. In Proceedings of the Second ACM Israeli Experimental Systems Conference (SYSTOR'09), 2009.
12. R. Kothiyal, V. Tarasov, P. Sehgal, and E. Zadok. Energy and performance evaluation of lossless file data compression on server systems. In Proceedings of the Second ACM Israeli Experimental Systems Conference (SYSTOR'09), 2009.
13. A. W. Leung, S. Pasupathy, G. Goodson, and E. L. Miller. Measurement and analysis of large-scale network file system workloads. In Proceedings of the USENIX Annual Technical Conference (ATC'08), 2008.
14. A. Liguori and E. Hensbergen. Experiences with content addressable storage and virtual disks. In Proceedings of the Workshop on I/O Virtualization (WIOV), 2008.
15. D. Meyer and W. Bolosky. A study of practical deduplication. In Proceedings of the Nineth USENIX Conference on File and Storage Technologies (FAST'11), 2011.
16. NetApp. NetApp deduplication for FAS. Deployment and implementation, 4th revision. Technical Report TR-3505, NetApp, 2008.
17. N. Park and D. Lilja. Characterizing datasets for data deduplication in backup applications. In Proceedings of the IEEE International Symposium on Workload Characterization (IISWC), 2010.
18. N. Park, W. Xiao, K. Choi, and D. J. Lilja. A statistical evaluation of the impact of parameter selection on storage system benchmark. In Proceedings of the 7th IEEE International Workshop on Storage Network Architecture and Parallel I/Os (SNAPI), 2011.
19. D. Roselli, J. R. Lorch, and T. E. Anderson. A comparison of file system workloads. In Proceedings of the Annual USENIX Technical Conference, 2000.
20. M. Satyanarayanan. A study of file sizes and functional lifetimes. In Proceedings of the 8th ACM Symposium on Operating System Principles (SOSP'81), 1981.
21. K. A. Smith and M. I. Seltzer. File system aging-increasing the relevance of file system benchmarks. In Proceedings of the 1997 International Conference on Measurement and Modeling of Computer Systems (SIG-METRICS 1997), 1997.
22. W. Vogels. File system usage in Windows NT 4.0. In Proceedings of the 17th ACM Symposium on Operating Systems Principles, 1999.
23. G. Wallace, F. Douglis, H. Qian, P. Shilane, S. Smaldone, M. Chamness, and W. Hsu. Characteristics of backup workloads in production systems. In Proceedings of the Tenth USENIX Conference on File and Storage Technologies (FAST'12), 2012.
24. W. Xia, H. Jiang, D. Feng, and Y. Hua. SiLo: A similarity-locality based near-exact deduplication scheme with low RAM overhead and high throughput. In Proceedings of the USENIX Annual Technical Conference (ATC), 2011.
25. B. Zhu, K. Li, and H. Patterson. Avoiding the disk bottleneck in the data domain deduplication file system. In Proceedings of the Sixth USENIX Conference on File and Storage Technologies (FAST'08), 2008.