A highly parallel GPU-based hash accelerator for a data deduplication system

Proceedings of the IASTED International Conference on Parallel and Distributed Computing and Systems

Volumn , Issue , 2009, Pages 268-275

  Li, Xin ^a   Lilja, David J ^a  

^a University of Minnesota   (United States)

Author keywords

CUDA; Deduplication system; GPU computing; Hash computing

Indexed keywords

DATA ORGANIZATION; DATA SEGMENT; DATA STORAGE SYSTEMS; DEDUPLICATION; GPU COMPUTING; GPU PROCESSING; HASH ALGORITHM; HASH COMPUTATION; HIERARCHICAL DATA STRUCTURE; MEMORY MODELS; NEW APPROACHES; REDUNDANT DATA; SMALL SIZE; STORAGE SPACES; STORAGE SYSTEMS;

DATA STORAGE EQUIPMENT; DATA STRUCTURES; HASH FUNCTIONS;

DATA PROCESSING;

EID: 77952364312     PISSN: 10272658     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (23)

1. "Deduplication." [Online]. Available: http://www. chicorporation.com.
2. B. Zhu, K. Li, and H. Patterson, "Avoiding the disk bottleneck in the data domain deduplication file system," in FAST'08: Proceedings of the 16th USENIX Conference on File and Storage Technologies. Berkeley, CA, USA: USENIX Association, 2008, pp. 1-14.
3. U. Manber, "Finding similar files in a large file system," in WTEC'94: Proceedings of the USENIX Winter 1994 Technical Conference on USENIX Winter 1994 Technical Conference. Berkeley, CA, USA: USENIX Association, 1994, pp. 2-2.
4. S. Brin, J. Davis, and H. Garćia-Molina, "Copy detection mechanisms for digital documents," in SIGMOD '95: Proceedings of the 1995 ACM SIGMOD international conference on Management of data. New York, NY, USA: ACM, 1995, pp. 398-409.
5. S. C. Rhea, R. Cox, and A. Pesterev, "Fast, inexpensive content-addressed storage in foundation." in USENIX Annual Technical Conference, R. Isaacs and Y. Zhou, Eds. USENIX Association, 2008, pp. 143-156. [Online]. Available: http://dblp.uni-trier.de/db/conf/usenix/ usenix2008.html#RheaCP08
6. B. Lewis, "Deduplication comes of age," White paper, NetApp, Sep. 2007. [Online]. Available: http://www.netapp.com/us/communities/tech-ontap/ dedupe-0708.html
7. Z. Fan, F. Qiu, A. Kaufman, and S. Yoakum-Stover, "Gpu cluster for high performance computing," in SC '04: Proceedings of the 2004 ACM/IEEE conference on Supercomputing. Washington, DC, USA: IEEE Computer Society, 2004, p. 47.
8. I. Buck, T. Foley, D. Horn, J. Sugerman, K. Fatahalian, M. Houston, and P. Hanrahan, "Brook for gpus: stream computing on graphics hardware," in SIGGRAPH '04: ACM SIGGRAPH 2004 Papers. New York, NY, USA: ACM, 2004, pp. 777-786.
9. C. I. Rodrigues, D. J. Hardy, J. E. Stone, K. Schulten, and W.-M. W. Hwu, "Gpu acceleration of cutoff pair potentials for molecular modeling applications," in CF '08: Proceedings of the 5th conference on Computing frontiers. New York, NY, USA: ACM, 2008, pp. 273-282.
10. J. Nickolls, I. Buck, M. Garland, and K. Skadron, "Scalable parallel programming with cuda," in SIGGRAPH '08: ACM SIGGRAPH 2008 classes. New York, NY, USA: ACM, 2008, pp. 1-14.
11. S. Al-Kiswany, A. Gharaibeh, E. Santos-Neto, G. Yuan, and M. Ripeanu, "Storegpu: exploiting graphics processing units to accelerate distributed storage systems," in HPDC '08: Proceedings of the 17th international symposium on High performance distributed computing. New York, NY, USA: ACM, 2008, pp. 165-174.
12. S. Al-Kiswany, A. Gharaibeh, E. Santos-Neto, and M. Ripeanu, "On gpu's viability as a middleware accelerator," Cluster Computing. [Online]. Available: http://dx.doi.org/10.1007/s10586-009-0076-0.
13. Secure Hash Standard. Washington: National Institute of Standards and Technology, 2002, federal Information Processing Standard 180-2. [Online]. Available: http://csrc.nist.gov/publications/fips/.
14. M. Curry, A. Skjellum, H. Ward, and R. Brightwell, "Arbitrary dimension reed-solomon coding and decoding for extended raid on gpus," in PDSW'08: the 3rd Petascale Data Storage Workshop, Nov. 2008, pp. 1-3.
15. D. C. no Díz, D. Moser, A. Schoenegger, S. Pruggnaller, and A. S. Frangakis, "Performance evaluation of image processing algorithms on the gpu," Journal of Structural Biology, vol. 164, no. 1, pp. 153 -160, 2008. [Online]. Available: http://www. sciencedirect.com/science/article/B6WM5- 4T2DKVN-1/2/ce152ac8918d332de72c28d25cc2cdf2.
16. N. Satish, M. Harris, and M. Garland, "Designing efficient sorting algorithms for manycore gpus," in IPDPS '09: Proceedings of the 2009 IEEE International Symposium on Parallel & Distributed Processing. Washington, DC, USA: IEEE Computer Society, 2009, pp. 1-10.
17. M. Garland, S. Le Grand, J. Nickolls, J. Anderson, J. Hardwick, S. Morton, E. Phillips, Y. Zhang, and V. Volkov, "Parallel computing experiences with cuda," IEEE Micro, vol. 28, no. 4, pp. 13-27, 2008.
18. A. Muthitacharoen, B. Chen, and D. Mazières, "A lowbandwidth network file system," in SOSP '01: Proceedings of the eighteenth ACM symposium on Operating systems principles. New York, NY, USA: ACM, 2001, pp. 174-187.
19. K. Eshghi, M. Lillibridge, L. Wilcock, G. Belrose, and R. Hawkes, "Jumbo store: providing efficient incremental upload and versioning for a utility rendering service," in FAST'07: Proceedings of the 5th USENIX conference on File and Storage Technologies. Berkeley, CA, USA: USENIX Association, 2007, pp. 22-22.
20. M. O. Rabin, "Fingerprinting by random polynomials," Center for Research in Computing Technology, Harvard University, Tech. Rep. TR-15-81, 1981.
21. M. L. Curry, A. Skjellum, H. L. Ward, and R. Brightwell, "Accelerating reed-solomon coding in raid systems with gpus," in IPDPS '08: Proceedings of the 2008 IEEE International Symposium on Parallel & Distributed Processing, 2008, pp. 1-6. [Online]. Available: http://dx.doi.org/ 10.1109/IPDPS.2008.4536322.
22. I. Damgård, "A design principle for hash functions," in CRYPTO '89: Proceedings of the 9th Annual International Cryptology Conference on Advances in Cryptology. London, UK: Springer-Verlag, 1990, pp. 416-427.
23. NVIDIA CUDA Compute Unified ice Architecture Programming Guide, 2008. [Online]. Available: http:// eloper.download.nvidia.com/compute/cuda/1n0/NVIDIAn CUDAnProgrammingnGuiden1.0.pdf.