HyCache+: Towards scalable high-performance caching middleware for Parallel file systems

Proceedings - 14th IEEE/ACM International Symposium on Cluster, Cloud, and Grid Computing, CCGrid 2014

Volumn , Issue , 2014, Pages 267-276

  Zhao, Dongfang ^a   Qiao, Kan ^a   Raicu, Ioan ^a,b  

^a ILLINOIS INSTITUTE OF TECHNOLOGY   (United States)

^b ARGONNE NATIONAL LABORATORY   (United States)

Author keywords

distributed caching; heterogeneous storage; parallel and distributed file systems

Indexed keywords

DIGITAL STORAGE; GRID COMPUTING; MIDDLEWARE; SCHEDULING; SCHEDULING ALGORITHMS; SUPERCOMPUTERS;

DISTRIBUTED CACHING; DISTRIBUTED FILE SYSTEMS; HIGH PERFORMANCE SYSTEMS; HIGH SPEED INTERCONNECT; MASSIVELY PARALLELS; NETWORKING INFRASTRUCTURE; PARALLEL APPLICATION; PARALLEL FILE SYSTEM;

FILE ORGANIZATION;

EID: 84904548225     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/CCGrid.2014.11     Document Type: Conference Paper

References (56)

1. FUSE Project. http://fuse.sourceforge.net.
2. Google protocol buffers. http://code.google.com/p/protobuf/.
3. Intrepid. https://www.alcf.anl.gov/intrepid.
4. Titan. http://phys.org/news/2013-04-supercomputer-titan-world-fastest- storage.html.
5. S. Albers, N. Garg, and S. Leonardi. Minimizing stall time in single and parallel disk systems. In Proceedings of the thirtieth annual ACM symposium on Theory of computing, STOC '98, pages 454-462, 1998.
6. C. Ambühl and B. Weber. Parallel prefetching and caching is hard. In STACS, pages 211-221, 2004.
7. R. Bellman. Dynamic programming treatment of the travelling salesman problem. J. ACM, 9(1):61-63, Jan. 1962.
8. G. Calinescu, S. Kapoor, K. Qiao, and J. Shin. Stochastic strategic routing reduces attack effects. In Global Telecommunications Conference (GLOBECOM 2011), 2011 IEEE, pages 1-5, Dec 2011.
9. G. Calinescu and K. Qiao. Asymmetric topology control: Exact solutions and fast approximations. In IEEE International Conference on Computer Communications (INFOCOM '12), pages 783-791, March 2012.
10. P. Cao, E. W. Felten, A. R. Karlin, and K. Li. A study of integrated prefetching and caching strategies. SIGMETRICS Perform. Eval. Rev., 23(1):188-197, May 1995.
11. R. Fares, B. Romoser, Z. Zong, M. Nijim, and X. Qin. Performance evaluation of traditional caching policies on a large system with petabytes of data. In Networking, Architecture and Storage (NAS), 2012 IEEE 7th International Conference on, pages 227-234, 2012.
12. P. A. Freeman, D. L. Crawford, S. Kim, and J. L. Munoz. Cyberinfrastructure for science and engineering: Promises and challenges. Proceedings of the IEEE, 93(3):682-691, 2005.
13. S. Ghemawat, H. Gobioff, and S.-T. Leung. The Google file system. In Proceedings of the nineteenth ACM symposium on Operating systems principles, SOSP '03, pages 29-43, 2003.
14. P. Gonzalez-Ferez, J. Piernas, and T. Cortes. The ram enhanced disk cache project (redcap). In Proceedings of the 24th IEEE Conference on Mass Storage Systems and Technologies, MSST '07, pages 251-256, 2007.
15. Y. Gu and R. L. Grossman. Udt: Udp-based data transfer for high-speed wide area networks. Comput. Netw., 51(7):1777-1799, May 2007.
16. S. Huang, Q. Wei, J. Chen, C. Chen, and D. Feng. Improving flash-based disk cache with lazy adaptive replacement. In Mass Storage Systems and Technologies (MSST), 2013 IEEE 29th Symposium on, pages 1-10, 2013.
17. H. Jin, K. Qiao, X.-H. Sun, and Y. Li. Performance under failures of mapreduce applications. In Proceedings of the 2011 11th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, CCGRID '11, pages 608-609, 2011.
18. R. Li, R. Guo, Z. Xu, and W. Feng. A prefetching model based on access popularity for geospatial data in a cluster-based caching system. Int. J. Geogr. Inf. Sci., 26(10):1831-1844, Oct. 2012.
19. T. Li, X. Zhou, K. Brandstatter, D. Zhao, K. Wang, A. Ra-jendran, Z. Zhang, and I. Raicu. Zht: A light-weight reliable persistent dynamic scalable zero-hop distributed hash table. In Proceedings of the 2013 IEEE 27th International Symposium on Parallel and Distributed Processing, IPDPS '13, pages 775-787, 2013.
20. Z. Li, C. Wilson, Z. Jiang, Y. Liu, B. Zhao, C. Jin, Z.-L. Zhang, and Y. Dai. Efficient batched synchronization in dropbox-like cloud storage services. In Proceedings of the 14th International Middleware Conference, Middleware '13, Beijing, China, 2013.
21. J. Lin, Q. Lu, X. Ding, Z. Zhang, X. Zhang, and P. Sadayappan. Enabling software management for multicore caches with a lightweight hardware support. In Proceedings of the 2009 ACM/IEEE conference on Supercomputing, SC '09, pages 14:1-14:12, 2009.
22. R. Lohfert, J. Lu, and D. Zhao. Solving sql constraints by incremental translation to sat. In 21st International Conference on Industrial, Engineering and Other Applications of Applied Intelligent Systems, 2008.
23. A. Manzanares, X. Ruan, S. Yin, J. Xie, Z. Ding, Y. Tian, J. Majors, and X. Qin. Energy efficient prefetching with buffer disks for cluster file systems. In Proceedings of the 2010 39th International Conference on Parallel Processing, ICPP '10, pages 404-413, 2010.
24. D. Meister, J. Kaiser, and A. Brinkmann. Block locality caching for data deduplication. In Proceedings of the 6th International Systems and Storage Conference, SYSTOR '13, pages 15:1-15:12, 2013.
25. J. S. Plank, M. Blaum, and J. L. Hafner. Sd codes: Erasure codes designed for how storage systems really fail. In Proceedings of the 11th USENIX conference on File and Storage Technologies, FAST '13, 2013.
26. J. S. Plank, J. Luo, C. D. Schuman, L. Xu, and Z. Wilcox-O'Hearn. A performance evaluation and examination of open-source erasure coding libraries for storage. In Proccedings of the 7th conference on File and storage technologies, FAST '09, pages 253-265, 2009.
27. S. Podlipnig and L. Böszörmenyi. A survey of web cache replacement strategies. ACM Comput. Surv., 35(4):374-398, Dec. 2003.
28. K. Qiao, F. Tao, L. Zhang, and Z. Li. A ga maintained by binary heap and transitive reduction for addressing psp. In Intelligent Computing and Integrated Systems (ICISS), 2010 International Conference on, pages 12-15, Oct 2010.
29. I. Raicu. Many-task computing: bridging the gap between high-throughput computing and high-performance computing. PhD thesis, Chicago, IL, USA, 2009.
30. I. Raicu, I. Foster, A. Szalay, and G. Turcu. Astroportal: A science gateway for large-scale astronomy data analysis. In TeraGrid Conference, 2006.
31. I. Raicu, I. Foster, M. Wilde, Z. Zhang, K. Iskra, P. Beck-man, Y. Zhao, A. Szalay, A. Choudhary, P. Little, C. Moretti, A. Chaudhary, and D. Thain. Middleware support for many-task computing. Cluster Computing, 13(3):291-314, Sept. 2010.
32. I. Raicu, I. Foster, Y. Zhao, A. Szalay, P. Little, C. M. Moretti, A. Chaudhary, and D. Thain. Towards Data Intensive Many-Task Computing. book chapter in Data Intensive Distributed Computing: Challenges and Solutions for Large-Scale Information Management. IGI Global Publishers, 2011.
33. I. Raicu, I. T. Foster, Y. Zhao, P. Little, C. M. Moretti, A. Chaudhary, and D. Thain. The quest for scalable support of data intensive workloads in distributed systems. In Proceedings of the 18th ACM international symposium on High performance distributed computing, HPDC '09, pages 207-216, 2009.
34. I. Raicu, Y. Zhao, C. Dumitrescu, I. Foster, and M. Wilde. Falkon: a fast and light-weight task execution framework. In Proceedings of the 2007 ACM/IEEE conference on Supercomputing, SC '07, pages 43:1-43:12, 2007.
35. F. Schmuck and R. Haskin. Gpfs: A shared-disk file system for large computing clusters. In Proceedings of the 1st USENIX Conference on File and Storage Technologies, FAST '02, 2002.
36. L. Shi, Z. Liu, and L. Xu. Bwcc: A fs-cache based cooperative caching system for network storage system. In Proceedings of the 2012 IEEE International Conference on Cluster Computing, CLUSTER '12, pages 546-550, 2012.
37. C. Shou, D. Zhao, T. Malik, and I. Raicu. Towards a provenance-aware distributed filesystem. In 5th USENIX Workshop on the Theory and Practice of Provenance (TaPP '13), 2013.
38. K. Shvachko, H. Kuang, S. Radia, and R. Chansler. The hadoop distributed file system. In Proceedings of the 2010 IEEE 26th Symposium on Mass Storage Systems and Technologies (MSST), MSST '10, pages 1-10, 2010.
39. F. Tao, K. Qiao, L. Zhang, Z. Li, and A. Nee. GA-BHTR: an improved genetic algorithm for partner selection in virtual manufacturing. International Journal of Production Research, 50(8):2079-2100, 2012.
40. M. Wilde, I. Raicu, A. Espinosa, Z. Zhang, B. Clifford, M. Hategan, K. Iskra, P. Beckman, and I. Foster. Extreme-scale scripting: Opportunities for large task-parallel applications on petascale computers. In Scientific Discovery through Advanced Computing Conference, SciDAC '09, 2009.
41. C. Wu, X. He, Q. Cao, C. Xie, and S. Wan. Hint-k: An efficient multi-level cache using k-step hints. IEEE Transactions on Parallel and Distributed Systems, 99(PrePrints):1, 2013.
42. P. Xia, D. Feng, H. Jiang, L. Tian, and F. Wang. Farmer: a novel approach to file access correlation mining and evaluation reference model for optimizing peta-scale file system performance. In Proceedings of the 17th international symposium on High performance distributed computing, HPDC '08, pages 185-196, 2008.
43. Y. Xu, C. Xing, and L. Zhou. A cache replacement algorithm in hierarchical storage of continuous media object. In Advances in Web-Age Information Management: 5th International Conference, pages 157-166, 2004.
44. J. Yue, Y. Zhu, Z. Cai, and L. Lin. Energy and thermal aware buffer cache replacement algorithm. In Proceedings of the 2010 IEEE 26th Symposium on Mass Storage Systems and Technologies (MSST), MSST '10, pages 1-10, 2010.
45. D. Zhan, H. Jiang, and S. C. Seth. Locality & utility co-optimization for practical capacity management of shared last level caches. In Proceedings of the 26th ACM international conference on Supercomputing, ICS '12, pages 279-290, 2012.
46. D. Zhao, K. Burlingame, C. Debains, P. Alvarez-Tabio, and I. Raicu. Towards high-performance and cost-effective distributed storage systems with information dispersal algorithms. In Proceedings of the 2013 IEEE International Conference on Cluster Computing, CLUSTER '13, 2013.
47. D. Zhao and I. Raicu. Distributed file systems for exascale computing. In International Conference for High Performance Computing, Networking, Storage and Analysis (SC '12), 2012.
48. D. Zhao and I. Raicu. HyCache: A user-level caching middleware for distributed file systems. In Proceedings of the 2013 IEEE 27th International Symposium on Parallel and Distributed Processing Workshops and PhD Forum, IPDPSW '13, pages 1997-2006, 2013.
49. D. Zhao, C. Shou, T. Malik, and I. Raicu. Distributed data provenance for large-scale data-intensive computing. In Proceedings of the 2013 IEEE International Conference on Cluster Computing, CLUSTER '13, 2013.
50. D. Zhao and L. Yang. Incremental construction of neighborhood graphs for nonlinear dimensionality reduction. In Proceedings of the 18th International Conference on Pattern Recognition-Volume 03, ICPR '06, pages 177-180, 2006.
51. D. Zhao and L. Yang. Incremental isometric embedding of high-dimensional data using connected neighborhood graphs. IEEE Trans. Pattern Anal. Mach. Intell., 31(1):86-98, Jan. 2009.
52. D. Zhao, J. Yin, and I. Raicu. Improving the i/o throughput for data-intensive scientific applications with efficient compression mechanisms. In International Conference for High Performance Computing, Networking, Storage and Analysis (SC '13), 2013.
53. D. Zhao, D. Zhang, K. Wang, and I. Raicu. Exploring reliability of exascale systems through simulations. In Proceedings of the High Performance Computing Symposium, HPC '13, pages 1:1-1:9, 2013.
54. Y. Zhao, M. Hategan, B. Clifford, I. Foster, G. von Laszewski, V. Nefedova, I. Raicu, T. Stef-Praun, and M. Wilde. Swift: Fast, reliable, loosely coupled parallel computation. In Services, 2007 IEEE Congress on, pages 199-206, 2007.
55. Y. Zhao, I. Raicu, S. Lu, and X. Fei. Opportunities and challenges in running scientific workflows on the cloud. In IEEE International Conference on Network-based Distributed Computing and Knowledge Discovery, CyberC '11, 2011.
56. Z. Zhu and X. Zhang. Access-mode predictions for low-power cache design. IEEE Micro, 22(2):58-71, Mar. 2002.