Memcached design on high performance RDMA capable interconnects

Proceedings of the International Conference on Parallel Processing

Volumn , Issue , 2011, Pages 743-752

  Jose, Jithin ^a   Subramoni, Hari ^a   Luo, Miao ^a   Zhang, Minjia ^a   Huang, Jian ^a   Wasi Ur Rahman, Md ^a   Islam, Nusrat S ^a   Ouyang, Xiangyong ^a   Wang, Hao ^a   Sur, Sayantan ^a   Panda, Dhabaleswar K ^a  

^a The Ohio State University   (United States)

Author keywords

INCR; Infiniband; Memcached; RDMA; UCR

Indexed keywords

INCR; INFINIBAND; MEMCACHED; RDMA; UCR;

COMPUTER SYSTEM FIREWALLS; ETHERNET; ETHERS; QUEUEING NETWORKS; SEMANTICS; TELECOMMUNICATION NETWORKS; TRANSMISSION CONTROL PROTOCOL;

DESIGN;

EID: 80155183109     PISSN: 01903918     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICPP.2011.37     Document Type: Conference Paper

References (25)

1. "memcachefs: a memcache filesystem using FUSE," http://memcachefs.sourceforge.net/.
2. Todd Hoff, "Facebook's Memcached Multiget Hole: More Machines != More Capacity," http://highscalability.com/blog/2009/10/26/ facebooksmemcached-multiget-hole-more-machines-more-capacit.html.
3. "Memcached: High-Performance, Distributed Memory Object Caching System," http://memcached.org/.
4. OpenFabrics Alliance, http://www.openfabrics.org/.
5. MVAPICH2: High Performance MPI over InfiniBand, iWARP and RoCE, http://mvapich.cse.ohio-state.edu/.
6. InfiniBand Trade Association, http://www.infinibandta.org.
7. Paul Saab, "Facebook Engineering Notes: Scaling Memcached at Facebook," https://www.facebook.com/note.php?note id=39391378919.
8. The Apache Software Foundation, "The Apache Hadoop Project," http://hadoop.apache.org/.
9. J. Jose, M. Luo, S. Sur, and D. K. Panda, "Unifying UPC and MPI Runtimes: Experience with MVAPICH," in Fourth Conference on Partitioned Global Address Space Programming Model (PGAS '10).
10. M. Koop, S. Sur, Q. Gao, and D. K. Panda, "High Performance MPI Design using Unreliable Datagram for Ultra-Scale InfiniBand Clusters," in 21st ACM Int'l Conference on Supercomputing (ICS07), June 2007.
11. S. Sur, L. Chai, H.-W. Jin, and D. K. Panda, "Shared Receive Queue Based Scalable MPI Design for InfiniBand Clusters," in IEEE Int'l Parallel and Distributed Processing Symposium (IPDPS 2006), April 2006.
12. S. Sur, H. Wang, J. Huang, X. Ouyang, and D. K. Panda, "Can High-Performance Interconnects Benefit Hadoop Distributed File System?" in Workshop on Micro Architectural Support for Virtualization, Data Center Computing, and Clouds, in Conjunction with MICRO, Atlanta, GA, 2010.
13. H. Subramoni, P. Lai, M. Luo, and D. K. Panda, "RDMA over Ethernet - A Preliminary Study," in Proceedings of the 2009 Workshop on High Performance Interconnects for Distributed Computing (HPIDC'09).
14. P. Balaji, H. V. Shah, and D. K. Panda, "Sockets vs RDMA Interface over 10-Gigabit Networks: An In-depth analysis of the Memory Traffic Bottleneck," in Workshop on Remote Direct Memory Access (RDMA): Applications, Implementations, and Technologies (RAIT), in conjunction with IEEE Cluster, 2004.
15. RDMA Consortium, "Architectural Specifications for RDMA over TCP/IP," http://www.rdmaconsortium.org/.
16. B. Fitzpatrick, "Distributed caching with memcached," Linux Journal, vol. 2004, August 2004. [Online]. Available: http://portal.acm.org/ citation.cfm?id=1012889.1012894
17. J. Appavoo, A.Waterland, and e. a. Da Silva, "Providing a cloud network infrastructure on a supercomputer," in Proceedings of the 19th ACM Int'l Symposium on High Performance Distributed Computing, ser. HPDC '10. New York, NY, USA: ACM, 2010, pp. 385-394.
18. P. Balaji, S. Narravula, K. Vaidyanathan, S. Krishnamoorthy, J. Wu, and D. K. Panda, "Sockets Direct Protocol over InfiniBand in Clusters: Is it Beneficial?" in The Proceedings of the IEEE Int'l Symposium on Performance Analysis of Systems and Software, Austin, Texas, March 10-12 2004.
19. D. Goldenberg, M. Kagan, R. Ravid, and M. S. Tsirkin, "Transparently Achieving Superior Socket Performance using Zero Copy Socket Direct Protocol over 20 Gb/s InfiniBand Links," in in 2005 IEEE Int'l Conference on Cluster Computing (Cluster), 2005, pp. 1-10.
20. J. Dean and S. Ghemawat, "MapReduce: simplified data processing on large clusters," in In OSDI04: Proceedings of the 6th conference on Symposium on Opearting Systems Design and Implementation. USENIX Association, 2004.
21. K. Vaidyanathan, S. Narravula, P. Balaji, and D. K. Panda, "Designing Efficient Systems Services and Primitives for Next-Generation Data-Centers," in Workshop on NSF Next Generation Software(NGS) Program; held in conjunction with IPDPS, 2007.
22. T. von Eicken, D. E. Culler, S. C. Goldstein, and K. E. Schauser, "Active messages: a mechanism for integrated communication and computation," SIGARCH Comput. Archit. News, vol. 20, pp. 256-266, April 1992. [Online]. Available: http://doi.acm.org/10.1145/146628.140382
23. Editor: Dan Bonachea, "GASNet specification v1.1," U. C. Berkeley, Tech. Rep. UCB/CSD-02-1207, 2008.
24. IBM, "Message Passing Interface and Low-Level Application Programming Interface (LAPI)," http://www-03.ibm.com/systems/software/ parallel/index.html.
25. "libmemcached: Open Source C/C++ Client Library and Tools for Memcached," http://libmemcached.org/.