MICA: A holistic approach to fast in-memory key-value storage

Proceedings of the 11th USENIX Symposium on Networked Systems Design and Implementation, NSDI 2014

Volumn , Issue , 2014, Pages 429-444

  Lim, Hyeontaek ^a   Han, Dongsu ^b   Andersen, David G ^a   Kaminsky, Michael ^c  

^a CARNEGIE MELLON UNIVERSITY   (United States)

^b Korea Advanced Institute of Science and Technology (KAIST)   (South Korea)

^c INTEL CORPORATION   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

CONCURRENCY CONTROL; DATA HANDLING; DATA STRUCTURES; DIGITAL STORAGE; MICA; SYSTEMS ANALYSIS;

DATA STRUCTURE DESIGN; HIGH THROUGHPUT; HOLISTIC APPROACH; MULTI-CORE SYSTEMS; MULTICORE ARCHITECTURES; PARALLEL DATA ACCESS; STATE-OF-THE-ART SYSTEM; WRITE-INTENSIVE WORKLOADS;

COMPUTER ARCHITECTURE;

EID: 85073375532     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (48)

1. Ramcloud project wiki: clusterperf November 12, 2012, 2012. https://ramcloud.stanford.edu/wiki/display/ramcloud/clusterperf+Novemb er+12%2C+2012.
2. A. Ahmed, M. Aly, J. Gonzalez, S. Narayanamurthy, and A. J. Smola. Scalable inference in latent variable models. In Proceedings of the fifth ACM international conference on Web search and data mining, Feb. 2012.
3. A. Anand, C. Muthukrishnan, S. Kappes, A. Akella, and S. Nath. Cheap and large CAMs for high performance data-intensive networked systems. In Proc. 7th USENIX NSDI, Apr. 2010.
4. D. G. Andersen, J. Franklin, M. Kaminsky, A. Phanishayee, L. Tan, and V. Vasudevan. FAWN: A fast array of wimpy nodes. In Proc. 22nd ACM Symposium on Operating Systems Principles (SOSP), Oct. 2009.
5. B. Atikoglu, Y. Xu, E. Frachtenberg, S. Jiang, and M. Paleczny. Workload analysis of a large-scale key-value store. In Proceedings of the SIGMETRICS’12, June 2012.
6. M. Berezecki, E. Frachtenberg, M. Paleczny, and K. Steele. Many-core key-value store. http://gigaom2.files.wordpress.com/20 11/07/facebook- tilera- whitepaper.pdf, 2011.
7. M. Blott, K. Karras, L. Liu, K. Vissers, J. Bär, and Z. István. Achieving 10Gbps line-rate key-value stores with FPGAs. In Proceedings of the 5th USENIX Workshop on Hot Topics in Cloud Computing, June 2013.
8. CityHash. http://code.google.com/p/cityhash/, 2014.
9. B. Cooper, A. Silberstein, E. Tam, R. Ramakrishnan, and R. Sears. Benchmarking cloud serving systems with YCSB. In Proc. 1st ACM Symposium on Cloud Computing (SOCC), June 2010.
10. J. Dean and L. A. Barroso. The tail at scale. Communications of the ACM, 56(2):74–80, Feb. 2013.
11. G. DeCandia, D. Hastorun, M. Jampani, G. Kakulapati, A. Lakshman, A. Pilchin, S. Sivasubramanian, P. Vosshall, and W. Vogels. Dynamo: Amazon’s highly available key-value store. In Proc. 21st ACM Symposium on Operating Systems Principles (SOSP), Oct. 2007.
12. M. Dobrescu, N. Egi, K. Argyraki, B.-G. Chun, K. Fall, G. Iannaccone, A. Knies, M. Manesh, and S. Ratnasamy. RouteBricks: Exploiting parallelism to scale software routers. In Proc. 22nd ACM Symposium on Operating Systems Principles (SOSP), Oct. 2009.
13. R. Escriva, B. Wong, and E. G. Sirer. HyperDex: A distributed, searchable key-value store. In Proc. ACM SIG-COMM, Aug. 2012.
14. Facebook’s memcached multiget hole: More machines != more capacity. http://highscalability.com/blog/2009/10/26/facebooks-memcachedmultiget-hole-more-machines-more-cap acit.html, 2009.
15. B. Fan, D. G. Andersen, and M. Kaminsky. MemC3: Compact and concurrent memcache with dumber caching and smarter hashing. In Proc. 10th USENIX NSDI, Apr. 2013.
16. M. Flajslik and M. Rosenblum. Network interface design for low latency request-response protocols. In Proceedings of the 2013 USENIX Conference on Annual Technical Conference, June 2013.
17. J. Gray, P. Sundaresan, S. Englert, K. Baclawski, and P. J. Weinberger. Quickly generating billion-record synthetic databases. In Proceedings of the 1994 ACM SIGMOD international conference on Management of data, May 1994.
18. D. Han, A. Anand, F. Dogar, B. Li, H. Lim, M. Machado, A. Mukundan, W. Wu, A. Akella, D. G. Andersen, J. W. Byers, S. Seshan, and P. Steenkiste. XIA: Efficient support for evolvable internetworking. In Proc. 9th USENIX NSDI, Apr. 2012.
19. S. Han, K. Jang, K. Park, and S. Moon. PacketShader: a GPU-accelerated software router. In Proc. ACM SIG-COMM, Aug. 2010.
20. S. Han, S. Marshall, B.-G. Chun, and S. Ratnasamy. MegaPipe: a new programming interface for scalable network I/O. In Proceedings of the 10th USENIX conference on Operating Systems Design and Implementation, Oct. 2012.
21. M. D. Hill and A. J. Smith. Evaluating associativity in CPU caches. IEEE Transactions on Computers, 38(12): 1612–1630, Dec. 1989.
22. Intel. Intel Data Plane Development Kit (Intel DPDK). http://www.intel.com/go/dpdk, 2014.
23. Intel 82599 10 Gigabit Ethernet Controller: Datasheet. http://www.intel.com/content/www/us/en/ethernet-controllers/82599-10-gbecontroller-datasheet.html, 2014.
24. E. Jeong, S. Woo, M. Jamshed, H. Jeong, S. Ihm, D. Han, and K. Park. mTCP: a highly scalable user-level TCP stack for multicore systems. In Proc. 11th USENIX NSDI, Apr. 2014.
25. R. Kapoor, G. Porter, M. Tewari, G. M. Voelker, and A. Vahdat. Chronos: Predictable low latency for data center applications. In Proceedings of the Third ACM Symposium on Cloud Computing, Oct. 2012.
26. D. E. Knuth. The Art of Computer Programming, Volume 1: Fundamental Algorithms. Addison Wesley Longman Publishing Co., Inc., 1997. First edition published in 1968.
27. D. Lea. A memory allocator. http://g.oswego.e du/dl/html/malloc.html, 2000.
28. H. Lim, B. Fan, D. G. Andersen, and M. Kaminsky. SILT: A memory-efficient, high-performance key-value store. In Proc. 23rd ACM Symposium on Operating Systems Principles (SOSP), Oct. 2011.
29. K. Lim, D. Meisner, A. G. Saidi, P. Ranganathan, and T. F. Wenisch. Thin servers with Smart Pipes: Designing SoC accelerators for Memcached. In Proceedings of the 40th Annual International Symposium on Computer Architecture, June 2013.
30. Y. Mao, E. Kohler, and R. T. Morris. Cache craftiness for fast multicore key-value storage. In Proc. 7th ACM European Conference on Computer Systems (EuroSys), Apr. 2012.
31. Mellanox ConnectX-3 product brief. http://www.mellanox.com/related-docs/prod _adapter_cards/ConnectX3_EN_Card.pdf, 2013.
32. A distributed memory object caching system. http://memcached.org/, 2014.
33. Z. Metreveli, N. Zeldovich, and M. F. Kaashoek. CPHash: a cache-partitioned hash table. In Proceedings of the 17th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, Feb. 2012.
34. M. M. Michael. High performance dynamic lock-free hash tables and list-based sets. In Proceedings of the fourteenth annual ACM symposium on Parallel algorithms and architectures, July 2002.
35. C. Mitchell, Y. Geng, and J. Li. Using one-sided RDMA reads to build a fast, CPU-efficient key-value store. In Proceedings of the 2013 conference on USENIX Annual technical conference, June 2013.
36. R. Nishtala, H. Fugal, S. Grimm, M. Kwiatkowski, H. Lee, H. C. Li, R. McElroy, M. Paleczny, D. Peek, P. Saab, D. Stafford, T. Tung, and V. Venkataramani. Scaling Memcache at Facebook. In Proc. 10th USENIX NSDI, Apr. 2013.
37. D. Ongaro, S. M. Rumble, R. Stutsman, J. Ousterhout, and M. Rosenblum. Fast crash recovery in RAMCloud. In Proc. 23rd ACM Symposium on Operating Systems Principles (SOSP), Oct. 2011.
38. Optimized approximative pow() in C / C++. http://martin.ankerl.com/2012/01/25/optim ized-approximative-pow-in-c-and-cpp/, 2012.
39. R. Pagh and F. Rodler. Cuckoo hashing. Journal of Algorithms, 51(2):122–144, May 2004.
40. A. Pavlo, C. Curino, and S. Zdonik. Skew-aware automatic database partitioning in shared-nothing, parallel OLTP systems. In SIGMOD’12: Proceedings of the 2012 international conference on Management of Data, May 2012.
41. A. Pesterev, J. Strauss, N. Zeldovich, and R. T. Morris. Improving network connection locality on multicore systems. In Proceedings of the 7th ACM european conference on Computer Systems, Apr. 2012.
42. P. Purdom, S. Stigler, and T.-O. Cheam. Statistical investigation of three storage allocation algorithms. BIT Numerical Mathematics, 11(2), 1971.
43. L. Rizzo. netmap: a novel framework for fast packet I/O. In Proceedings of the 2012 USENIX conference on Annual Technical Conference, June 2012.
44. M. Stonebraker, S. Madden, D. J. Abadi, S. Harizopoulos, N. Hachem, and P. Helland. The end of an architectural era (it’s time for a complete rewrite). In Proc. VLDB, Sept. 2007.
45. S. Tu, W. Zheng, E. Kohler, B. Liskov, and S. Madden. Speedy transactions in multicore in-memory databases. In Proc. 24th ACM Symposium on Operating Systems Principles (SOSP), Nov. 2013.
46. VoltDB, the NewSQL database for high velocity applications. http://voltdb.com/, 2014.
47. P. Wilson, M. Johnstone, M. Neely, and D. Boles. Dynamic storage allocation: A survey and critical review. Lecture Notes in Computer Science, 1995.
48. D. Zhou, B. Fan, H. Lim, D. G. Andersen, and M. Kaminsky. Scalable, High Performance Ethernet Forwarding with CuckooSwitch. In Proc. 9th International Conference on emerging Networking EXperiments and Technologies (CoNEXT), Dec. 2013.