A flexible QoS fortified distributed key-value storage system for the cloud

Proceedings - 2015 IEEE International Conference on Big Data, IEEE Big Data 2015

Volumn , Issue , 2015, Pages 515-522

  Li, Tonglin ^a   Wang, Ke ^b   Zhao, Dongfang ^a   Qiao, Kan ^c   Sadooghi, Iman ^a   Zhou, Xiaobing ^d   Raicu, Ioan ^a  

^a ILLINOIS INSTITUTE OF TECHNOLOGY   (United States)

^b INTEL CORPORATION   (United States)

^c GOOGLE INC   (United States)

^d Hortonworks ^*  (United States)

Author keywords

Distributed key value store; Distributed systems; NoSQL database; QoS; Request batching

Indexed keywords

DIGITAL STORAGE; DISTRIBUTED DATABASE SYSTEMS; QUALITY OF SERVICE;

DISTRIBUTED KEY-VALUE STORES; DISTRIBUTED SYSTEMS; LARGE-SCALE APPLICATIONS; NOSQL DATABASE; PERFORMANCE REQUIREMENTS; QOS (QUALITY OF SERVICE); RELATIONAL DATABASE; REQUEST BATCHING;

BIG DATA;

EID: 84963718266     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/BigData.2015.7363794     Document Type: Conference Paper

References (48)

1. B. Atikoglu, Y. Xu, E. Frachtenberg, S. Jiang, and M. Paleczny, "Workload analysis of a large-scale key-value store, " SIGMETRICS '12, 2012.
2. J. C. Corbett, J. Dean, and M. Epstein, "Spanner: Google's globally-distributed database, " OSDI, 2012.
3. D. B. Terry, V. Prabhakaran, R. Kotla, M. Balakrishnan, M. K. Aguilera, and H. Abu-Libdeh, "Consistency-based service level agreements for cloud storage, " SOSP '13.
4. K. Wang, A. Kulkarni, M. Lang, D. Arnold, and I. Raicu, "Towards scalable distributed workload manager with monitoring-based weakly consistent resource stealing, " ACM HPDC '15, 2015.
5. K. Wang, A. Kulkarni, M. Lang, D. Arnold, and I. Raicu, "Exploring the design tradeoffs for extreme-scale high-performance computing system software, " in IEEE Transaction on Parallel and Distributed Systems (TPDS), 2015.
6. D. Zhao, Z. Zhang, X. Zhou, T. Li, K. Wang, D. Kimpe, P. Carns, R. Ross, and I. Raicu, "Fusionfs: Towards supporting data-intensive scientific applications on extreme-scale high-performance computing systems, " in Big Data, 2014 IEEE International Conference on.
7. D. Zhao, C. Shou, T. Malik, and I. Raicu, "Distributed data provenance for largescale data-intensive computing, " CLUSTER, 2013.
8. D. Zhao, N. Liu, D. Kimpe, R. Ross, X.-H. Sun, and I. Raicu, "Towards exploring data-intensive scientific applications at extreme scales through systems and simulations, " TPDS, pp. 1-1, 2015.
9. I. Sadooghi, K. Wang, S. Srivastava, D. Patel, D. Zhao, T. Li, and I. Raicu, "Fabriq: Leveraging distributed hash tables towards distributed publish-subscribe message queues, " IEEE/ACM International Symposium on Big Data Computing (BDC), 2015.
10. K. Wang, X. Zhou, T. Li, M. Lang, and I. Raicu, "Optimizing load balancing and data-locality with data-aware scheduling, " IEEE BigData'14.
11. K. Wang, X. Zhou, H. Chen, M. Lang, and I. Raicu, "Next generation job management systems for extreme-scale ensemble computing, " HPDC '14, 2014.
12. T. Li, R. Verma, X. Duan, H. Jin, and I. Raicu, "Exploring distributed hash tables in highend computing, " SIGMETRICS Performance Evaluation Review, 2011.
13. T. Li, X. Zhou, K. Brandstatter, D. Zhao, K. Wang, A. Rajendran, Z. Zhang, and I. Raicu, "ZHT: A light-weight reliable persistent dynamic scalable zero-hop distributed hash table, " IPDPS '13.
14. P. Xiong, H. Hacigumus, and J. F. Naughton, "A software-defined networking based approach for performance management of analytical queries on distributed data stores, " SIGMOD '14, 2014.
15. H. Chetto, M. Silly, and T. Bouchentouf, "Dynamic scheduling of real-time tasks under precedence constraints, " Real Time Systems, The International Journal of Time-Critical Computing Systems, 1990.
16. M. Berg, F. V. D. D. Schouten, and J. Jansen, "Optimal batch provisioning to customers subject to a delay-limit, " Manage. Sci., vol. 44, pp. 684-697, May 1998.
17. T. Cucinotta, "Access control for adaptive reservations on multi-user systems, " in Real-Time and Embedded Technology and Applications Symposium, 2008. RTAS '08. IEEE, 2008.
18. H. Lim, D. Han, D. G. Andersen, and M. Kaminsky, "Mica: A holistic approach to fast in-memory key-value storage, " NSDI'14.
19. K. Wang, A. Kulkarni, M. Lang, D. Arnold, and I. Raicu, "Using simulation to explore distributed key-value stores for extreme-scale system services, " SC '13, 2013.
20. T. Chiba, M. den Burger, T. Kielmann, and S. Matsuoka, "Dynamic load-balanced multicast for data-intensive applications on clouds, " CCGrid 2010, pp. 5-14, May 2010.
21. N. Liu, J. Cope, P. Carns, C. Carothers, R. Ross, G. Grider, A. Crume, and C. Maltzahn, "On the role of burst buffers in leadership-class storage systems, " MSST'12, 2012.
22. K. Sato, H. Sato, and S. Matsuoka, "A model-based algorithm for optimizing i/o intensive applications in clouds using vm-based migration, " CCGRID '09, pp. 466-471, May 2009.
23. W. Frings, F. Wolf, and V. Petkov, "Scalable massively parallel i/o to tasklocal files, " in High Performance Computing Networking, Storage and Analysis, Proceedings of the Conference on, pp. 1-11, Nov 2009.
24. X. Ouyang, D. Nellans, R. Wipfel, D. Flynn, and D. Panda, "Beyond block i/o: Rethinking traditional storage primitives, " HPCA '11, pp. 301-311, Feb 2011.
25. M. Jung, J. Shalf, and M. Kandemir, "Design of a large-scale storage-class rram system, " ICS '13, (New York, NY, USA), pp. 103-114, ACM, 2013.
26. T. Chen and K. Taura, "Paralite: Supporting collective queries in database system to parallelize user-defined executable, " CCGRID '12, (Washington, DC, USA), pp. 474-481, IEEE Computer Society, 2012.
27. T. Li, X. Zhou, K. Wang, D. Zhao, I. Sadooghi, Z. Zhang, and I. Raicu, "A convergence of key-value storage systems from clouds to supercomputers, " Concurr. Comput. : Pract. Exper. (CCPE), 2015.
28. T. Li, "A convergence of NoSQL storage systems from clouds to supercomputers, " Illinois Institute of Technology, PhD Proposal, 2014.
29. T. Li and I. Raicu, "Distributed nosql storage for extreme-scale system services, " in IEEE/ACM Supercomputing PhD Showcase, IEEE/ACM, 2015.
30. I. Sadooghi, S. Palur, A. Anthony, I. Kapur, K. Belagodu, P. Purandare, K. Ramamurty, K. Wang, and I. Raicu, "Achieving efficient distributed scheduling with message queues in the cloud for many-task computing and high-performance computing, " CCGrid'14, 2014.
31. K. Wang, N. Liu, I. Sadooghi, X. Yang, X. Zhou, T. Li, M. Lang, X.-H. Sun, and I. Raicu, "Overcoming Hadoop scaling limitations through distributed task execution, " IEEE Cluster'15, 2015.
32. K. Wang, K. Qiao, I. Sadooghi, X. Zhou, T. Li, M. Lang, and I. Raicu, "Loadbalanced and locality-aware scheduling for data-intensive workloads at extreme scales, " Concurrency and Computation: Practice and Experience, 2015.
33. T. Li, I. Raicu, and L. Ramakrishnan, "Scalable state management for scientific applications in the cloud, " BigData Congress '14.
34. T. Li, K. Keahey, R. Sankaran, P. Beckman, and I. Raicu, "A cloud-based interactive data infrastructure for sensor networks, " IEEE/ACM Supercomputing/SC'14.
35. T. Li, K. Keahey, K. Wang, D. Zhao, and I. Raicu, "A dynamically scalable cloud data infrastructure for sensor networks, " ACM ScienceCloud 15.
36. T. Li, C. Ma, J. Li, X. Zhou, K. Wang, D. Zhao, I. Sadooghi, and I. Raicu, "GRAPH/Z: A key-value store based scalable graph processing system, " Cluster'15.
37. I. Sadooghi, J. Hernandez Martin, T. Li, K. Brandstatter, Y. Zhao, K. Maheshwari, T. Pais Pitta de Lacerda Ruivo, S. Timm, G. Garzoglio, and I. Raicu, "Understanding the performance and potential of cloud computing for scientific applications, " 2015.
38. I. Sadooghi, D. Zhao, T. Li, and I. Raicu, "Understanding the cost of cloud computing and storage, " GCASR, 2012.
39. Z. Wu, M. Butkiewicz, D. Perkins, E. Katz-Bassett, and H. V. Madhyastha, "Spanstore: Cost-effective geo-replicated storage spanning multiple cloud services, " SOSP '13.
40. Y. Mao, E. Kohler, and R. T. Morris, "Cache craftiness for fast multicore key-value storage, " EuroSys '12.
41. P. Wang, G. Sun, S. Jiang, J. Ouyang, S. Lin, C. Zhang, and J. Cong, "An efficient design and implementation of lsm-tree based key-value store on open-channel ssd, " EuroSys '14.
42. H. Lim, B. Fan, D. G. Andersen, and M. Kaminsky, "Silt: A memory-efficient, high-performance key-value store, " SOSP '11, 2011.
43. R. Nishtala, H. Fugal, S. Grimm, and M. Kwiatkowski, "Scaling memcache at facebook, " in Proceedings of the USENIX Conference on Networked Systems Design and Implementation, NSDI, (Lombard, IL), pp. 385-398, 2013.
44. "DynamoDB. " http://aws. Amazon. com/dynamodb/. Accessed: 2014-12-15.
45. High Scalability Blog, "Facebook's Memcached multiget hole: More machines != more capacity. " http://highscalability. com/blog/2009/10/26/ facebooks-memcached-multiget-hole-more-machines-more-capacit. html. Accessed: 2014-12-15.
46. D. Cheng, Y. Guo, and X. Zhou, "Self-tuning batching with dvfs for improving performance and energy efficiency in servers, " MASCOTS '13, (Washington, DC, USA), pp. 40-49, IEEE Computer Society, 2013.
47. Y. Wang and X. Wang, "Virtual batching: Request batching for server energy conservation in virtualized data centers, " IEEE Trans. Parallel Distrib. Syst., vol. 24, pp. 1695-1705, Aug. 2013.
48. C. Stewart, A. Chakrabarti, and R. Griffith, "Zoolander: Efficiently meeting very strict, low-latency slos, " in Proceedings of the 10th International Conference on Autonomic Computing (ICAC 13), (San Jose, CA), pp. 265-277, USENIX, 2013.