Workload-aware incremental repartitioning of shared-nothing distributed databases for scalable OLTP applications

Future Generation Computer Systems

Volumn 56, Issue , 2016, Pages 421-435

  Kamal, Joarder ^a   Murshed, Manzur ^b   Buyya, Rajkumar ^c  

^a MONASH UNIVERSITY   (Australia)

^b UNIVERSITY OF BALLARAT   (Australia)

^c UNIVERSITY OF MELBOURNE   (Australia)

Author keywords

Data migration; Distributed transactions; Incremental repartitioning; Load balance; OLTP; Shared nothing

Indexed keywords

ALGORITHMS; DATABASE SYSTEMS; MAPPING; NETWORK MANAGEMENT;

DATA MIGRATION; DISTRIBUTED TRANSACTION; INCREMENTAL REPARTITIONING; LOAD BALANCE; OLTP; SHARED NOTHING;

CLUSTERING ALGORITHMS;

EID: 84958764290     PISSN: 0167739X     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.future.2015.09.024     Document Type: Article

References (31)

1. IBM Data Hub, www.ibmbigdatahub.com/infographic/four-vs-big-data [Online: Last accessed 2015-03-02].
2. L. Gu, D. Zeng, P. Li, and S. Guo Cost minimization for big data processing in geo-distributed data centers IEEE Trans. Emerging Top. Comput. 2 3 2014 314 323
3. R. Johnson, I. Pandis, and A. Ailamaki Eliminating unscalable communication in transaction processing VLDB J. 23 1 2014 1 23
4. J. Gray, and A. Reuter Transaction Processing: Concepts and Techniques first ed. 1992 Morgan Kaufmann Publishers Inc. San Francisco, CA, USA
5. C. Curino, E. Jones, Y. Zhang, and S. Madden Schism: a workload-driven approach to database replication and partitioning Proc. VLDB Endow. 3 1-2 2010 48 57
6. A. Quamar, K.A. Kumar, and A. Deshpande SWORD: scalable workload-aware data placement for transactional workloads Proceedings of the 16th International Conference on Extending Database Technology EDBT'13 2013 ACM NY, USA 430 441
7. Vitess - scaling MySQL databases for large scale web services, https://github.com/youtube/vitess [Online: Last accessed 2015-03-02].
8. MySQL toolkit for managing billions of rows and hundreds of database machines. https://github.com/tumblr/jetpants [Online: Last accessed 2015-03-02].
9. A flexible sharding framework for creating eventually-consistent distributed datastores. https://github.com/twitter/gizzard/ [Online: Last accessed 2015-03-02].
10. Apache Giraph. http://giraph.apache.org/ [Online: Last accessed 2015-03-02].
11. P. Scheuermann, G. Weikum, and P. Zabback Data partitioning and load balancing in parallel disk systems VLDB J. 7 1 1998 48 66
12. M. Mehta, and D.J. DeWitt Data placement in shared-nothing parallel database systems VLDB J. 6 1 1997 53 72
13. K.A. Kumar, A. Quamar, A. Deshpande, and S. Khuller Sword: workload-aware data placement and replica selection for cloud data management systems VLDB J. 23 6 2014 845 870
14. A. Pavlo, C. Curino, and S. Zdonik Skew-aware automatic database partitioning in shared-nothing, parallel OLTP systems Proceedings of the 2012 ACM SIGMOD International Conference on Management of Data SIGMOD'12 2012 ACM NY, USA 61 72
15. T. Rafiq Elasca: Workload-aware elastic scalability for partition based database systems (Master's thesis) 2013 University of Waterloo Canada http://uwspace.uwaterloo.ca/handle/10012/7525
16. P. Tözün, I. Pandis, R. Johnson, and A. Ailamaki Scalable and dynamically balanced shared-everything oltp with physiological partitioning VLDB J. 22 2 2013 151 175
17. I. Pandis, R. Johnson, N. Hardavellas, and A. Ailamaki Data-oriented transaction execution Proc. VLDB Endow. 3 1-2 2010 928 939
18. J.M. Pujol, V. Erramilli, G. Siganos, X. Yang, N. Laoutaris, P. Chhabra, and P. Rodriguez The little engine(s) that could: Scaling online social networks IEEE/ACM Trans. Netw. (TON) 20 4 2012 1162 1175
19. A. Turk, R.O. Selvitopi, H. Ferhatosmanoglu, and C. Aykanat Temporal workload-aware replicated partitioning for social networks IEEE Trans. Knowl. Data Eng. 26 11 2014 2832 2845
20. B.P. Swift Data placement in a scalable transactional data store (Master's thesis) 2012 Vrije Universiteit Amsterdam, Netherland http://www.globule.org/publi/DPSTDS-master2012.pdf
21. Roaming in GSM Network. http://en.wikipedia.org/wiki/Roaming [Online: Last accessed 2015-03-02].
22. M. Nicola, and M. Jarke Performance modeling of distributed and replicated databases IEEE Trans. Knowl. Data Eng. 12 4 2000 645 672
23. X/Open XA. http://en.wikipedia.org/wiki/X/Open-XA [Online: Last accessed 2015-03-02].
24. Open Database Connectivity. http://en.wikipedia.org/wiki/Open-Database-Connectivity [Online: Last accessed 2015-03-02].
25. D. Karger, E. Lehman, T. Leighton, R. Panigrahy, M. Levine, and D. Lewin Consistent hashing and random trees: distributed caching protocols for relieving hot spots on the world wide web Proceedings of the Twenty-Ninth Annual ACM Symposium on Theory of Computing STOC'97 1997 ACM NY, USA 654 663
26. Mobile IP. http://en.wikipedia.org/wiki/Mobile-IP [Online: Last accessed 2015-03-02].
27. J. Kamal, M. Murshed, R. Buyya, Workload-aware incremental repartitioning of shared-nothing distributed databases for scalable cloud applications, in: 2014 IEEE/ACM 7th International Conference on Utility and Cloud Computing, UCC, 2014, pp. 213-222.
28. SSJ: Stochastic Simulation in Java. http://simul.iro.umontreal.ca/ssj/indexe.html [Online: Last accessed 2015-03-02].
29. TPC-C - on-line transaction processing benchmark. http://www.tpc.org/tpcc/ [Online: Last accessed 2015-03-02].
30. G. Karypis, and V. Kumar Multilevel k-way partitioning scheme for irregular graphs J. Parallel Distrib. Comput. 48 1 1998 96 129
31. G. Karypis, and V. Kumar Multilevel k-way hypergraph partitioning Proceedings of the 36th Annual ACM/IEEE Design Automation Conference DAC'99 1999 ACM NY, USA 343 348