Cost-Aware Streaming Workflow Allocation on Geo-Distributed Data Centers

IEEE Transactions on Computers

Volumn 66, Issue 2, 2017, Pages 256-271

  Chen, Wuhui ^a   Paik, Incheon ^a   Li, Zhenni ^a  

^a UNIVERSITY OF AIZU   (Japan)

Author keywords

Big data processing; geo distributed DCs; streaming workflow; streaming workflow allocation

Indexed keywords

BATCH DATA PROCESSING; COSTS; DATA HANDLING; HEURISTIC ALGORITHMS; INTEGER PROGRAMMING; LIFE CYCLE; SEMANTICS;

ALLOCATION ALGORITHM; ALLOCATION PROBLEMS; COST MINIMIZATION; DATA CENTER (DCS); DISTRIBUTED DATA; GEO-DISTRIBUTED DCS; MIXED INTEGER LINEAR PROGRAMMING; PERFORMANCE GAIN;

BIG DATA;

EID: 85009944756     PISSN: 00189340     EISSN: None     Source Type: Journal    
DOI: 10.1109/TC.2016.2595579     Document Type: Article

References (39)

1. L. Gu, D. Zeng, S. Guo, Y. Xiang, and J. Hu, "A general communication cost optimization framework for big data stream processing in geo-distributed data centers," in IEEE Trans. Comput., vol. 65, no. 1, pp. 19-29, Jan. 2016.
2. D. Sun, G. Fu, X. Liu, and H. Zhang, "Optimizing data streamgraph for big data streamcomputing in cloud datacenter environments," J. Advancements Comput. Technol., vol. 6, no. 5, pp. 53-65, 2014.
3. D. Sun, G. Zhang, S. Yang, W. Zheng, S. U. Khan, and K. Li, "Restream: Real-time and energy-efficient resource scheduling in big data stream computing environments," Inf. Sci., vol. 319, pp. 92-112, 2015.
4. A. Ghazal, et al., "BigBench: Towards an industry standard benchmark for big data analytics," in Proc. ACM SIGCOMM, 2013, pp. 1197-1208.
5. W. Hummer, B. Satzger, and S. Dustdar, "Elastic stream processing in the cloud," Wiley Interdisciplinary Rev. Data Mining Knowl. Discovery, vol. 3, no. 5, pp. 333-345, 2013.
6. Z. A. Mann, "Allocation of virtual machines in cloud data centers-A survey of problem models and optimization algorithms," ACM Comput. Surveys, vol. 48, no. 1, pp. 1-31, 2015.
7. F. L. Pires and B. Baran, "Virtual machine placement literature review," Polytechnic School, Nat. Univ. Asuncion, San Lorenzo, Paraguay, 2015. [Online]. Available: http://arxiv.org/abs/1ss506.01509
8. D. M. Batista, N. L. S. da Fonseca, and F. K. Miyazawa, "A set of schedulers for grid networks," in Proc. ACM Symp. Appl. Comput., 2007, pp. 209-213.
9. N. Bobroff, A. Kochut, and K. Beaty, "Dynamic placement of virtual machines for managing SLA violations," in Proc. 10th IFIP/IEEE Int. Symp. Integr. Netw. Manage., 2007, pp. 119-128.
10. A. Verma, P. Ahuja, and A. Neogi, "pMapper: power and migration cost aware application placement in virtualized systems," in Proc. 9th ACM/IFIP/USENIX Int. Conf. Middleware, 2008, pp. 243-264.
11. A. Beloglazov, J. Abawajy, and R. Buyya, "Energy-aware resource allocation heuristics for efficient management of data centers for cloud computing," Future Generation Comput. Syst., vol. 28, no. 5, pp. 755-768, 2012.
12. DataCenter Locations. (2013, Jul.). [Online]. Available: http://www.google.com/about/datacenters/inside/locations/index.html
13. B. Kaminski and P. Szufel, "On optimization of simulation execution on Amazon EC2 spot market," Simulation Modelling Practice Theory, vol. 58, no. 2, pp. 172-187, 2015.
14. K. Chen, Y. Xu, K. Xi, and H. Chao, "Intelligent virtual machine placement for cost efficiency in geo-distributed cloud systems," in Proc. IEEE 16th Int. Conf. Commun., 2013, pp. 3498-3503.
15. Interconnecting Geographically Dispersed Data Centers Using VPLS Design and System Assurance Guide, Cisco Systems, Inc., San Jose, CA, USA, 2009.
16. L. I. Kuncheva and I. Zliobaite, "On the window size for classification in changing environments," Intell. Data Anal., vol. 13, no. 6, pp. 861-872, 2009.
17. E. Olmezogullari and I. Ari, "Online association rule mining over fast data," in Proc. IEEE Int. Congress Big Data, 2013, pp. 110-117.
18. C.-G. Lee and Z. Ma, "The generalized quadratic assignment problem," Dept. Mech. Ind. Eng., Univ. Toronto, Toronto, Canada, Tech. Rep. MIE-OR TR2005-01, 2005.
19. S. Dasgupta, C. H. Papadimitriou, and U. V. Vazirani, Algorithms. New York, NY, USA: McGraw-Hill, 2008.
20. B. Hendrickson, Chaco: Software for partitioning graphs, 1994. [Online]. Available: http://www3.cs.stonybrook.edu/algorith/implement/chaco/implement.shtml
21. G. Karypis and V. Kumar, "A fast and high quality multilevel scheme for partitioning irregular graphs," SIAM J. Scientific Comput., vol. 20, no. 1, pp. 359-392, 1999.
22. D. Huang, D. Yang, H. Zhang, and L. Wu, "Energy-aware virtual machine placement in data centers," in Proc. IEEE Global Commun. Conf., 2012, pp. 3243-3249.
23. A. Anand, J. Lakshmi, and S. Nandy, "Virtual machine placement optimization supporting performance SLAs," in Proc. IEEE 5th Int. Conf. Cloud Comput. Technol. Sci., pp. 298-305, 2013.
24. W. Fang, X. Liang, S. Li, L. Chiaraviglio, and N. Xiong, "VMplanner: Optimizing virtual machine placement and traffic flow routing to reduce network power costs in cloud data centers," Comput. Netw., vol. 57, no. 1, pp. 179-196, 2013.
25. A. Beloglazov and R. Buyya, "Optimal online deterministic algorithms and adaptive heuristics for energy and performance efficient dynamic consolidation of virtual machines in cloud data centers," Concurrency Comput. Practice Experience, vol. 24, no. 13, pp. 1397-1420, 2012.
26. M. Guazzone, C. Anglano, and M. Canonico, "Exploiting VM migration for the automated power and performance management of green cloud computing systems," in Proc. 1st Int. Conf. Energy Efficient Data Centers, 2012, pp. 81-92.
27. M. Guazzone, C. Anglano, and M. Canonico, "Exploiting VM migration for the automated power and performance management of green cloud computing systems," Univ. Piemonte Orientale, Vercelli, VC, Italy, Tech. Rep. TR-INF-2012-04-02-UNIPMN, 2012.
28. T. A. L. Genez, L. F. Bittencourt, and E. R. M. Madeira, "Workflow scheduling for SaaS/PaaS cloud providers considering two SLA levels," in IEEE Proc. Netw. Operations Manage. Symp., 2012, pp. 906-912.
29. U. Lampe, M. Siebenhaar, R. Hans, D. Schuller, and R. Steinmetz, "Let the clouds compute: Cost-Efficient workload distribution in infrastructure clouds," in Proc. 9th Int. Conf. Economics Grids Clouds Syst. Services, 2012, pp. 91-101.
30. J. T. Piao and J. Yan, "A network-aware virtual machine placement and migration approach in cloud computing," in Proc. IEEE 9th Int. Conf. Grid Cooperative Comput., 2010, pp. 87-92.
31. L. Gu, D. Zeng, P. Li, and S. Guo, "Cost minimization for big data processing in geo-distributed data centers," IEEE Trans. Emerg. Topics Comput., vol. 2, no. 3, pp. 314-323, Sep. 2014.
32. L. F. Bittencourt, R. Sakellariou, and E. R. M. Madeira, "Using relative costs in workflow scheduling to cope with input data uncertainty," in Proc. 10th Int. Workshop Middleware Grids Clouds e-Sci., 2012, pp. 1-6.
33. D. Oliveira, K. A. Ocana, F. Baiao, and M. Mattoso, "A provenance-based adaptive scheduling heuristic for parallel scientific workflows in clouds," J. Grid Comput., vol. 10, no. 3, pp. 521-552, 2012.
34. J. W. Jiang, T. Lan, S. Ha, M. Chen, and M. Chiang, "Joint VM placement and routing for data center traffic engineering," in Proc. IEEE INFOCOM, 2012, pp. 2876-2880.
35. D. Riemer, L. Stojanovic, and N. Stojanovic, "SEPP: Semanticsbased management of fast data streams," in Proc. IEEE 7th Int. Conf. Service-Oriented Comput. Appl., 2014, pp. 113-118.
36. D. Riemer, F. Kaulfersch, R. Hutmacher, and L. Stojanovic, "DEBS grand challenge: StreamPipes: Solving the challenge with semantic stream processing pipelines," in Proc. 9th ACM Int. Conf. Distrib. Event-Based Syst., 2015, pp. 330-331.
37. S. G. Ahmad, C. S. Liew, M. M. Rafique, E. U. Munir, and S. U. Khan, "Data-intensive workflow optimization based on application task graph partitioning in heterogeneous computing systems," in Proc. IEEE 4th Int. Conf. Big Data Cloud Comput., 2014, pp. 129-136.
38. A. C. Zhou and B. He, "Transformation-based monetary cost optimizations for workflows in the cloud," IEEE Trans. Cloud Comput., vol. 2, no. 1, pp. 85-98, Jan. 2014.
39. C. S. Liew, M. Atkinson, J. van Hemert, and L. Han, "Towards optimizing distributed data streaming graphs using parallel streams," in Proc. Int. Workshop Data Intensive Distrib. Comput., 2010, pp. 725-736.