Re-Stream: Real-time and energy-efficient resource scheduling in big data stream computing environments

Information Sciences

Volumn 319, Issue , 2015, Pages 92-112

  Sun, Dawei ^a   Zhang, Guangyan ^a   Yang, Songlin ^a   Zheng, Weimin ^a   Khan, Samee U ^b   Li, Keqin ^c  

^a TSINGHUA UNIVERSITY   (China)

^b North Dakota State University   (United States)

^c STATE UNIVERSITY OF NEW YORK   (United States)

Author keywords

Big data; Critical path; Directed acyclic graph; Energy efficiency; Real time; Stream computing

Indexed keywords

BIG DATA; COMPUTATION THEORY; DIRECTED GRAPHS; ECONOMIC AND SOCIAL EFFECTS; ENERGY UTILIZATION; GRAPH THEORY; GREEN COMPUTING; OPTIMIZATION; RESPONSE TIME (COMPUTER SYSTEMS); SCHEDULING;

CRITICAL PATHS; DIRECTED ACYCLIC GRAPH (DAG); HIGH ENERGY EFFICIENCY; MATHEMATICAL RELATIONSHIP; OPTIMIZATION FRAMEWORK; REAL TIME; RESOURCE ALLOCATION SCHEMES; STREAM COMPUTING;

ENERGY EFFICIENCY;

EID: 84930870603     PISSN: 00200255     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.ins.2015.03.027     Document Type: Article

References (43)

1. A. Amini, T.Y. Wah, and H. Saboohi On density-based data streams clustering algorithms: a survey J. Comput. Sci. Technol. 29 1 2014 116 141
2. L. Aniello, R. Baldoni, and L. Querzoni Adaptive online scheduling in storm Proc. 7th ACM International Conference on Distributed Event-Based Systems, DEBS 2013 2013 ACM Press 207 218
3. J. Baliga, R.W.A. Ayre, K. Hinton, and R.S. Tucker Green cloud computing: balancing energy in processing, storage, and transport Proc. IEEE 99 1 2011 149 167
4. S. Baskiyar, and R. Abdel-Kader Energy aware DAG scheduling on heterogeneous systems Cluster Comput. 13 4 2010 373 383
5. A. Beloglazov, J. Abawajy, and R. Buyya Energy-aware resource allocation heuristics for efficient management of data centers for Cloud computing Future Gener. Comput. Syst. 28 5 2012 755 768
6. J. Chauhan, S.A. Chowdhury, and D. Makaroff Performance evaluation of Yahoo! S4: A first look Proc. 7th International Conference on P2P, Parallel, Grid, Cloud and Internet Computing, 3PGCIC 2012 2012 IEEE Press 58 65
7. M.I. Daoud, and N. Kharma A hybrid heuristic-genetic algorithm for task scheduling in heterogeneous processor networks J. Parall. Distr. Comput. 71 11 2011 1518 1531
8. M. Dayarathna, and S. Toyotaro Automatic optimization of stream programs via source program operator graph transformations Distr. Parall. Databases 31 4 2013 543 599
9. H. Demirkan, and D. Delen Leveraging the capabilities of service-oriented decision support systems: putting analytics and big data in cloud Decis. Support Syst. 55 1 2013 412 421
10. G. Guo, Y. Zhao, and G.Q. Yang Cooperation of multiple mobile sensors with minimum energy cost for mobility and communication Inform. Sci. 254 2014 69 82
11. G.J. Hornga, T.Y. Changb, and S.T. Chengc An effective node-selection scheme for the energy efficiency of solar-powered WSNs in a stream environment Exp. Syst. Appl. 41 7 2014 3143 3156
12. C.H. Hsu, K.D. Slagter, S.C. Chen, and Y.C. Chung Optimizing energy consumption with task consolidation in clouds Inform. Sci. 258 2014 452 462
13. J.W. Jang, M. Jeon, H.S. Kim, H. Jo, J.S. Kim, and S. Maeng Energy reduction in consolidated servers through memory-aware virtual machine scheduling IEEE Trans. Comput. 60 4 2011 552 564
14. S.U. Khan, and I. Ahmad A cooperative game theoretical technique for joint optimization of energy consumption and response time in computational grids IEEE Trans. Parall. Distr. Syst. 20 3 2009 346 360
15. R. Khandekar, K. Hildrum, S. Parekh, D. Rajan, J. Wolf, K.L. Wu, H. Andrade, and B. Gedik COLA: optimizing stream processing applications via graph partitioning Proc. 10th ACM/IFIP/USENIX International Conference on Middleware, Middleware 2009 2009 ACM Press 1 20
16. N. Kima, J. Chob, and E. Seob Energy-credit scheduler: an energy-aware virtual machine scheduler for cloud systems Future Gener. Comput. Syst. 32 3 2014 128 137
17. J. Lu, and D. Li Bias correction in a small sample from big data IEEE Trans. Knowl. Data Eng. 25 11 2013 2658 2663
18. D.S. Ma, and R. Bondade Enabling power-efficient DVFS operations on silicon IEEE Circ. Syst. Mag. 10 1 2011 14 30
19. D.G. Murray, F. McSherry, R. Isaacs, M. Isard, P. Barham, and M. Abadi Naiad: a timely dataflow system Proc. 24th ACM Symposium on Operating Systems Principles, SOSP 2013 2013 ACM Press 439 455
20. L. Neumeyer, B. Robbins, A. Nair, and A. Kesari S4: distributed stream computing platform Proc. 10th IEEE International Conference on Data Mining Workshops, ICDMW 2010 2010 IEEE Press 170 177
21. M. Pedram Energy-efficient datacenters IEEE Trans. Comput.-Aid. Des. Integr. Circ. Syst. 31 10 2012 1465 1484
22. Z. Qian, Y. He, C. Su, Z. Wu, H. Zhu, T. Zhang, L. Zhou, Y. Yu, and Z. Zhang TimeStream: reliable stream computation in the cloud Proc. 8th ACM European Conference on Computer Systems, EuroSys 2013 2013 ACM Press 1 14
23. M. Rahman, R. Hassan, R. Ranjan, and R. Buyya Adaptive workflow scheduling for dynamic grid and cloud computing environment Concur. Comput.-Pract. Exp. 25 13 2013 1816 1842
24. W. Shang, Z.M. Jiang, H. Hemmati, B. Adams, A.E. Hassan, and P. Martin Assisting developers of big data analytics applications when deploying on Hadoop clouds Proc. 35th International Conference on Software Engineering, ICSE 2013 2013 IEEE Press 402 411
25. H. Shao, L. Rao, Z. Wang, X. Liu, Z. Wang, and Kui Ren Optimal load balancing and energy cost management for Internet data centers in deregulated electricity markets IEEE Trans. Parall. Distr. Syst. 25 10 2014 2659 2669
26. M. Sharifi, S. Shahrivari, and H. Salimi PASTA: a power-aware solution to scheduling of precedence-constrained tasks on heterogeneous computing resources Computing 95 1 2013 67 88
27. S.J. Shin, D.S. Lee, and W.S. Lee CP-tree: an adaptive synopsis structure for compressing frequent itemsets over online data streams Inform. Sci. 278 2014 559 576
28. D. Simoncelli, M. Dusi, F. Gringoli, and S. Niccolini Stream-monitoring with BlockMon: convergence of network measurements and data analytics platforms ACM Sigcomm Comput. Commun. Rev. 43 2 2013 29 35
29. O. Sinnen, A. To, and M. Kaur Contention-aware scheduling with task duplication J. Parall. Distr. Comput. 71 1 2011 77 86
30. G.L. Stavrinides, and H.D. Karatza Scheduling multiple task graphs in heterogeneous distributed real-time systems by exploiting schedule holes with bin packing techniques Simul. Model. Practice Theory 19 1 2011 540 552
31. Storm. < http://storm-project.net/ >.
32. S. Su, J. Li, Q. Huang, X. Huang, K. Shuang, and J. Wang Cost-efficient task scheduling for executing large programs in the cloud Parall. Comput. 39 4-5 2013 177 188
33. J.M. Tien Big data: unleashing information J. Syst. Sci. Syst. Eng. 22 2 2013 127 151
34. R. Tu, X. Wang, and Y. Yang Energy-saving model for SDN data centers J. Supercomput. 70 3 2014 1477 1495
35. H. Vinay, and V. Sarma Energy-efficient operation of multicore processors by DVFS, task migration, and active cooling IEEE Trans. Comput. 63 2 2014 349 360
36. C.D. Wang, J.H. Lai, D. Huang, and W.S. Zheng SVStream: a support vector-based algorithm for clustering data streams IEEE Trans. Knowl. Data Eng. 25 6 2013 1410 1424
37. J. Xu, Z. Chen, J. Tang, and S. Su T-Storm: traffic-aware online scheduling in storm Proc. 2014 IEEE 34th International Conference on Distributed Computing Systems, ICDCS 2014 2014 IEEE Press 535 544
38. Y. Xu, K. Li, L. He, and T.K. Truong A DAG scheduling scheme on heterogeneous computing systems using double molecular structure-based chemical reaction optimization J. Parall. Distr. Comput. 73 9 2013 1306 1322
39. M. Zaharia, T. Das, H.Y. Li, T. Hunter, S. Shenker, and I. Stoica Discretized streams: fault-tolerant streaming computation at scale Proc. 24th ACM Symposium on Operating Systems Principles, SOSP 2013 2013 ACM Press 423 438
40. Y. Zhao, and J. Wu Dache: a data aware caching for big-data applications using the MapReduce framework Proc. 32nd IEEE Conference on Computer Communications, INFOCOM 2013 2013 IEEE Press 35 39
41. S. Zhuravlev, J.C. Saez, S. Blagodurov, A. Fedorova, and M. Prieto Survey of energy-cognizant scheduling techniques IEEE Trans. Parall. Distr. Syst. 24 7 2013 1447 1464
42. M. Zihayat, and A. An Mining top-k high utility patterns over data streams Inform. Sci. 285 2014 138 161
43. Z.L. Zong, A. Manzanares, X.J. Ruan, and X. Qin EAD and PEBD: two energy-aware duplication scheduling algorithms for parallel tasks on homogeneous clusters IEEE Trans. Comput. 60 3 2011 360 374