Extending MapReduce across clouds with bstream

IEEE Transactions on Cloud Computing

Volumn 2, Issue 3, 2014, Pages 362-376

  Kailasam, Sriram ^a   Dhawalia, Prateek ^a   Balaji, S J ^a   Iyer, Geeta ^a   Dharanipragada, Janakiram ^a  

^a INDIAN INSTITUTE OF TECHNOLOGY MADRAS   (India)

Author keywords

inter cloud; MapReduce; stream processing

Indexed keywords

BATCH DATA PROCESSING; DATA STREAMS; DATA TRANSFER; INTEGRATED CIRCUITS;

HADOOP MAPREDUCE; INTER CLOUDS; MAP-REDUCE; MAPREDUCE FRAMEWORKS; NETWORK LATENCIES; OVER PROVISIONING; PERFORMANCE BENEFITS; STREAM PROCESSING;

PIPELINE PROCESSING SYSTEMS;

EID: 84908456121     PISSN: None     EISSN: 21687161     Source Type: Journal    
DOI: 10.1109/TCC.2014.2316810     Document Type: Article

References (28)

1. D. Borthakur, J. Gray, J. S. Sarma, K. Muthukkaruppan, N. Spiegelberg, H. Kuang, K. Ranganathan, D. Molkov, A. Menon, S. Rash, R. Schmidt, and A. Aiyer, "Apache hadoop goes realtime at facebook," in Proc. ACM Int. Conf. Manage. Data, 2011, pp. 1071-1080
2. C. Olston, G. Chiou, L. Chitnis, F. Liu, Y. Han, M. Larsson, A. Neumann, V. B. Rao, V. Sankarasubramanian, S. Seth, C. Tian, T. ZiCornell, and X. Wang, "Nova: continuous Pig/Hadoop workflows," in Proc. ACM Int. Conf. Manage. Data, 2011, pp. 1081-1090
3. T. Condie, N. Conway, P. Alvaro, J. M. Hellerstein, K. Elmeleegy, and R. Sears, "MapReduce online," in Proc. 7th USENIX Conf. Netw. Syst. Des. Implementation, 2010, p. 21
4. H. Kim, S. Chaudhari, M. Parashar, and C. Marty, "Online risk analytics on the cloud," in Proc. IEEE/ACM 9th Int. Symp. Cluster Comput. Grid, 2009, pp. 484-489
5. C. Vecchiola, R. N. Calheiros, D. Karunamoorthy, and R. Buyya, "Deadline-driven provisioning of resources for scientific applications in hybrid clouds with Aneka," Future Generation Comput. Syst., vol. 28, no. 1, pp. 58-65, 2012
6. S. Kailasam, N. Gnanasambandam, J. Dharanipragada, and N. Sharma, "Optimizing ordered throughput using autonomic cloud bursting schedulers," IEEE Trans. Softw. Eng., vol. 39, no. 11, pp. 1564-1581, Nov. 2013
7. S.Kim,J.Won,H.Han,H.Eom,andH.Y.Yeom,"Improvinghadoop performance in intercloud environments," ACMSIGMETRICS Perform. Eval. Rev., vol. 39, no. 3, pp.107-109,Dec. 2011
8. B. Heintz, C. Wang, A. Chandra, and J. Weissman, "Cross-phase optimization in MapReduce," in Proc. IEEE Int. Conf. Cloud Eng., 2013, pp. 338-347
9. B. Heintz, A. Chandra, and R. K. Sitaraman. (2012, Feb. ). Optimizing MapReduce for highly distributed environments, Univ. Minnesota, Minneapolis, MN, USA, Tech. Rep. [Online]. Available: http://www.cs.umn.edu/tech-reports-upload/tr2012/12-003.pdf, TR 12-003
10. J. Dean and S. Ghemawat, "MapReduce: Simplified data processing on large clusters," in Proc. 6th USENIX Conf. Operating Syst. Des. Implementation, 2004, p. 10
11. K. Kc and K. Anyanwu, "Scheduling hadoop jobs to meet deadlines," in Proc. IEEE 2nd Int. Conf. Cloud Comput. Technol. Sci., 2010, pp. 388-392
12. A. Verma, L. Cherkasova, and V. S. Kumar, "Deadline-based workload management for MapReduce environments: Pieces of the performance puzzle," in Proc. IEEE/IFIP Int. Conf. Netw. Operations Manage. Symp., 2012, pp. 900-905
13. A. Verma, L. Cherkasova, and R. H. Campbell, "ARIA: Automatic resource inference and allocation for MapReduce environments," in Proc. 8th ACM Int. Conf. Autonomic Comput., 2011, pp. 235-244
14. H. Herodotou, H. Lim, G. Luo, N. Borisov, L. Dong, F. B. Cetin, and S. Babu, "Starfish: A self-Tuning system for big data analytics," in Proc. 5th Int. Conf. Innovative Data Syst. Res., 2011, pp. 261- 272
15. A. D. Ferguson, P. Bodik, S. Kandula, E. Boutin, and R. Fonseca, "Jockey: Guaranteed job latency in data parallel clusters," in Proc. 7th ACM Eur. Conf. Comput. Syst., 2012, pp. 99-112
16. Y. Luo, Z. Guo, Y. Sun, B. Plale, J. Qiu, and W. W. Li, "A hierarchical framework for cross-domain MapReduce execution," in Proc. 2nd ACM Int. Workshop Emerging Comput. Methods Life Sc., 2011, pp. 15-22
17. M. Mathess and R. N. Calheiros, R. Buyya, "Scaling MapReduce applications across hybrid clouds to meet soft deadlines," in Proc. IEEE 27th Int. Conf. Adv. Inf. Netw. Appl., 2013, pp. 629-636
18. T. Bicer, D. Chiu, and G. Agrawal, "A framework for data-intensive computing with cloud bursting," in Proc. IEEE Int. Conf. Cluster Comput., 2011, pp. 169-177
19. T. Bicer, D. Chiu, and G. Agrawal, "Time and cost sensitive dataintensive computing on hybrid clouds," in Proc. IEEE/ACM 12th Int. Symp. Cluster, Cloud Grid Comput., 2012, pp. 636-643
20. S. Hegde. (2011, May). Autonomic cloudbursting for MapReduce framework using a deadline based scheduler, Electrical and Computer Engineering Department, Masters thesis, Univ. Rutgers, New Brunswick, NJ, USA, [Online]. Available: http://nsfcac. rutgers.edu/people/parashar/Papers/samprita-Thesis.pdf
21. H. Kim and M. Parashar, "Cometcloud: An autonomic cloud engine," in Cloud Computing: Principles and Paradigms, Hoboken, NJ, USA: Wiley, 2011, Ch. 10, pp. 275-297
22. R. Kienzler, R. Bruggmann, A. Ranganathan, and N. Tatbul, "Stream as you go: The case for incremental data access and processing in the cloud," in Proc. IEEE Int. Workshop Data Manage. Cloud, 2012, pp. 159-166
23. D. Logothetis, C. Trezzo, K. C. Webb, and K. Yocum, "In-situ MapReduce for log processing," in Proc. USENIX Annu. Tech. Conf., 2011, p. 9
24. M. Cardosa, C. Wang, A. Nangia, A. Chandra, and J. Weissman, "Exploring mapReduce efficiency with highly-distributed data," in Proc. 2nd ACM Int. Workshop MapReduce Appl., 2011, pp. 27-34
25. H. Gadre, I. Rodero, and M. Parashar, "Investigating MapReduce framework extensions for efficient processing of geographically scattered datasets," ACM SIGMETRICS Perform. Eval. Rev., vol. 39, no. 3, pp. 116-118, Dec. 2011
26. H. Lin, X. Ma, J. Archuleta, W.-c. Feng, M. Gardner, and Z. Zhang, "Moon: MapReduce on opportunistic environments," in Proc. 19th ACM Int. Symp. High Performance Distrib. Comput., 2010, pp. 95-106
27. F. Costa, L. Veiga, and P. Ferreira, "Internet-scale support for map-reduce processing," J. Internet Serv. Appl., vol. 4, no. 1, pp. 1-17, 2013
28. F. Marozzo, D. Talia, and P. Trunfio, "P2P-MapReduce: Parallel data processing in dynamic cloud environments," J. Comput. Syst. Sci., vol. 78, no. 5, pp. 1382-1402, Sep. 2012