A framework for data-intensive computing with cloud bursting

Proceedings - IEEE International Conference on Cluster Computing, ICCC

Volumn , Issue , 2011, Pages 169-177

  Bicer, Tekin ^a   Chiu, David ^b   Agrawal, Gagan ^a  

^a OHIO STATE UNIVERSITY   (United States)

^b Washington State University   (United States)

Author keywords

Cloud bursting; Data intensive computing; Map reduce

Indexed keywords

CLOUD PROVIDERS; COMPUTE RESOURCES; COMPUTING RESOURCE; DATA SET SIZE; DATA SETS; DATA-INTENSIVE COMPUTING; LOAD IMBALANCE; LOCAL CLUSTER; MAP-REDUCE; ORDERS OF MAGNITUDE; REMOTE DATA; SOFTWARE FRAMEWORKS;

CLUSTER COMPUTING; DATA PROCESSING; DATA REDUCTION;

CLOUD COMPUTING;

EID: 80955141016     PISSN: 15525244     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/CLUSTER.2011.21     Document Type: Conference Paper

References (29)

1. T. Condie, N. Conway, P. Alvaro, J. M. Hellerstein, K. Elmeleegy, and R. Sears. MapReduce Online. In NSDI, pages 313-328, 2010.
2. M. de Assuncao, A. di Costanzo, and R. Buyya. Evaluating the Cost- Benefit of Using Cloud Computing to Extend the Capacity of Clusters. In Proccedings of High Performance Distributed Computing (HPDC), pages 141-150, June 2009.
3. J. Dean and S. Ghemawat. Mapreduce: Simplified data processing on large clusters. In OSDI, pages 137-150, 2004.
4. J. Dean and S. Ghemawat. MapReduce: Simplified Data Processing on Large Clusters. In Proceedings of OSDI, pages 137-150, 2004.
5. J. Dean and S. Ghemawat. MapReduce: Simplified Data Processing on Large Clusters. Commun. ACM, 51(1):107-113, 2008.
6. E. Deelman, J. Blythe, Y. Gil, C. Kesselman, G. Mehta, K. Vahi, A. Lazzarini, A. Arbree, R. Cavanaugh, and S. Koranda. Mapping Abstract Complex Workflows onto Grid Environments. In Journal of Grid Computing, pages 9-23, 2003.
7. E. Deelman, G. Singh, M. Livny, B. Berriman, and J. Good. The Cost of Doing Science on the Cloud: The Montage Example. In SC '08: Proceedings of the 2008 ACM/IEEE conference on Supercomputing, pages 1-12, Piscataway, NJ, USA, 2008. IEEE Press.
8. J. Ekanayake, S. Pallickara, and G. Fox. MapReduce for Data Intensive Scientific Analyses. In IEEE Fourth International Conference on e- Science, pages 277-284, Dec 2008.
9. Google App Engine, http://code.google.com/appengine.
10. Z. Hill and M. Humphrey. A Quantitative Analysis of High Performance Computing with Amazon's EC2 Infrastructure: The Death of the LocaL Cluster? In Proceedings of the 10th IEEE/ACM International Conference on Grid Computing, Banff, Alberta, Canada, Oct 2009.
11. M. Isard, M. Budiu, Y. Yu, A. Birrell, and D. Fetterly. Dryad: Distributed Data-Parallel Programs from Sequential Building Blocks. In Proceedings of the 2007 EuroSys Conference, pages 59-72. ACM, 2007.
12. W. Jiang, V. T. Ravi, and G. Agrawal. A Map-Reduce System with an Alternate API for Multi-core Environments. In CCGRID, pages 84-93, 2010.
13. R. Jin and G. Agrawal. A Middleware for Developing Parallel Data Mining Implementations. In Proceedings of the first SIAM conference on Data Mining, Apr. 2001.
14. R. Jin and G. Agrawal. Shared Memory Parallelization of Data Mining Algorithms: Techniques, Programming Interface, and Performance. In Proceedings of the second SIAM conference on Data Mining, Apr. 2002.
15. G. Juve, E. Deelman, K. Vahi, G. Mehta, G. B. Berriman, B. P. Berman, and P. Maechling. Data Sharing Options for Scientific Workflows on Amazon EC2. In SC, pages 1-9, 2010.
16. K. Kambatla, A. Pathak, and H. Pucha. Towards optimizing hadoop provisioning in the cloud. In HotCloud'09: Proceedings of the 2009 conference on Hot topics in cloud computing, Berkeley, CA, USA, 2009. USENIX Association.
17. K. Kambatla, N. Rapolu, S. Jagannathan, and A. Grama. Relaxed Synchronization and Eager Scheduling in MapReduce. Technical Report CSD-TR-09-010, Department of Computer Science, Purdue University, 2009.
18. D. Kondo, B. Javadi, P. Malecot, F. Cappello, and D. P. Anderson. Cost-benefit Analysis of Cloud Computing versus Desktop Grids. In IPDPS '09: Proceedings of the 2009 IEEE International Symposium on Parallel&Distributed Processing, pages 1-12, Washington, DC, USA, 2009. IEEE Computer Society.
19. H. Lin, X. Ma, J. S. Archuleta, W. chun Feng, M. K. Gardner, and Z. Zhang. MOON: MapReduce On Opportunistic eNvironments. In S. Hariri and K. Keahey, editors, HPDC, pages 95-106. ACM, 2010.
20. J. Li, et al. eScience in the Cloud: A MODIS Satellite Data Reprojection and Reduction Pipeline in the Windows Azure Platform. In IPDPS '10: Proceedings of the 2010 IEEE International Symposium on Parallel&Distributed Processing, Washington, DC, USA, 2010. IEEE Computer Society.
21. P. Marshall, K. Keahey, and T. Freeman. Elastic Site: Using Clouds to Elastically Extend Site Resources. In Proceedings of Conference on Cluster, Cloud, and Grid Computing (CCGRID), May 2010.
22. C. Olston, B. Reed, U. Srivastava, R. Kumar, and A. Tomkins. Pig Latin: A Not-So-Foreign Language for Data Processing. In Proceedings of SIGMOD Conference, pages 1099-1110. ACM, 2008.
23. L. Page, S. Brin, R. Motwani, and T. Winograd. The PageRank Citation Ranking: Bringing Order to the Web. Technical Report 1999-66, Stanford InfoLab, November 1999. Previous number = SIDL-WP-1999- 0120.
24. M. R. Palankar, A. Iamnitchi, M. Ripeanu, and S. Garfinkel. Amazon S3 for Science Grids: A Viable Solution? In DADC '08: Proceedings of the 2008 international workshop on Data-aware distributed computing, pages 55-64, New York, NY, USA, 2008. ACM.
25. R. Pike, S. Dorward, R. Griesemer, and S. Quinlan. Interpreting the Data: Parallel Analysis with Sawzall. Scientific Programming, 13(4):277-298, 2005. (Pubitemid 43063583)
26. C. Ranger, R. Raghuraman, A. Penmetsa, G. R. Bradski, and C. Kozyrakis. Evaluating MapReduce for Multi-core and Multiprocessor Systems. In Proceedings of 13th International Conference on High-Performance Computer Architecture (HPCA), pages 13-24. IEEE Computer Society, 2007.
27. C. Vecchiola, S. Pandey, and R. Buyya. High-Performance Cloud Computing: A View of Scientific Applications. Parallel Architectures, Algorithms, and Networks, International Symposium on, 0:4-16, 2009.
28. A. Verma, N. Zea, B. Cho, I. Gupta, and R. H. Campbell. Breaking the MapReduce Stage Barrier. In CLUSTER. IEEE, 2010.
29. M. Zaharia, A. Konwinski, A. D. Joseph, R. H. Katz, and I. Stoica. Improving MapReduce Performance in Heterogeneous Environments. In OSDI, pages 29-42, 2008.