DELMA: Dynamically elastic mapreduce framework for CPU-intensive applications

Proceedings - 11th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, CCGrid 2011

Volumn , Issue , 2011, Pages 454-463

  Fadika, Zacharia ^a   Govindaraju, Madhusudhan ^a  

^a State University of New York at Binghamton   (United States)

Author keywords

Cloud Computing; DELMA; Hadoop; MapReduce

Indexed keywords

CLUSTER CONFIGURATIONS; CLUSTER SIZES; CPU-INTENSIVE; DATA SET SIZE; DATA SETS; DATA-INTENSIVE APPLICATION; DELMA; DESIGN DECISIONS; DISTRIBUTED APPLICATIONS; DYNAMIC ELASTICITY; HADOOP; HIERARCHICAL DATA; INPUT DATAS; INPUT SIZE; MAP-REDUCE; RUNTIMES; SCIENTIFIC APPLICATIONS; SMALL CLUSTERS;

CLUSTER ANALYSIS; CLUSTER COMPUTING; DATA HANDLING; ELASTICITY; GRID COMPUTING; SHRINKAGE; STATISTICAL TESTS;

CLOUD COMPUTING;

EID: 79961138274     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/CCGrid.2011.71     Document Type: Conference Paper

References (27)

1. J. Dean and S. Ghemawat, "MapReduce: Simplified Data Processing on Large Clusters," Communications of the ACM, vol. 51, no. 1, pp. 107-113, 2008.
2. G. Orenstein, "Digging Deeper Into Data With Hadoop," Available at http://gigaom.com/2009/06/07/digging-deeper-into-data-with-hadoop, 2009.
3. A. S. Szalay, P. Z. Kunszt, A. Thakar, J. Gray, D. Slutz, and R. J. Brunner, "Designing and mining multi-terabyte astronomy archives: the sloan digital sky survey," SIGMOD, 2000.
4. S. W. S. Shimin Chen, "Map-reduce meets wider varieties of applications." May 2008.
5. Microsoft Research. [Online]. Available: http://www.microsoft.com/ windowsazure/
6. AMAZON, "Amazon Elastic Computer Cluster." [Online]. Available: http://aws.amazon.com/ec2
7. Pervasive Technology Institute. [Online]. Available: http://www. iterativemapreduce.org/
8. Apache Hadoop. [Online]. Available: http://hadoop.apache.org
9. D. Borthakur, "The hadoop distributed file system: Architecture and design," Hadoop Project, 2007.
10. S. Weil, S. A. Brandt, E. L. Miller, D. D. E. Long, and C. Maltzahn, "Ceph: A Scalable, High-Performance Distributed File System," in OSDI '06: Proceedings of the 7th Conference on Operating Systems Design and Implementation, November 2006.
11. L.-F. Cabrera, D. D. E., and L. Swift, "Using distributed disk striping to provide high I/O data rates," Computer Systems, 1991.
12. T. Anderson, M. Dahlin, J. Neefe, D. Patterson, D. Roselli, and R. Wang, "Serverless network file systems," in 15th ACM Symposium on Operating System Principles, December 1995.
13. L. Heshan, A. Ma, and M. Feng, "Moon: Mapreduce on opportunistic environments," in HPDC '10: the ACM International Symposium on High Performance Distributed Computing. ACM, 2010.
14. Z. Fadika, M. R. Head, and M. Govindaraju, "Parallel and Distributed Approach for Processing Large-Scale XML Datasets," in 10th IEEE/ACM International Conference on Grid Computing, October 2009, pp. 5-7.
15. L. G. V. T. Ravi and G. Agrawal, "Supporting Load Balancing for Distributed Data-Intensive Applications," in IEEE International Conference on High Performance Computing (HiPC'09), Kochi, India, December 2009.
16. G. Singh, S. Bharathi, A. Chervenak, E. Deelman, C. Kesselman, M. Manohar, S. Patil, and L. Pearlman, "A Metadata Catalog Service for Data Intensive Applications," in SC '03: Proceedings of the 2003 ACM/IEEE conference on Supercomputing. Washington, DC, USA: IEEE Computer Society, 2003, p. 33.
17. AxisJava, "The Apache Project," 2002, http://ws.apache.org/ axis/.
18. D. Veillard, "The XML C parser and toolkit of Gnome," 2006, http://xmlsoft.org/.
19. Y. Oren, "Piccolo is a small, extremely fast XML parser for Java," 2006, http://piccolo.sourceforge.net/.
20. M. R. Head, M. Govindaraju, A. Slominski, P. Liu, N. Abu-Ghazaleh, R. van Engelen, K. Chiu, and M. J. Lewis, "A Benchmark Suite for SOAP-based Communication in Grid Web Services," in SC '05: Proceedings of the 2005 ACM/IEEE conference on Supercomputing. Washington, DC, USA: IEEE Computer Society, 2005, p. 19.
21. W. Zhang and R. A. van Engelen, "A Table-Driven Streaming XML Parsing Methodology for High-Performance Web Services," in ICWS '06: Proceedings of the IEEE International Conference on Web Services. Los Alamitos, CA, USA: IEEE Computer Society, 2006, pp. 197-204.
22. World Wide Web Consortium, "w3c." [Online]. Available: http://www.w3.org
23. Hierarchical Data Format (HDF), http://hdf.ncsa.uiuc.edu/.
24. Network Common Data Form (netCDF), http://www.unidata.ucar.edu/packages/ netcdf/.
25. D. Thain and M. Livny, "Building reliable clients and servers," in The Grid: Blueprint for a New Computing Infrastructure, I. Foster and C. Kesselman, Eds. Morgan Kaufmann, 2003.
26. J. Yu and R. Buyya, "Gridbus workflow enactment engine," in Grid Computing: Infrastructure, Service, and Applications, W. J. L. Wang and J. Chen, Eds., April 2009, pp. 119-146.
27. D. Thain, T. Tannenbaum, and M. Livny, Grid Computing: Making the Global Infrastructure a Reality. Wiley, 2003, ch. 11, Condor and the Grid, pp 299-335.