Performance study of parallel programming on cloud computing environments using MapReduce

2010 International Conference on Information Science and Applications, ICISA 2010

Volumn , Issue , 2010, Pages

  Shih, Wen Chung ^a   Tseng, Shian Shyong ^a   Yang, Chao Tung ^b  

^a ASIA UNIVERSITY   (Taiwan)

^b TUNGHAI UNIVERSITY   (Taiwan)

Author keywords

Cloud computing; Hadoop; High performance computing; MapReduce; Parallel programming

Indexed keywords

CLOUD COMPUTING; COMPUTING POWER; DIVISIBLE LOAD APPLICATIONS; HADOOP; HIGH PERFORMANCE COMPUTING; PARALLELIZATIONS; PERFORMANCE STUDY; SCIENTISTS AND ENGINEERS; TOTAL COMPLETION TIME;

DISTRIBUTED COMPUTER SYSTEMS; INFORMATION SCIENCE;

PARALLEL PROGRAMMING;

EID: 77954454112     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICISA.2010.5480515     Document Type: Conference Paper

References (33)

1. Foster, I., The Grid: A New Infrastructure for 21st Century Science. Physics Today, 2002. 55(2): p. 42-47.
2. Foster, I. and C. Kesselman, Globus: A Metacomputing Infrastructure Toolkit. International Journal of Supercomputer Applications and High Performance Computing, 1997. 11(2): p. 115-128.
3. Boeres, C., et al., An EasyGrid portal for scheduling system-aware applications on computational Grids. Concurrency and Computation: Practice and Experience, 2006. 18(6): p. 553-566.
4. Boeres, C. and V.E.F. Rebello, EasyGrid: towards a framework for the automatic Grid enabling of legacy MPI applications. Concurrency and Computation: Practice and Experience, 2004. 16(5): p. 425-432.
5. Nascimento, A.P., et al., Distributed and dynamic self-scheduling of parallel MPI Grid applications. Concurrency and Computation: Practice and Experience, 2006. in press.
6. Beaumont, O., et al., Scheduling divisible loads on star and tree networks: results and open problems. Parallel and Distributed Systems, IEEE Transactions on, 2005. 16(3): p. 207-218.
7. Maciej, D. and L. Marcin, Multi-installment Divisible Load Processing in Heterogeneous Systems with Limited Memory. : Parallel Processing and Applied Mathematics. 2006. 847-854 (Pubitemid 44020099)
8. Maciej, D. and L. Marcin, On Optimum Multi-installment Divisible Load Processing in Heterogeneous Distributed Systems. : Euro-Par 2005 Parallel Processing. 2005. 231-240 (Pubitemid 41490821)
9. Kruskal, C. and A. Weiss, Allocating independent subtaskson parallel processors. IEEE Transactions on Software Engineering, 1984. 11: p. 1001-1016.
10. Polychronopoulos, C.D. and D.J. Kuck, Guided self-scheduling: A practical scheduling scheme for parallel supercomputers. 1987, IEEE Computer Society. p. 1425-1439.
11. Susan Flynn, H., S. Edith, and E.F. Lawrence, Factoring: a method for scheduling parallel loops. 1992, ACM Press. p. 90-101.
12. Tzen, T.H. and L.M. Ni, Trapezoid Self-Scheduling: A Practical Scheduling Scheme for Parallel Compilers. IEEE Transactions on Parallel and Distributed Systems, 1993. 4(1): p. 87-98.
13. Yang, C.T. and S.C. Chang, A Parallel Loop Self-Scheduling on Extremely Heterogeneous PC Clusters. Journal of Information Science and Engineering, 2004. 20(2): p. 263-273.
14. Yang, C.T., K.W. Cheng, and K.C. Li, An Efficient Parallel Loop Self- Scheduling Scheme for Cluster Environments. Journal of Supercomputing, 2005. 34: p. 315-335.
15. Shih, W.-C., C.-T. Yang, and S.-S. Tseng, A performance-based parallel loop scheduling on grid environments. The Journal of Supercomputing, 2007. 41(3): p. 247-267.
16. Yang, C.-T., W.-C. Shih, and S.-S. Tseng, Dynamic Partitioning of Loop Iterations on Heterogeneous PC Clusters. to appear in The Journal of Supercomputing, 2008.
17. THU. The TIGER Grid. 2006 [cited 2006; Available from: http://gamma2.hpc.csie.thu.edu.tw/ganglia/
18. Globus. The Globus Project. 2004 [cited 2006; Available from: http://www.globus.org/.
19. MPICH. MPICH-G2. 2004 [cited 2006; Available from: http://www.hpclab.niu. edu/mpi/.
20. Han, J. and M. Kamber, Data Mining: Concepts and Techniques. 2001: Morgan Kaufmann.
21. Agrawal, R. and J.C. Shafer, Parallel Mining of Association Rules. IEEE Transactions on Knowledge and Data Engineering, 1996. 8(6): p. 962-969.
22. Agrawal, R. and R. Srikant. Fast algorithms for Mining Association Rules. in Proc. 20th Very Large Data Bases Conf. 1994.
23. Zaki, M.J., Parallel and Distributed Association Mining: A Survey. IEEE Concurrency, 1999. 7(4): p. 14-25.
24. Mandelbrot, B.B., Fractal Geometry of Nature. 1988, New York: W. H. Freeman.
25. Herrera, J., et al. Loosely-coupled loop scheduling in computational grids. in The 20th International Parallel and Distributed Processing Symposium (IPDPS 2006). 2006.
26. Penmatsa, S., et al. Implementation of Distributed Loop Scheduling Schemes on the TeraGrid. in IEEE International Parallel and Distributed Processing Symposium (IPDPS 2007). 2007.
27. Cannataro, M., et al., Distributed data mining on grids: services, tools, and applications. Systems, Man, and Cybernetics, Part B, IEEE Transactions on, 2004. 34(6): p. 2451-2465.
28. Fiolet, V., et al. Optimal Grid Exploitation Algorithms for Data Mining. in The Fifth International Symposium on Parallel and Distributed Computing (ISPDC '06). 2006.
29. Jiang, W.-S. and J.-H. Yu. Distributed data mining on the grid. in Proceedings of 2005 International Conference on Machine Learning and Cybernetics 2005.
30. Shih, W.C., Yang C.T. and Tseng S.S., "Using a Performance-based Skeleton to Implement Divisible Load Applications on Grid Computing Environments," Journal of Information Science and Engineering (JISE), 25(1), pp. 59-81, 2009.
31. Apache, "Hadoop", http://lucene.apache.org/hadoop/, 2006.
32. J. Dean and S. Ghemawat, "Mapreduce: Simplified data processing on large clusters," in OSDI'04: Proceedings of the 6th Symposium on Operating System Design and Implementation. San Francisco, California, USA. USENIX Association, pp. 137-150. December 6-8, 2004.
33. THU, "Cloud Testbed", http://140.128.98.4:50070/dfshealth.jsp, 2009.