Scheduling divisible MapReduce computations

Journal of Parallel and Distributed Computing

Volumn 71, Issue 3, 2011, Pages 450-459

  Berlinska J ^a   Drozdowski, M ^b  

^a ADAM MICKIEWICZ UNIVERSITY   (Poland)

^b POZNAN UNIVERSITY OF TECHNOLOGY   (Poland)

Author keywords

Divisible loads; MapReduce; Parallel processing; Performance evaluation; Scheduling

Indexed keywords

DISTRIBUTED COMPUTATIONS; DIVISIBLE LOAD SCHEDULING; DIVISIBLE LOAD THEORY; DIVISIBLE LOADS; MAP-REDUCE; PARALLEL PROCESSING; PARTITIONING ALGORITHMS; PERFORMANCE EVALUATION; PERFORMANCE LIMITS; PRECEDENCE CONSTRAINTS; PROCESSING APPLICATIONS;

PARALLEL PROGRAMMING;

MULTIPROCESSING SYSTEMS;

EID: 78751672868     PISSN: 07437315     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.jpdc.2010.12.004     Document Type: Article

References (26)

1. R. Agrawal, and H.V. Jagadish Partitioning techniques for large-grained parallelism IEEE Transactions on Computers 37 1988 1627 1634
2. O. Beaumont, H. Casanova, A. Legrand, Y. Robert, and Y. Yang Scheduling divisible loads on star and tree networks: results and open problems IEEE Transactions on Parallel and Distributed Systems 16 2005 207 218
3. J. Berliska, M. Drozdowski, Dominance properties for divisible mapreduce computations, Institute of Computing Science, Pozna University of Technology, Tech. Rep. RA-09/09, 2009. http://www.cs.put.poznan.pl/mdrozdowski/rapIIn/ ra0909.pdf.
4. J. Berliska, M. Drozdowski, and M. Lawenda Experimental study of scheduling with memory constraints using hybrid methods Journal of Computational and Applied Mathematics 232 2009 638 654
5. V. Bharadwaj, D. Ghose, V. Mani, and T. Robertazzi Scheduling Divisible Loads in Parallel and Distributed Systems 1996 IEEE Computer Society Press Los Alamitos, CA
6. J. Baewicz, and M. Drozdowski Scheduling divisible jobs on hypercubes Parallel Computing 21 1995 1945 1956
7. Y.-C. Cheng, and T.G. Robertazzi Distributed computation with communication delay IEEE Transactions on Aerospace and Electronic Systems 24 1988 700 712
8. Y.-C. Cheng, and T.G. Robertazzi Distributed computation for a tree network with communication delays IEEE Transactions on Aerospace and Electronic Systems 26 1990 511 516
9. N. Comino, and V.L. Narasimhan A novel data distribution technique for hostclient type parallel applications IEEE Transactions on Parallel and Distributed Systems 13 2002 97 110
10. J. Dean, S. Ghemawat, MapReduce: simplified data processing on large clusters, in: Proc. OSDI'04: Sixth Symposium on Operating System Design and Implementation, San Francisco, CA, 2004, pp. 137150. http://labs.google.com/ papers/mapreduce.html.
11. M. Drozdowski Scheduling for Parallel Processing 2009 Springer London
12. M. Drozdowski, and W. Gazek Scheduling divisible loads in a three-dimensional mesh of processors Parallel Computing 25 1999 381 404
13. M. Drozdowski, and P. Wolniewicz Experiments with scheduling divisible tasks in clusters of workstations A. Bode, T. Ludwig, W. Karl, R. Wismuller, Proceedings of 6th Euro-Par Conference Lecture Notes in Computer Science vol. 1900 2000 311 319
14. D. Ghose, and H.J. Kim Load partitioning and trade-off study for large matrix-vector computations in multicast bus networks with communication delays Journal of Parallel and Distributed Computing 55 1998 32 59
15. X. Li, V. Bharadwaj, and C.C. Ko Processing divisible loads on single-level tree networks with buffer constraints IEEE Transactions on Aerospace and Electronic Systems 36 2000 1298 1308
16. X. Li, V. Bharadwaj, and C.C. Ko Distributed image processing on a network of workstations International Journal of Computers and Applications 25 2003 1 10
17. T. Lim, T.G. Robertazzi, Efficient parallel video processing through concurrent communication on a multi-port star network, in: Proc. of the 40th Conference on Information Sciences and Systems, Princeton, NJ, 2006, pp. 458463.
18. J. Lin, and C. Dyer Data-Intensive Text Processing with MapReduce 2010 Morgan & Claypool
19. Lpsolve reference guide, 2010, http://lpsolve.sourceforge.net/5.5/.
20. R. Pike, S. Dorward, R. Griesemer, and S. Quinlan Interpreting the data: parallel analysis with Sawzall Scientific Programming 13 2005 277 298
21. K. van der Raadt, Y. Yang, H. Casanova, Practical divisible load scheduling on grid platforms with APST-DV, in: Proc. of the 19th IPDPS'05, 2005, pp. 29.b.
22. C. Ranger, R. Raghuraman, A. Penmetsa, G. Bradski, C. Kozyrakis, Evaluating MapReduce for multi-core and multiprocessor systems, in: Proceedings of International Symposium on High Performance Computer Architecture, HPCA, 2007, pp. 1324.
23. T. Robertazzi Ten reasons to use divisible load theory IEEE Computer 36 2003 63 68
24. J. Sohn, T.G. Robertazzi, and S. Luryi Optimizing computing costs using divisible load analysis IEEE Transactions on Parallel and Distributed Systems 9 1998 225 234
25. Wikipedia, MapReduce, http://en.wikipedia.org/wiki/MapReduce, [Online; 17-February-2010].
26. Y. Yang, H. Casanova, M. Drozdowski, M. Lawenda, A. Legrand, On the complexity of multi-round divisible load scheduling, INRIA Rhne-Alpes, Research Report 6096, 2007, http://hal.inria.fr/inria-00123711/en/.