Benchmarking a MapReduce environment on a full virtualisation platform

CISIS 2010 - The 4th International Conference on Complex, Intelligent and Software Intensive Systems

Volumn , Issue , 2010, Pages 433-438

  Kontagora, Maryam ^a   González Vélez, Horacio ^b  

^a ROBERT GORDON UNIVERSITY   (United Kingdom)

^b IDEAS Research Institute   (United Kingdom)

Author keywords

Algorithmic skeletons; MapReduce; Master worker; Parallel programming; Server virtualization; Structured parallelism; Task farm

Indexed keywords

ALGORITHMIC SKELETON; MAPREDUCE; MASTER WORKER; MASTER-WORKERS; TASK FARMS; VIRTUALIZATIONS;

COMPUTER SOFTWARE; LOCAL AREA NETWORKS; PARALLEL ARCHITECTURES;

PARALLEL PROGRAMMING;

EID: 77952696128     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/CISIS.2010.45     Document Type: Conference Paper

References (23)

1. J. Dean and S. Ghemawat, "MapReduce: Simplified data processing on large clusters," in OSDI'04. San Francisco: USENIX, Dec. 2004, pp. 137-150.
2. -, "MapReduce: Simplified data processing on large clusters," Commun. ACM, vol. 51, no. 1, pp. 107-113, 2008.
3. J. P. Buzen and U. O. Gagliardi, "The evolution of virtual machine architecture," in AFIPS'73. New York: ACM, Jun. 1973, pp. 291-299.
4. A. Whitaker, M. Shaw, and S. D. Gribble, "Scale and performance in the Denali isolation kernel," SIGOPS Oper. Syst. Rev., vol. 36, no. SI, pp. 195-209, 2002.
5. T. G. Robertazzi, "Ten reasons to use divisible load theory," Computer, vol. 36, no. 5, pp. 63-68, 2003.
6. H. González-Vélez and M. Cole, "Adaptive statistical scheduling of divisible workloads in heterogeneous systems," Journal of Scheduling, 2009, in Press. DOI: 10.1007/s10951-009-0138-4.
7. The Apache Software Foundation, "Hadoop Map-Reduce tutorial," Hadoop Project, hadoop.apache.org/, Manual version 0.15, Jan. 2008.
8. C. Ranger, R. Raghuraman, A. Penmetsa, G. Bradski, and C. Kozyrakis, "Evaluating MapReduce for multi-core and multiprocessor systems," in HPCA-13. Phoenix: IEEE, Feb. 2007, pp. 13-24.
9. A. Pisoni, "Skynet," Geni.com, skynet.rubyforge.org/, Manual version 0.9.3, May 2007.
10. Nokia Research Center, "Disco," Nokia Research Center, discoproject.org, Manual version 0.2.3, Aug. 2009.
11. H. González-Vélez, "An adaptive skeletal task farm for grids," in Euro-Par 2005, ser. Lect. Notes Comput. Sc., no. 3648. Lisbon: Springer-Verlag, Aug. 2005, pp. 401-410.
12. VMware, "Understanding full virtualization, paravirtualization, and hardware assist," VMware, Inc., White paper Revision: 20070911, 2007.
13. M. Cole, Algorithmic Skeletons: Structured Management of Parallel Computation. London: Pitman/MIT Press, 1989.
14. B. Bacci, M. Danelutto, S. Pelagatti, and M. Vanneschi, "SkIE: A heterogeneous environment for HPC applications," Parallel Comput., vol. 25, no. 13, pp. 1827-1852, 1999.
15. M. Cole, "Bringing skeletons out of the closet: a pragmatic manifesto for skeletal parallel programming," Parallel Comput., vol. 30, no. 3, pp. 389-406, 2004.
16. M. Danelutto, "Adaptive task farm implementation strategies," in PDP 2004. LaCoruña: IEEE, Feb. 2004, pp. 416-423.
17. H. González-Vélez, "Self-adaptive skeletal task farm for computational grids," Parallel Comput., vol. 32, no. 7-8, pp. 479-490, 2006.
18. H. Kuchen and J. Striegnitz, "Features from functional programming for a C++ skeleton library," Concurr. Comput.-Pract. Exp., vol. 17, no. 7-8, pp. 739-756, 2005.
19. H. González-Vélez and M. Cole, "Adaptive structured parallelism for distributed heterogeneous architectures: A methodological approach with pipelines and farms," Concurr. Comput.-Pract. Exp., 2009, in Press. DOI: 10.1002/cpe.1549.
20. A. B. Nagarajan, F. Mueller, C. Engelmann, and S. L. Scott, "Proactive fault tolerance for HPC with Xen virtualization," in ICS '07. Seattle: ACM, Jun. 2007, pp. 23-32.
21. L. Youseff, R. Wolski, B. Gorda, and C. Krintz, "Paravirtualization for HPC systems," in ISPA'06 Wksp, ser. Lect. Notes Comput. Sc., vol. 4331. Sorrento: Springer-Verlag, 2006, pp. 474-486.
22. M. Zaharia, A. Konwinski, A. Joseph, R. Katz, and I. Stoica, "Improving MapReduce performance in heterogeneous environments," in USENIX OSDI'08, San Diego, Dec. 2008.
23. T. Sandholm and K. Lai, "MapReduce optimization using regulated dynamic prioritization," in SIGMETRICS'09. Seattle: ACM, 2009, pp. 299-310.