A simulation approach to evaluating design decisions in MapReduce setups

Proceedings - IEEE Computer Society's Annual International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunications Systems, MASCOTS

Volumn , Issue , 2009, Pages 5-15

  Wang, Guanying ^a   Butt, Ali R ^a   Pandey, Prashant ^b   Gupta, Karan ^b  

^a VIRGINIA POLYTECHNIC INSTITUTE AND STATE UNIVERSITY   (United States)

^b IBM ALMADEN RESEARCH CENTER   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

APPLICATION PERFORMANCE; CLOUD COMPUTING; DATA LAYOUTS; DATA LOCALITY; DATA-INTENSIVE APPLICATION; DESIGN DECISIONS; DESIGN SPACES; HARDWARE FAILURES; HIGH EFFICIENCY; INTER-CONNECTS; NETWORK CONFIGURATION; RUNTIMES; SIMULATION APPROACH; TIME-TO-SOLUTION;

TELECOMMUNICATION SYSTEMS;

COMPUTER SIMULATION;

EID: 76349087821     PISSN: 15267539     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/MASCOT.2009.5366973     Document Type: Conference Paper

References (21)

1. Jeffrey Dean and Sanjay Ghemawat. Mapreduce: Simplified data processing on large clusters. Commun. of the ACM, 51(1):107-113, 2008.
2. Jeffrey Dean. Experiences with mapreduce, an abstraction for large-scale computation. In Proc. IEEE PACT, 2006.
3. Apache Software Foundation. Hadoop, May 2007. http://hadoop.apache.org/ core/.
4. Amazon. Amazon Elastic Compute Cloud (Amazon EC2). http://aws.amazon.com/ ec2/.
5. Colby Ranger, Ramanan Raghuraman, Arun Penmetsa, Gary Bradski, and Christos Kozyrakis. Evaluating mapreduce for multi-core and multiprocessor systems. In Proc. IEEE HPCA, 2007.
6. Adam Pisoni. Skynet, Apr. 2008. http://skynet.rubyforge.org.
7. Hadoop User Mailing List Archive, Mar. 2009. http://mail-archives.apache. org/mod-mbox/hadoop-core-user/.
8. The Terasort benchmark, Jun 2009. http://sortbenchmark.org/.
9. Chuanxiong Guo, Haitao Wu, Kun Tan, Lei Shi, Yongguang Zhang, and Songwu Lu. Dcell: a scalable and fault-tolerant network structure for data centers. SIGCOMM Comput. Commun. Rev., 38(4):75-86, 2008.
10. ns-2, Aug 2008. http://nsnam.isi.edu/nsnam/index.php/Main-Page.
11. Guanying Wang, Ali R. Butt, Prashant Pandey, and Karan Gupta. Using realistic simulation for performance analysis of mapreduce setups. In Proc. LSAP, 2009.
12. DiskSim, Aug 2008. http://www.pdl.cmu.edu/DiskSim/.
13. Hadoop's implementation of the Terasort bench-mark, Mar 2009. http://hadoop.apache.org/core/docs/current/api/org/apache/hadoop/examples/ terasort/package-summary.html.
14. Amar Phanishayee, Elie Krevat, Vijay Vasudevan, David G. Andersen, Gregory R. Ganger, Garth A. Gibson, and Srinivasan Seshan. Measurement and analysis of TCP throughput collapse in cluster-based storage systems. In Proc. USENIX FAST, 2008.
15. Ian Foster (Ed.) and Carl Kesselman (Ed.). The GRID: Blueprint for a New Computing Infrastructure. Morgan Kaufmann Publishers, 1999.
16. Kento Aida, Atsuko Takefusa, Hidemoto Nakada, Satoshi Matsuoka, Satoshi Sekiguchi, and Umpei Nagashima. Performance Evaluation Model for Scheduling in Global Computing Systems. Int. J. High Perform. Comput. Appl., 14(3):268-279, 2000.
17. H. J. Song, X. Liu, D. Jakobsen, R. Bhagwan, X. Zhang, K. Taura, and A. Chien. The MicroGrid: A scientific tool for modeling Computational Grids. Sci. Program., 8(3):127-141, 2000.
18. Henri Casanova. Simgrid: A Toolkit for the Simulation of Application Scheduling. In Proc. IEEE CCGRID, 2001.
19. Rajkumar Buyya and M. Manzur Murshed. GridSim: A Toolkit for the Modeling and Simulation of Distributed Resource Management and Scheduling for Grid Computing. CoRR, cs.DC/0203019, 2002.
20. Jerome Boulon, Andy Konwinski, Runping Qi, Ariel Rabkin, Eric Yang, and Mac Yang. Chukwa, a large-scale monitoring system. In Proc. CCA, 2008.
21. Rodrigo Fonseca, George Porter, Randy H. Katz, Scott Shenker, and Ion Stoica. X-Trace: A Pervasive Network Tracing Framework. In Proc. USENIX NSDI, 2007.