Using realistic simulation for performance analysis of mapreduce setups

Proc. 1st ACM Workshop on Large-Scale System and Application Performance, LSAP2009, Co-located with the 2009 International Symposium on High Performance Distributed Computing Conference, HPDC'09

Volumn , Issue , 2009, Pages 19-26

  Wang, Guanying ^a   Butt, Ali R ^a   Pandey, Prashant ^b   Gupta, Karan ^b  

^a VIRGINIA POLYTECHNIC INSTITUTE AND STATE UNIVERSITY   (United States)

^b IBM ALMADEN RESEARCH CENTER   (United States)

Author keywords

Cloud computing; Hadoop; MapReduce; Simulation

Indexed keywords

APPLICATION PERFORMANCE; CLOUD COMPUTING; DATA VOLUME; DATA-INTENSIVE COMPUTING; DESIGN SPACES; DESIGN TOOL; HIGH-LEVEL ABSTRACTION; MEDIUM-SCALE PRODUCTION; PERFORMANCE ANALYSIS; PERFORMANCE BOTTLENECKS; PERFORMANCE IMPROVEMENTS; PLANNING TOOLS; PROGRAMMING MODELS; REALISTIC SIMULATION; RUNTIMES; SCIENTIFIC COMPUTING; TRANSPARENT ACCESS;

DESIGN; DIE CASTING; LARGE SCALE SYSTEMS; SIMULATORS;

COMPUTER SCIENCE;

EID: 70449730088     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1552272.1552278     Document Type: Conference Paper

References (19)

1. DiskSim, Aug 2008. http://www.pdl.cmu.edu/DiskSim/.
2. ns-2, Aug 2008. http://nsnam.isi.edu/nsnam/index.php/Main-Page.
3. Disco Project, Jan. 2009. http://discoproject.org/.
4. Hadoop User Mailing List Archive, Mar. 2009. http://mail-archives.apache. org/mod-mbox/hadoop-core-user/.
5. JIRA: HADOOP-3473, Feb 2009. http://issues.apache.org/jira/browse/HADOOP- 3473.
6. Terasort, Mar 2009. http://hadoop.apache.org/core/docs/current/api/org/ apache/hadoop/examples/terasort/package-summary.html.
7. Adam Pisoni. Skynet, Apr. 2008. http://skynet.rubyforge.org.
8. K. Aida, A. Takefusa, H. Nakada, S. Matsuoka, S. Sekiguchi, and U. Nagashima. Performance Evaluation Model for Scheduling in Global Computing Systems. Int. J. High Perform. Comput. Appl., 14(3):268-279, 2000.
9. Apache Software Foundation. Hadoop, May 2007. http://hadoop.apache.org/ core/.
10. J. Boulon, A. Konwinski, R. Qi, A. Rabkin, E. Yang, and M. Yang. Chukwa, a large-scale monitoring system. In Proc. CCA, 2008.
11. R. Buyya and M. M. Murshed. GridSim: A Toolkit for the Modeling and Simulation of Distributed Resource Management and Scheduling for Grid Computing. CoRR, cs.DC/0203019, 2002.
12. H. Casanova. Simgrid: A Toolkit for the Simulation of Application Scheduling. In Proc. IEEE CCGRID, 2001.
13. J. Dean. Experiences with mapreduce, an abstraction for large-scale computation. In Proc. IEEE PACT, 2006.
14. J. Dean and S. Ghemawat. Mapreduce: Simplified data processing on large clusters. Comm. of the ACM, 51(1):107-113, 2008.
15. R. Fonseca, G. Porter, R. H. Katz, S. Shenker, and I. Stoica. X-Trace: A Pervasive Network Tracing Framework. In Proc. USENIX NSDI, 2007.
16. I. Foster (Ed.) and C. Kesselman (Ed.). The GRID: Blueprint for a New Computing Infrastructure. Morgan Kaufmann Publishers, 1999.
17. A. Phanishayee, E. Krevat, V. Vasudevan, D. G. Andersen, G. R. Ganger, G. A. Gibson, and S. Seshan. Measurement and analysis of TCP throughput collapse in cluster-based storage systems. In Proc. USENIX FAST, 2008.
18. C. Ranger, R. Raghuraman, A. Penmetsa, G. Bradski, and C. Kozyrakis. Evaluating mapreduce for multi-core and multiprocessor systems. In Proc. IEEE HPCA, pages 13-24, 2007.
19. H. J. Song, X. Liu, D. Jakobsen, R. Bhagwan, X. Zhang, K. Taura, and A. Chien. The MicroGrid: A scientific tool for modeling Computational Grids. Sci. Program., 8(3):127-141, 2000.