MR-Scope: A real-time tracing tool for MapReduce

HPDC 2010 - Proceedings of the 19th ACM International Symposium on High Performance Distributed Computing

Volumn , Issue , 2010, Pages 849-855

  Huang, Dachuan ^a   Shi, Xuanhua ^a   Ibrahim, Shadi ^a   Lu, Lu ^a   Liu, Hongzhang ^a   Wu, Song ^a   Jin, Hai ^a  

^a HUAZHONG UNIVERSITY OF SCIENCE AND TECHNOLOGY   (China)

Author keywords

Hadoop; MapReduce; Real time Tracing; Visualization

Indexed keywords

DATA-INTENSIVE APPLICATION; FILE SYSTEMS; HADOOP; LOG FILE; MAP-REDUCE; OPEN SOURCE IMPLEMENTATION; PROGRAMMING MODELS; REAL-TIME TRACING;

VISUALIZATION;

EID: 78650028829     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1851476.1851598     Document Type: Conference Paper

References (17)

1. Dean, J. and Ghemawat, S. MapReduce: Simplified Data Processing on Large Clusters. In Proceedings of OSDI. 2004, 137-150.
2. Hadoop: http://hadoop.apache.org/2010.
3. Ghemawat, S., Gobioff, H., and Leung, S. The Google file system. In Proceedings of SOSP. 2003, 29-43.
4. Tan, J., Pan, X., Kavulya, S., Gandhi, R., and Narasimhan, P. Mochi: Visual Log-Analysis Based Tools for Debugging Hadoop. CMU-PDL-09-103, Technical Report, May 2009.
5. Fonseca, R., Porter, G., Katz, R. H., Shenker, S., and Stoica, I. X-Trace: A Pervasive Network Tracing Framework. In Proceedings of NSDI. 2007.
6. Konwinski, A., Zaharia, M., Katz, R., and Stoica I. XTracing Hadoop. Presentation in Hadoop Summit, Mar 2008.
7. Massie, M. L., Chun, B. N., and Culler, D. E. The ganglia distributed monitoring system: design, implementation, and experience. In Parallel Computing. Vol.30, No.7, 2004, 817-840.
8. Quiroz, A., Gnanasambandam, N., Parashar, M., and Sharma, N. Robust clustering analysis for the management of selfmonitoring distributed systems. In Proceedings of Cluster Computing. 2009, 73-85.
9. Zaharia, M., Konwinski, A., Joseph, A. D., Katz, R. H., and Stoica, I. Improving MapReduce Performance in Heterogeneous Environments. In Proceedings of OSDI. 2008, 29-42.
10. Boulon, J., Konwinski, A., Qi, R., Rabkin, A., Yang, E., and Yang, M. Chukwa: A large-scale monitoring system. In Proceedings of Cloud Computing and Its Applications, 2008.
11. Ibrahim, S., Jin, H., Cheng, B., Cao, H., Wu, S., and Qi, L. CLOUDLET: towards mapreduce implementation on virtual machines. In Proceedings of HPDC. 2009, 65-66.
12. Ibrahim, S., Jin, H., Lu, L., Qi, L., Wu, S., and Shi, X. Evaluating MapReduce on Virtual Machines: The Hadoop Case. In Proceedings of CloudCom. 2009, 519-528.
13. Cretu-Ciocarlie, G. F., Budiu, M., and Goldszmidt, M. Hunting for Problems with Artemis. In Proceedings of WASL. 2008.
14. He, B. S., Fang, W., Luo, Q., Govindaraju, N. K., and Wang, T. Mars: a MapReduce framework on graphics processors. In Proceedings of PACT. 2008, 260-269.
15. Chen, S, and Schlosser, S. W. Map-Reduce Meets Wider Varieties of Applications. IRP-TR-08-05, Technical Report, Intel Research Pittsburgh, May 2008.
16. ParaGraph User Guide. ParaGraph: A Performance Visualization Tool for MPI http://www.csar.illinois.edu/software/paragraph/
17. MPI standard: http://www.mpi-forum.org/docs/mpi-11- html/mpi-report.html 2010.