M3R: Increased performance for in-emory hadoop jobs

Proceedings of the VLDB Endowment

Volumn 5, Issue 12, 2012, Pages 1736-1747

  Shinnar, Avraham ^a   Cunningham, David ^a   Herta, Benjamin ^a   Saraswat, Vijay ^a  

^a IBM RESEARCH   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

MAMMALS;

CLUSTER MEMORY; FRONT END; HIGHER-LEVEL LANGUAGES; INPUT SIZE; MAIN MEMORY; MAP-REDUCE; MEAN TIME TO FAILURE; SPARSE MATRIX-VECTOR MULTIPLY;

ENGINES;

EID: 84873127518     PISSN: None     EISSN: 21508097     Source Type: Journal    
DOI: 10.14778/2367502.2367513     Document Type: Article

References (21)

1. IBM BigSheets. http://www-01.ibm.com/software/data/infosphere/biginsights/.
2. IBM InfoSphere BigInsights. http://www-01.ibm.com/software/ebusiness/jstart/bigsheets/.
3. MapR Hadoop distribution. http://www.mapr.com/products.
4. MapReduce tutorial. http://hadoop.apache.org/common/docs/current/mapred_tutorial.html.
5. Platform Symphony MapReduce. http://www.platform.com/Products/MapReduce/overview.
6. Welcome to Apache Hadoop. http://hadoop.apache.org.
7. K. S. Beyer, V. Ercegovac, R. Gemulla, A. Balmin, M. Y. Eltabakh, C.-C. Kanne, F. ̈Ozcan, and E. J. Shekita. Jaql: A scripting language for large scale semistructured data analysis. PVLDB, 4(12):1272-1283, 2011.
8. D. Borthakur, J. Gray, J. S. Sarma, K. Muthukkaruppan, N. Spiegelberg, H. Kuang, K. Ranganathan, D. Molkov, A. Menon, S. Rash, R. Schmidt, and A. Aiyer. Apache hadoop goes realtime at facebook. In SIGMOD, pages 1071-1080, 2011.
9. Y. Bu, B. Howe, M. Balazinska, and M. D. Ernst. HaLoop: Efficient iterative data processing on large clusters. In VLDB, pages 285-296, 2010.
10. P. Charles, C. Grothoff, V. Saraswat, C. Donawa, A. Kielstra, K. Ebcioglu, C. von Praun, and V. Sarkar. X10: an object-oriented approach to non-uniform cluster computing. In OOPSLA, pages 519-538, 2005.
11. T. Condie, N. Conway, P. Alvaro, J. M. Hellerstein, K. Elmeleegy, and R. Sears. Mapreduce online. In NSDI, pages 21-21, 2010.
12. J. Dean and S. Ghemawat. Mapreduce: simplified data processing on large clusters. Commun. ACM, 51(1):107-113, 2008.
13. J. Ekanayake, H. Li, B. Zhang, T. Gunarathne, S.-H. Bae, J. Qiu, and G. Fox. Twister: a runtime for iterative mapreduce. In HPDC, pages 810-818, 2010.
14. S. Ghemawat, H. Gobioff, and S.-T. Leung. The google file system. SIGOPS Oper. Syst. Rev., 37(5):29-43, 2003.
15. A. Ghoting, R. Krishnamurthy, E. Pednault, B. Reinwald, V. Sindhwani, S. Tatikonda, Y. Tian, and S. Vaithyanathan. Systemml: Declarative machine learning on mapreduce. In ICDE, pages 231-242, 2011.
16. C. Olston, B. Reed, U. Srivastava, R. Kumar, and A. Tomkins. Pig latin: a not-so-foreign language for data processing. In SIGMOD, pages 1099-1110, 2008.
17. V. Saraswat, B. Bloom, I. Peshansky, O. Tardieu, and D. Grove. The X10 reference manual, v2.2. June 2011.
18. The Apache Software Foundation. Welcome to hbase! http://hbase.apache.org, 2010.
19. A. Thusoo, J. S. Sarma, N. Jain, Z. Shao, P. Chakka, S. Anthony, H. Liu, P. Wyckoff, and R. Murthy. Hive: a warehousing solution over a map-reduce framework. PVLDB, 2(2):1626-1629, 2009.
20. T. White. Hadoop: The Definitive Guide. O'Reilly Media, original edition, 2009.
21. M. Zaharia, M. Chowdhury, T. Das, A. Dave, J. Ma, M. McCauley, M. Franklin, S. Shenker, and I. Stoica. Resilient distributed datasets: A fault-tolerant abstraction for in-memory cluster computing. Technical Report UCB/EECS-2011-82, EECS Department, University of California, Berkeley, 2011.