Breaking the MapReduce stage barrier

Proceedings - IEEE International Conference on Cluster Computing, ICCC

Volumn , Issue , 2010, Pages 235-244

  Verma, Abhishek ^a   Zea, Nicolas ^a   Cho, Brian ^a   Gupta, Indranil ^a   Campbell, Roy H ^a  

^a University of Illinois at Urbana Champaign   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

JOB COMPLETION; MAP-REDUCE;

CLUSTER COMPUTING;

EID: 78649457559     PISSN: 15525244     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/CLUSTER.2010.29     Document Type: Conference Paper

References (26)

1. J. Dean and S. Ghemawat, "Mapreduce: Simplified data processing on large clusters," in Proc. of the USENIX Symposium on Operating Systems Design and Implementation (OSDI), 2004, pp. 137-149.
2. "Apache hadoop," http://hadoop.apache.org.
3. M. Isard, M. Budiu, Y. Yu, A. Birrell, and D. Fetterly, "Dryad: Distributed data-parallel programs from sequential building blocks," in Proc. of European Conference on Computer Systems (EuroSys), 2007, pp. 59-72.
4. L. J. Guibas and R. Sedgewick, "A dichromatic framework for balanced trees," in Proc. of Symposium on Foundations of Computer Science, 1978, pp. 8-21.
5. C. Chu, S. Kim, Y. Lin, Y. Yu, G. Bradski, A. Ng, and K. Olukotun, "Map-reduce for machine learning on multicore," Advances in Neural Information Processing Systems (NIPS), pp. 281-288, 2007.
6. T. Brants, A. C. Popat, P. Xu, F. J. Och, and J. Dean, "Large language models in machine translation," in Proc. of the Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL), 2007, pp. 858-867.
7. C. Dyer, A. Cordova, A. Mont, and J. Lin, "Fast, easy, and cheap: construction of statistical machine translation models with mapreduce," in Proc. of the Third Workshop on Statistical Machine Translation (StatMT). Association for Computational Linguistics, 2008, pp. 199-207.
8. A. Verma, X. Llora, D. E. Goldberg, and R. H. Campbell, "Scaling genetic algorithms using mapreduce," in Proc. of International Conference on Intelligent Systems Design and Applications, 2009.
9. F. Black and M. S. Scholes, "The pricing of options and corporate liabilities," Journal of Political Economy, vol. 81, no. 3, pp. 637-54, May-June 1973.
10. T. Elsayed, J. Lin, and D. W. Oard, "Pairwise document similarity in large collections with mapreduce," in Proc. of the Annual Meeting of the Association for Computational Linguistics on Human Language Technologies (HLT). Association for Computational Linguistics, 2008, pp. 265-268.
11. O. O'Malley and A. C. Murthy, "Winning a 60 second dash with a yellow elephant," http://sortbenchmark.org/Yahoo2009.pdf, 2009.
12. E. Fix and J. Hodges, "Discriminatory analysis, nonparametric discrimination: Consistency properties." USAF School of Aviation Medicine, Randolph Field, Texas, Tech. Rep., 1951.
13. T. White, Hadoop: The Definitive Guide. O'Reilly Media, Inc., 2009.
14. R. Farivar, A. Verma, E. Chan, and R. Campbell, "Mithra: Multiple data independent tasks on a heterogeneous resource architecture," in Proc. of IEEE International Conference on Cluster Computing and Workshops (CLUSTER), 31 2009-Sept. 4 2009, pp. 1-10.
15. M. A. Olson, K. Bostic, and M. Seltzer, "Berkeley db," in Proc. of the USENIX Annual Technical Conference (ATEC). USENIX Association, 1999, pp. 43-43.
16. "Tokyo cabinet," http://1978th.net/tokyocabinet/.
17. "Mongodb," http://www.mongodb.org.
18. "Illinois Cloud Computing Testbed," http://cloud.cs.illinois. edu/.
19. T. Condie, N. Conway, P. Alvaro, Hellerstein, K. Elmeleegy, and R. Sears, "Online aggregation and continuous query support in mapreduce," in NSDI'10: Seventh USENIX Symposium on Networked Systems Design and Implementation. Berkeley, CA, USA: USENIX Association, 2010.
20. A. Gupta and I. Mumick, "Maintenance of materialized views: Problems, techniques, and applications," Bulletin of the Technical Comm. on, vol. 51, p. 3.
21. D. DeWitt and R. Gerber, "Multiprocessor hash-based join algorithms," in Proc. of VLDB Conference. Citeseer, 1985.
22. M. Stonebraker, D. Abadi, D. DeWitt, S. Madden, E. Paulson, A. Pavlo, and A. Rasin, "MapReduce and parallel DBMSs: friends or foes?" Communications of the ACM, vol. 53, no. 1, pp. 64-71, 2010.
23. A. Pavlo, E. Paulson, A. Rasin, D. J. Abadi, D. J. Dewitt, S. Madden, and M. Stonebraker, "A comparison of approaches to large-scale data analysis," in Proc. of the ACM SIGMOD International Conference. ACM, June 2009. [Online]. Available: http://database.cs.brown.edu/sigmod09/benchmarks- sigmod09.pdf
24. H. Yang, A. Dasdan, R. Hsiao, and D. Parker, "Map-reduce-merge: simplified relational data processing on large clusters," Proc. of the 2007 ACM SIGMOD International conf. on Management of data, Jan 2007.
25. M. Zaharia, A. Konwinski, A. D. Joseph, R. Katz, and I. Stoica, "Improving mapreduce performance in heterogeneous environments," in Proc. of the USENIX Symposium on Operating Systems Design and Implementation (OSDI), 2008.
26. L. Popa, M. Budiu, Y. Yu, and M. Isard, "DryadInc: Reusing work in large-scale computations," in Proc. of Workshop on Hot Topics in Cloud Computing (HotCloud), 2009.