Upper and lower bounds on the cost of a map-reduce computation

Proceedings of the VLDB Endowment

Volumn 6, Issue 4, 2013, Pages 277-288

  Afrati, Foto N ^a   Sarma, Anish Das ^b   Salihoglu, Semih ^c   Ullman, Jeffrey D ^c  

^a NATIONAL TECHNICAL UNIVERSITY OF ATHENS   (Greece)

^b GOOGLE INC   (United States)

^c STANFORD UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COST REDUCTION; HAMMING DISTANCE;

COMMUNICATION COST; CONSTANT FACTORS; FINDING PAIRS; LOWER BOUNDS; MAP-REDUCE; MATRIX MULTIPLICATION; UPPER AND LOWER BOUNDS;

MATRIX ALGEBRA;

EID: 84881258668     PISSN: None     EISSN: 21508097     Source Type: Journal    
DOI: 10.14778/2535570.2488334     Document Type: Article

References (27)

1. F. Afrati and J. Ullman. Optimizing multiway joins in a map-reduce environment. IEEE Transactions on Knowledge and Data Engineering, 23(9):1282-1298, 2011.
2. F. N. Afrati, D. Fotakis,, and J. D. Ullman. Enumerating subgraph instances using map-reduce. Technical report, Stanford InfoLab, January 2012. Available at http://ilpubs.stanford.edu:8090/1020/.
3. F. N. Afrati, A. D. Sarma, D. Menestrina, A. Parameswaran,, and J. D. Ullman. Fuzzy joins using mapreduce. In ICDE, 2012.
4. F. N. Afrati, A. D. Sarma, S. Salihoglu,, and J. D. Ullman. Upper, and lower bounds on the cost of a map-reduce computation. CoRR, abs/1206.4377, 2012.
5. N. Alon. On the number of subgraphs of prescribed type of graphs with a given number of edges. Israel Journal of Mathematics, 38(1-2):116-130, 1981.
6. Amazon. Amazon Elastic Compute Cloud (Amazon EC2). Amazon Inc., 2008.
7. A. Atserias, M. Grohe,, and D. Marx. Size bounds, and query plans for relational joins. In FOCS, pages 739-748, 2008.
8. R. L. F. Cordeiro, C. T. Jr., A. J. M. Traina, J. L'opez, U. Kang,, and C. Faloutsos. Clustering very large multi-dimensional datasets with mapreduce. In KDD, 2011.
9. J. Dean, and S. Ghemawat. Mapreduce: simplified data processing on large clusters. In OSDI, pages 137-150, 2004.
10. W. Feller. An Introduction to Probability Theory, and Its Applications. Vol. 1, 3rd ed. New York: Wiley, 1968. (Stirling's Formula in Section 2.9).
11. A. Ghoting, R. Krishnamurthy, E. P. D. Pednault, B. Reinwald, V. Sindhwani, S. Tatikonda, Y. Tian,, and S. Vaithyanathan. Systemml: Declarative machine learning on mapreduce. In ICDE, pages 231-242, 2011.
12. M. Grohe, and D. Marx. Constraint solving via fractional edge covers. In SODA, pages 289-298, 2006.
13. W. Hasan, and R. Motwani. Optimization algorithms for exploiting the parallelism-communication tradeoff in pipelined parallelism. In VLDB, 1994.
14. R. Ikeda, H. Park,, and J. Widom. Provenance for generalized map, and reduce workflows. In CIDR, 2011.
15. H. J. Karloff, S. Suri,, and S. Vassilvitskii. A model of computation for mapreduce. In SODA, pages 938-948, 2010.
16. P. Koutris, and D. Suciu. Parallel evaluation of conjunctive queries. In PODS, pages 223-234, 2011.
17. Y. Kwon, M. Balazinska, B. Howe,, and J. A. Rolia. Skewtune: mitigating skew in mapreduce applications. In SIGMOD Conference, pages 25-36, 2012.
18. A. C. McKellar, and E. G. Coffman. Organizing matrices, and matrix operations for paged memory systems. Commun. ACM, 12(3):153-165, 1969.
19. A. Okcan, and M. Riedewald. Processing theta-joins using mapreduce. In SIGMOD Conference, pages 949-960, 2011.
20. C. Olston, and B. Reed. Inspector gadget: A framework for custom monitoring, and debugging of distributed dataflows. In Proc. of VLDB, 2011.
21. A. Pietracaprina, G. Pucci, M. Riondato, F. Silvestri,, and E. Upfal. Space-round tradeoffs for mapreduce computations. CoRR, abs/1111.2228, 2011.
22. A. Rajaraman, and J. D. Ullman. Mining of Massive Datasets. Cambridge University Press. Also available on-line at i.stanford.edu/~ullman/mmds.html, 2011.
23. T. Schank. Algorithmic Aspects of Triangle-Based Network. University of Karlsruhe (TH), 2007.
24. S. Suri, and S. Vassilvitskii. Counting triangles, and the curse of the last reducer. In WWW, 2011.
25. W.-C. Tan. Provenance in Databases: Past, Current,, and Future. IEEE Data Engineering Bulletin, 2008.
26. R. Vernica, M. J. Carey,, and C. Li. Efficient parallel set-similarity joins using mapreduce. In SIGMOD Conference, 2010.
27. T. White. Hadoop - The Definitive Guide: Storage, and Analysis at Internet Scale (2. ed.). O'Reilly, 2011.