DOT: A matrix model for analyzing, optimizing and deploying software for big data analytics in distributed systems

Proceedings of the 2nd ACM Symposium on Cloud Computing, SOCC 2011

Volumn , Issue , 2011, Pages

  Huai, Yin ^a   Lee, Rubao ^a   Zhang, Simon ^b   Xia, Cathy H ^a   Zhang, Xiaodong ^a  

^a Ohio State University   (United States)

^b Department of Obstetrics Gynecology   (United States)

Author keywords

Big data analytics; Distributed systems; System modeling; System scalability

Indexed keywords

COMPUTING POWER; DATA ANALYTICS; DATA SETS; DATA TRANSFORMATION; DISTRIBUTED SYSTEMS; FAULT-TOLERANT; GENERAL MODEL; INTERMEDIATE RESULTS; JOB EXECUTION; MATRIX MODEL; PARALLEL PROCESSING MODELS; PERFORMANCE REQUIREMENTS; PROCESSING OPERATIONS; SCALE-UP; SOFTWARE FRAMEWORKS; STORAGE RESOURCES; SYSTEM MODELING; SYSTEM SCALABILITY;

CLOUD COMPUTING; DATA HANDLING; DATA TRANSFER; FAULT TOLERANCE; FAULT TOLERANT COMPUTER SYSTEMS; PARALLEL PROCESSING SYSTEMS; SCALABILITY;

MATHEMATICAL MODELS;

EID: 82155168671     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2038916.2038920     Document Type: Conference Paper

References (29)

1. http://hadoop.apache.org/.
2. http: //en.wikipedia.org/wiki/Recurrencefirelation.
3. http: //en.wikipedia.org/wiki/K-meansficlustering.
4. http://www.tpc.org/tpch/.
5. http://aws.amazon.com/ec2/.
6. http://en.wikipedia.org/wiki/ParallelfiRandomfiAccessfiMachine.
7. A. Abouzied, K. Bajda-Pawlikowski, D. J. Abadi, A. Silberschatz, and A. Rasin. HadoopDB: An Architectural Hybrid of MapReduce and DBMS Technologies for Analytical Workloads. In VLDB, Lyon, France, 2009.
8. R. Avnur and J. M. Hellerstein. Eddies: Continuously Adaptive Query Processing. In SIGMOD, 2000.
9. S. Chandrasekaran, O. Cooper, A. Deshpande, M. J. Franklin, J. M. Hellerstein, W. Hong, S. Krishnamurthy, S. Madden, V. Raman, F. Reiss, and M. A. Shah. TelegraphCQ: Continuous Dataflow Processing for an Uncertain World. In CIDR, 2003.
10. D. E. Culler, R. M. Karp, D. A. Patterson, A. Sahay, E. E. Santos, K. E. Schauser, R. Subramonian, and T. von Eicken. LogP: A Practical Model of Parallel Computation. Commun. ACM, 39(11):78-85, 1996. (Pubitemid 126428083)
11. J. Dean and S. Ghemawat. MapReduce: Simplified Data Processing on Large Clusters. In OSDI, 2004.
12. J. Dittrich, J.-A. Quiané-Ruiz, A. Jindal, Y. Kargin, V. Setty, and J. Schad. Hadoop++: Making a Yellow Elephant Run Like a Cheetah (Without It Even Noticing). PVLDB, 3(1):518-529, 2010.
13. J. Feldman, S. Muthukrishnan, A. Sidiropoulos, C. Stein, and Z. Svitkina. On Distributing Symmetric Streaming Computations. In SODA, 2008.
14. J. Gray. Distributed Computing Economics. ACM Queue, 6(3):63-68, 2008.
15. Y. He, R. Lee, Y. Huai, Z. Shao, N. Jain, X. Zhang, and Z. Xu. RCFile: A Fast and Space-efficient Data Placement Structure in MapReduce-based Warehouse Systems. In ICDE, 2011.
16. B. Hindman, A. Konwinski, M. Zaharia, and I. Stoica. A Common Substrate for Cluster Computing. In HotCloud'09, 2009.
17. M. Isard, M. Budiu, Y. Yu, A. Birrell, and D. Fetterly. Dryad: Distributed Data-Parallel Programs from Sequential Building Blocks. In EuroSys, 2007.
18. H. J. Karloff, S. Suri, and S. Vassilvitskii. A Model of Computation for MapReduce. In SODA, 2010.
19. W. Kim. On Optimizing an SQL-like Nested Query. ACM Trans. Database Syst., 7(3):443-469, 1982.
20. R. Lee, T. Luo, Y. Huai, F. Wang, Y. He, and X. Zhang. YSmart: Yet another SQL-to-MapReduce Translator. In ICDCS, 2011.
21. R. Lee, M. Zhou, and H. Liao. Request Window: an Approach to Improve Throughput of RDBMS-based Data Integration System by Utilizing Data Sharing Across Concurrent Distributed Queries. In VLDB, 2007.
22. G. Malewicz, M. H. Austern, A. J. C. Bik, J. C. Dehnert, I. Horn, N. Leiser, and G. Czajkowski. Pregel: A System for Large-Scale Graph Processing. In SIGMOD, 2010.
23. D. G. Murray and S. Hand. Ciel: a universal execution engine for distributed data-flow computing. In NSDI '11, 2011.
24. L. Page, S. Brin, R. Motwani, and T. Winograd. The Pagerank Citation Ranking: Bringing Order to the Web. Technical Report 1999-66, Stanford InfoLab.
25. A. Pavlo, E. Paulson, A. Rasin, D. J. Abadi, D. J. DeWitt, S. Madden, and M. Stonebraker. A Comparison of Approaches to Large-Scale Data Analysis. In SIGMOD Conference, 2009.
26. A. Thusoo, J. S. Sarma, N. Jain, Z. Shao, P. Chakka, S. Anthony, H. Liu, P. Wyckoff, and R. Murthy. Hive - A Warehousing Solution Over a Map-Reduce Framework. PVLDB, 2(2):1626-1629, 2009.
27. L. G. Valiant. A Bridging Model for Parallel Computation. Commun. ACM, 33(8):103-111, 1990.
28. Y. Yu, P. K. Gunda, and M. Isard. Distributed Aggregation for Data-Parallel Computing: Interfaces and Implementations. In SOSP, 2009.
29. X. Zhang, Y. Yan, and K. He. Latency Metric: An Experimental Method for Measuring and Evaluating Parallel Program and Architecture Scalability. J. Parallel Distrib. Comput., 22(3):392-410, 1994.