A latency and fault-tolerance optimizer for online parallel query plans

Proceedings of the ACM SIGMOD International Conference on Management of Data

Volumn , Issue , 2011, Pages 241-252

  Upadhyaya, Prasang ^a   Kwon, YongChul ^a   Balazinska, Magdalena ^a  

^a UNIVERSITY OF WASHINGTON   (United States)

Author keywords

database; long running; management; online; parallel; queries; systems

Indexed keywords

FAULT TOLERANT COMPUTER SYSTEMS; ONLINE SYSTEMS; QUERY LANGUAGES; QUERY PROCESSING;

BLOCKINGS; FAULT TOLERANCE TECHNIQUES; FAULT-TOLERANT; LONG-RUNNING; ONLINE; OPTIMIZERS; PARALLEL; PARALLEL QUERIES; QUERY; SYSTEM;

FAULT TOLERANCE;

EID: 79959941077     PISSN: 07308078     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1989323.1989350     Document Type: Conference Paper

References (46)

1. Ashish Thusoo et. al. Hive - a petabyte scale data warehouse using hadoop. In Proc. of the 26th ICDE Conf., 2010.
2. M. Balazinska, H. Balakrishnan, S. Madden, and M. Stonebraker. Fault-tolerance in the Borealis distributed stream processing system. In Proc. of the SIGMOD Conf., June 2005.
3. S. P. Boyd, S. J. Kim, L. Vandenberghe, and A. Hassibi. A tutorial on geometric programming. Technical report, Stanford University, Info. Systems Laboratory, Dept. Elect. Eng., 2004.
4. B. Chandramouli, C. N. Bond, S. Babu, and J. Yang. Query suspend and resume. In Proc. of the SIGMOD Conf., 2007.
5. S. Chaudhuri, R. Kaushik, A. Pol, and R. Ramamurthy. Stop-and-restart style execution for long running decision support queries. In Proc. of the 33rd VLDB Conf., 2007.
6. Chen et. al. High availability and scalability guide for DB2 on linux, unix, and windows. IBM Redbooks http://www.redbooks.ibm.com/redbooks/pdfs/ sg247363.pdf, Sept. 2007.
7. Cvx. http://www.stanford.edu/~boyd/cvx/.
8. J. Dean. Software engineering advice from building large-scale distributed systems. http://research.google.com/people/jeff/stanford-295-talk. pdf.
9. J. Dean. Experiences with MapReduce, an abstraction for large-scale computation. Keynote I: PACT, 2006.
10. J. Dean and S. Ghemawat. MapReduce: simplified data processing on large clusters. In Proc. of the 6th OSDI Symp., 2004.
11. E. N. M. Elnozahy, L. Alvisi, Y.-M. Wang, and D. B. Johnson. A survey of rollback-recovery protocols in message-passing systems. ACM Computing Surveys, 34(3), 2002.
12. S. Ganguly, A. Goel, and A. Silberschatz. Efficient and accurate cost models for parallel query optimization (extended abstract). In Proc. of the 15rd PODS Symp., 1996.
13. S. Ganguly, W. Hasan, and R. Krishnamurthy. Query optimization for parallel execution. In Proc. of the SIGMOD Conf., pages 9-18, 1992.
14. Greenplum database. http://www.greenplum.com/.
15. Hadoop. http://hadoop.apache.org/.
16. W. Hasan and R. Motwani. Optimization algorithms for exploiting the parallelism-communication tradeoff in pipelined parallelism. In Proc. of the 20th VLDB Conf., 1994.
17. J. M. Hellerstein, R. Avnur, and V. Raman. Informix under CONTROL: Online query processing. Data Mining and Knowledge Discovery, 4(4):281-314, 2000.
18. J. M. Hellerstein, P. J. Haas, and H. J. Wang. Online aggregation. In Proc. of the SIGMOD Conf., 1997.
19. J.-H. Hwang, Y. Xing, U. Çetintemel, and S. Zdonik. A cooperative, self-configuring high-availability solution for stream processing. In Proc. of ICDE Conf., Apr. 2007.
20. M. Isard, M. Budiu, Y. Yu, A. Birrell, and D. Fetterly. Dryad: Distributed data-parallel programs from sequential building blocks. In Proc. of the EuroSys Conf., pages 59-72, 2007.
21. Jeong-Hyon Hwang et. al. High-availability algorithms for distributed stream processing. In Proc. of the 21st ICDE Conf., Apr. 2005.
22. S. Y. Ko, I. Hoque, B. Cho, and I. Gupta. Making cloud intermediate data fault-tolerant. In Proc. of the 1st ACM symposium on Cloud computing (SOCC), pages 181-192, 2010.
23. Y. Kwon, M. Balazinska, and A. Greenberg. Fault-tolerant stream processing using a distributed, replicated file system. In Proc. of the 34th VLDB Conf., 2008.
24. W. J. Labio, J. L. Wiener, H. Garcia-Molina, and V. Gorelik. Efficient resumption of interrupted warehouse loads. SIGMOD Record, 29(2):46-57, 2000.
25. A.-P. Liedes and A. Wolski. Siren: A memory-conserving, snapshot-consistent checkpoint algorithm for in-memory databases. In Proc. of the 22nd ICDE Conf., page 99, 2006.
26. D. Logothetis, C. Olston, B. Reed, K. C. Webb, and K. Yocum. Stateful bulk processing for incremental analytics. In Proc. of the 1st ACM symposium on Cloud computing (SOCC), 2010.
27. D. Lomet. Dependability, abstraction, and programming. In DASFAA '09: Proc. of the 14th Int. Conf. on Database Systems for Advanced Applications, pages 1-21, 2009.
28. Marcos Vaz Salles et. al. An evaluation of checkpoint recovery for massively multiplayer online games. In Proc. of the 35th VLDB Conf., 2009.
29. J. Mattingley and S. Boyd. Automatic code generation for real-time convex optimization. In Convex Optimization in Signal Processing Optimization. Cambridge U. Press, 2009.
30. C. Olston, B. Reed, U. Srivastava, R. Kumar, and A. Tomkins. Pig latin: a not-so-foreign language for data processing. In Proc. of the SIGMOD Conf., pages 1099-1110, 2008.
31. Oracle database. http://www.oracle.com/.
32. A. Pavlo et. al. A comparison of approaches to large-scale data analysis. In Proc. of the SIGMOD Conf., 2009.
33. A. Ray. Oracle data guard: Ensuring disaster recovery for the enterprise. An Oracle white paper, Mar. 2002.
34. K. Salem and H. Garcia-Molina. Checkpointing memory-resident databases. In Proc. of the 5th ICDE Conf., pages 452-462, 1989.
35. F. B. Schneider. Implementing fault-tolerant services using the state machine approach: a tutorial. ACM Computing Surveys, 22(4):299-319, 1990.
36. M. Shah, J. Hellerstein, and E. Brewer. Highly-available, fault-tolerant, parallel dataflows. In Proc. of the SIGMOD Conf., June 2004.
37. A. Simitsis, K. Wilkinson, U. Dayal, and M. Castellanos. Optimizing etl workflows for fault-tolerance. In Proc. of the 26th ICDE Conf., 2010.
38. R. Talmage. Database mirroring in SQL Server 2005. http://www.microsoft. com/technet/prodtechnol/sql/2005/dbmirror.mspx, Apr. 2005.
39. Teradata. http://www.teradata.com/.
40. R. H. Tütüncü, K. C. Toh, and M. J. Todd. Solving semidefinite-quadratic-linear programs using SDPT3. Mathematical programming, 95(2):189-217, 2003.
41. Tyson Condie et. al. MapReduce online. In Proc. of the 7th NSDI Symp., 2010.
42. P. Upadhyaya, Y. Kwon, and M. Balazinska. A latency and fault-tolerance optimizer for online parallel query plans. Technical report, Department of Computer Science and Engineering, Univ. of Washington, 2010.
43. Vertica, inc. http://www.vertica.com/.
44. C. Yang, C. Yen, C. Tan, and S. R. Madden. Osprey: Implementing MapReduce-style fault tolerance in a shared-nothing distributed database. In Proc. of the 26th ICDE Conf., 2010.
45. Yuan Yu et. al. DryadLINQ: A system for general-purpose distributed data-parallel computing using a high-level language. In Proc. of the 8th OSDI Symp., 2008.
46. M. Zaït, D. Florescu, and P. Valduriez. Benchmarking the DBS3 parallel query optimizer. IEEE Parallel Distrib. Technol., 4(2):26-40, 1996.