ParaTimer: A progress indicator for MapReduce DAGs

Proceedings of the ACM SIGMOD International Conference on Management of Data

Volumn , Issue , 2010, Pages 507-518

  Morton, Kristi ^a   Balazinska, Magdalena ^a   Grossman, Dan ^a  

^a UNIVERSITY OF WASHINGTON   (United States)

Author keywords

mapreduce; parallel database; progress estimation

Indexed keywords

CRITICAL PATHS; DATA SKEW; DIRECTED ACYCLIC GRAPHS; MAPREDUCE; PARALLEL DATA PROCESSING; PARALLEL DATABASE; QUERY EXECUTION; REAL SYSTEMS; RUNTIMES; SMALL-SCALE CLUSTERS;

DATA PROCESSING; ESTIMATION;

PARALLEL PROCESSING SYSTEMS;

EID: 77954732118     PISSN: 07308078     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1807167.1807223     Document Type: Conference Paper

References (30)

1. C. Ballinger. Born to be parallel: Why parallel origins give Teradata database an enduring performance edge. http://www.teradata.com/t/page/87083/ index.html.
2. S. Chaudhuri, R. Kaushik, and R. Ramamurthy. When can we trust progress estimators for SQL queries. In Proc. of the SIGMOD Conf., Jun 2005.
3. S. Chaudhuri, V. Narassaya, and R. Ramamurthy. Estimating progress of execution for SQL queries. In Proc. of the SIGMOD Conf., Jun 2004.
4. DB2. SQL/monitoring facility. http://www.sprdb2.com/SQLMFVSE.PDF, 2000.
5. DB2. DB2 Basics: The whys and how-tos of DB2 UDB monitoring. http://www.ibm.com/developerworks/db2/library/ techarticle/dm-0408hubel/index. html, 2004.
6. J. Dean and S. Ghemawat. MapReduce: simplified data processing on large clusters. In Proc. of the 6th OSDI Symp., 2004.
7. M. Dempsey. Monitoring active queries with Teradata Manager 5.0. http://www.teradataforum.com/attachments/a030318c.doc, 2001.
8. D. J. DeWitt, E. Paulson, E. Robinson, J. Naughton, J. Royalty, S. Shankar, and A. Krioukov. Clustera: an integrated computation and data management system. In Proc. of the 34th VLDB Conf., pages 28-41, 2008.
9. A. Ganapathi, H. Kuno, U. Dayal, J. L. Wiener, A. Fox, M. Jordan, and D. Patterson. Predicting multiple metrics for queries: Better decisions enabled by machine learning. In Proc. of the 25th ICDE Conf., pages 592-603, 2009.
10. Greenplum. Database performance monitor datasheet (Greenplum Database 3.2.1). http: //www.greenplum.com/pdf/Greenplum-Performance-Monitor.pdf.
11. Greenplum database. http://www.greenplum.com/.
12. Hadoop. http://hadoop.apache.org/.
13. J. M. Hellerstein, P. J. Haas, and H. J. Wang. Online aggregation. In Proc. of the SIGMOD Conf., 1997.
14. IBM zSeries SYSPLEX. http://publib.boulder.ibm.com/infocenter/\\dzichelp/ v2r2/ index.jsp?topic=/com.ibm.db2.doc.admin/xf6495.htm.
15. M. Isard, M. Budiu, Y. Yu, A. Birrell, and D. Fetterly. Dryad: Distributed data-parallel programs from sequential building blocks. In Proc. of the European Conference on Computer Systems (EuroSys), pages 59-72, 2007.
16. C. Jermaine, A. Dobra, S. Arumugam, S. Joshi, and A. Pol. A disk-based join with probabilistic guarantees. In Proc. of the SIGMOD Conf., pages 563-574, 2005.
17. Large Synoptic Survey Telescope. http://www.lsst.org/.
18. G. Luo, J. F. Naughton, C. J. Ellman, and M. Watzke. Increasing the accuracy and coverage of SQL progress indicators. In Proc. of the 20th ICDE Conf., 2004.
19. G. Luo, J. F. Naughton, C. J. Ellman, and M. Watzke. Toward a progress indicator for database queries. In Proc. of the SIGMOD Conf., Jun 2004.
20. G. Luo, J. F. Naughton, and P. S. Yu. Multi-query SQL progress indicators. In Proc. of the 10th EDBT Conf., 2006.
21. C. Mishra and N. Koudas. A lightweight online framework for query progress indicators. In Proc. of the 23rd ICDE Conf., 2007.
22. C. Mishra and M. Volkovs. ConEx: A system for monitoring queries (demonstration). In Proc. of the SIGMOD Conf., Jun 2007.
23. K. Morton, A. Friesen, M. Balazinska, and D. Grossman. Estimating the progress of MapReduce pipelines. In Proc. of the 26th ICDE Conf., 2010.
24. C. Olston, B. Reed, U. Srivastava, R. Kumar, and A. Tomkins. Pig latin: a not-so-foreign language for data processing. In Proc. of the SIGMOD Conf., pages 1099-1110, 2008.
25. Pig Progress Indicator. http://hadoop.apache.org/pig/.
26. G. Plivna. Long running operations in Oracle. http://www.gplivna.eu/ papers/v\$session-longops.htm, 2007.
27. A. Pruscino. Oracle RAC: Architecture and performance. In Proc. of the SIGMOD Conf., page 635, 2003.
28. R. Ramakrishnan and J. Gehrke. Database Management Systems. McGraw-Hill Science Engineering, third edition, 2002.
29. Vertica, inc. http://www.vertica.com/.
30. M. Zaharia, A. Konwinski, A. D. Joseph, R. Katz, and I. Stoica. Improving mapreduce performance in heterogeneous environments. Proc. of the 8th OSDI Symp., 2008.