Estimating the progress of MapReduce pipelines

Proceedings - International Conference on Data Engineering

Volumn , Issue , 2010, Pages 681-684

  Morton, Kristi ^a   Friesen, Abram ^a   Balazinska, Magdalena ^a   Grossman, Dan ^a  

^a UNIVERSITY OF WASHINGTON   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EXECUTION SPEED; NON-TRIVIAL; PARALLEL DATA PROCESSING; PROCESSING ENVIRONMENTS; QUERY EXECUTION; RUNNING-IN; SMALL-SCALE CLUSTERS; SQL QUERY; TECHNIQUES USED;

ASTROPHYSICS; DATA PROCESSING; GEOMETRICAL OPTICS; PARALLEL PROCESSING SYSTEMS;

ESTIMATION;

EID: 77952748724     PISSN: 10844627     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICDE.2010.5447919     Document Type: Conference Paper

References (23)

1. S. Chaudhuri, V. Narassaya, and R. Ramamurthy. Estimating progress of execution for SQL queries. In Proc. of the SIGMOD Conf., Jun 2004.
2. Chaudhuri et. al. When can we trust progress estimators for SQL queries. In Proc. of the SIGMOD Conf., Jun 2005.
3. DB2. SQL/monitoring facility. http://www.sprdb2.com/SQLMFVSE.PDF, 2000.
4. J. Dean and S. Ghemawat. MapReduce: simplified data processing on large clusters. In Proc. of the 6th OSDI Symp., 2004.
5. M. Dempsey. Monitoring active queries with Teradata Manager 5.0. http://www.teradataforum.com/attachments/a030318c.doc, 2001.
6. Greenplum. Database performance monitor datasheet (Greenplum Database 3.2.1). http://www.greenplum.com/pdf/Greenplum-Performance-Monitor.pdf.
7. Greenplum database. http://www.greenplum.com/.
8. Hadoop. http://hadoop.apache.org/.
9. J. M. Hellerstein, P. J. Haas, and H. J. Wang. Online aggregation. In Proc. of the SIGMOD Conf., 1997.
10. M. Isard, M. Budiu, Y. Yu, A. Birrell, and D. Fetterly. Dryad: Distributed data-parallel programs from sequential building blocks. In Proc. of EuroSys, pages 59-72, 2007.
11. Large Synoptic Survey Telescope. http://www.lsst.org/.
12. G. Luo, J. F. Naughton, C. J. Ellman, and M. Watzke. Increasing the accuracy and coverage of SQL progress indicators. In Proc. of the 20th ICDE Conf., 2004.
13. G. Luo, J. F. Naughton, C. J. Ellman, and M. Watzke. Toward a progress indicator for database queries. In Proc. of the SIGMOD Conf., Jun 2004.
14. G. Luo, J. F. Naughton, and P. S. Yu. Multi-query SQL progress indicators. In Proc. of the 10th EDBT Conf., 2006.
15. C. Mishra and N. Koudas. A lightweight online framework for query progress indicators. In Proc. of the 23rd ICDE Conf., 2007.
16. C. Mishra and M. Volkovs. ConEx: A system for monitoring queries (demonstration). In Proc. of the SIGMOD Conf., Jun 2007.
17. B. A. Myers. The importance of percent-done progress indicators for computer-human interfaces. In Proc. of CHI'85, pages 11-17, 1985.
18. Nuage project. http://nuage.cs.washington.edu/.
19. C. Olston, B. Reed, U. Srivastava, R. Kumar, and A. Tomkins. Pig latin: a not-so-foreign language for data processing. In Proc. of the SIGMOD Conf., pages 1099-1110, 2008.
20. Pig Progress Indicator. http://hadoop.apache.org/pig/.
21. PigMix Benchmarks. http://wiki.apache.org/pig/PigMix.
22. Teradata. http://www.teradata.com/.
23. Vertica, inc. http://www.vertica.com/.