WANalytics: Analytics for a geo-distributed data-intensive world

CIDR 2015 - 7th Biennial Conference on Innovative Data Systems Research

Volumn , Issue , 2015, Pages

  Vulimiri, Ashish ^a   Curino, Carlo ^b   Godfrey, Brighten ^a   Karanasos, Konstantinos ^b   Varghese, George ^b  

^a UIUC   (United States)

^b MICROSOFT   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

BANDWIDTH; BIG DATA;

CENTRAL LOCATIONS; DATA CENTER NETWORKS; DEPLOYED SYSTEMS; DISTRIBUTED DATA; GOVERNMENT REGULATION; LARGE ORGANIZATIONS; PRODUCTION WORKLOADS; WORKFLOW EXECUTION;

LAWS AND LEGISLATION;

EID: 85084012477     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (40)

1. Apache Oozie. http://oozie.apache.org.
2. Apache Storm. http://storm.incubator.apache.org.
3. Cloudera Impala. http://bit.ly/1eRUDeA.
4. Greenplum. http://bit.ly/1oL4Srq.
5. Greenplum. http://basho.com/riak/.
6. Netezza. http://www.ibm.com/software/data/netezza/.
7. Vertica. http://www.vertica.com/.
8. S. Agarwal, S. Kandula, N. Bruno, M.-C. Wu, I. Stoica, and J. Zhou. Reoptimizing data parallel computing. In Presented as part of the 9th USENIX Symposium on Networked Systems Design and Implementation (NSDI 12), pages 281–294, San Jose, CA, 2012. USENIX.
9. S. Agarwal, B. Mozafari, A. Panda, H. Milner, S. Madden, and I. Stoica. Blinkdb: Queries with bounded errors and bounded response times on very large data. In Proceedings of the 8th ACM European Conference on Computer Systems, EuroSys’13, pages 29–42, New York, NY, USA, 2013. ACM.
10. M. O. Akinde, M. H. Böhlen, T. Johnson, L. V. S. Lakshmanan, and D. Srivastava. Efficient olap query processing in distributed data warehouses. In Proceedings of the 8th International Conference on Extending Database Technology: Advances in Database Technology, EDBT’02, pages 336–353, London, UK, UK, 2002. Springer-Verlag.
11. G. Alonso, B. Reinwald, and C. Mohan. Distributed data management in workflow environments. In RIDE, 1997.
12. A. Auradkar, C. Botev, S. Das, D. De Maagd, A. Feinberg, P. Ganti, L. Gao, B. Ghosh, K. Gopalakrishna, and B. Harris. Data infrastructure at linkedin. In 2012 IEEE 28th International Conference on Data Engineering, pages 1370–1381. IEEE, 2012.
13. The Big Data Benchmark. https://amplab.cs.berkeley.edu/benchmark/.
14. S. Chaudhuri and U. Dayal. An overview of data warehousing and OLAP technology. SIGMOD Rec., 1997.
15. A. Chervenak, E. Deelman, M. Livny, M.-H. Su, R. Schuler, S. Bharathi, G. Mehta, and K. Vahi. Data placement for scientific applications in distributed environments. In Proceedings of the 8th IEEE/ACM International Conference on Grid Computing, GRID ’07, pages 267–274, Washington, DC, USA, 2007. IEEE Computer Society.
16. R. Cole, F. Funke, L. Giakoumakis, W. Guy, A. Kemper, S. Krompass, H. Kuno, R. Nambiar, T. Neumann, M. Poess, K.-U. Sattler, M. Seibold, E. Simon, and F. Waas. The mixed workload CH-benCHmark. In DBTest’11.
17. B. F. Cooper, R. Ramakrishnan, U. Srivastava, A. Silberstein, P. Bohannon, H.-A. Jacobsen, N. Puz, D. Weaver, and R. Yerneni. Pnuts: Yahoo!’s hosted data serving platform. Proc. VLDB Endow., 1(2):1277–1288, Aug. 2008.
18. J. C. Corbett, J. Dean, M. Epstein, A. Fikes, C. Frost, J. Furman, S. Ghemawat, A. Gubarev, C. Heiser, P. Hochschild, W. Hsieh, S. Kanthak, E. Kogan, H. Li, A. Lloyd, S. Melnik, D. Mwaura, D. Nagle, S. Quinlan, R. Rao, L. Rolig, Y. Saito, M. Szymaniak, C. Taylor, R. Wang, and D. Woodford. Spanner: Google’s globally-distributed database. In 10th USENIX Symposium on Operating Systems Design and Implementation (OSDI 12), pages 261–264, Hollywood, CA, Oct. 2012. USENIX Association.
19. E. Deelman, G. Singh, M.-H. Su, J. Blythe, Y. Gil, C. Kesselman, G. Mehta, K. Vahi, G. B. Berriman, J. Good, A. Laity, J. C. Jacob, and D. S. Katz. Pegasus: A framework for mapping complex scientific workflows onto distributed systems. Sci. Program., 13(3):219–237, July 2005.
20. A. Dey. Yahoo cross data-center data movement. http://yhoo.it/1nPRImNl, 2010.
21. A. Ghazal, T. Rabl, M. Hu, F. Raab, M. Poess, A. Crolotte, and H.-A. Jacobsen. Bigbench: Towards an industry standard benchmark for big data analytics. In Proceedings of the 2013 ACM SIGMOD International Conference on Management of Data, SIGMOD’13, pages 1197–1208, New York, NY, USA, 2013. ACM.
22. G. Giannikis, D. Makreshanski, G. Alonso, and D. Kossmann. Shared workload optimization. Proceedings of the VLDB Endowment, 7(6), 2014.
23. A. Gupta, F. Yang, J. Govig, A. Kirsch, K. Chan, K. Lai, S. Wu, S. Dhoot, A. Kumar, A. Agiwal, S. Bhansali, M. Hong, J. Cameron, M. Siddiqi, D. Jones, J. Shute, A. Gubarev, S. Venkataraman, and D. Agrawal. Mesa: Geo-replicated, near real-time, scalable data warehousing. In VLDB, 2014.
24. H. Herodotou and S. Babu. Profiling, what-if analysis, and cost-based optimization of mapreduce programs. PVLDB, 2011.
25. J.-H. Hwang, Y. Xing, U. Cetintemel, and S. Zdonik. A cooperative, self-configuring high-availability solution for stream processing. In Data Engineering, 2007. ICDE 2007. IEEE 23rd International Conference on, pages 176–185, April 2007.
26. K. Karanasos, A. Balmin, M. Kutsch, F. Ozcan, V. Ercegovac, C. Xia, and J. Jackson. Dynamically optimizing queries over large scale data platforms. In Proceedings of the 2014 ACM SIGMOD International Conference on Management of Data, SIGMOD’14, pages 943–954, New York, NY, USA, 2014. ACM.
27. D. Kossmann. The state of the art in distributed query processing. ACM Comput. Surv., 2000.
28. T. Kraska, G. Pang, M. J. Franklin, S. Madden, and A. Fekete. MDCC: Multi-data center consistency. In EuroSys, 2013.
29. G. Lee, J. Lin, C. Liu, A. Lorek, and D. Ryaboy. The unified logging infrastructure for data analytics at Twitter. PVLDB, 2012.
30. H. Lim, H. Herodotou, and S. Babu. Stubby: A transformation-based optimizer for mapreduce workflows. Proc. VLDB Endow., 5(11):1196–1207, July 2012.
31. L. F. Mackert and G. M. Lohman. R* optimizer validation and performance evaluation for distributed queries. In VLDB, 1986.
32. S. Madden. Database abstractions for managing sensor network data. Proc. of the IEEE, 98(11):1879–1886, 2010.
33. F. D. McSherry. Privacy integrated queries: an extensible platform for privacy-preserving data analysis. In SIGMOD, 2009.
34. C. Olston, B. Reed, U. Srivastava, R. Kumar, and A. Tomkins. Pig latin: A not-so-foreign language for data processing. In Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data, SIGMOD’08, pages 1099–1110, New York, NY, USA, 2008. ACM.
35. A. Rabkin, M. Arye, S. Sen, V. S. Pai, and M. J. Freedman. Aggregation and degradation in jetstream: Streaming analytics in the wide area. In 11th USENIX Symposium on Networked Systems Design and Implementation (NSDI 14), pages 275–288, Seattle, WA, Apr. 2014. USENIX Association.
36. L. Ravindranath, J. Padhye, S. Agarwal, R. Mahajan, I. Obermiller, and S. Shayandeh. Appinsight: Mobile app performance monitoring in the wild. In Proceedings of the 10th USENIX Conference on Operating Systems Design and Implementation, OSDI’12, pages 107–120, Berkeley, CA, USA, 2012. USENIX Association.
37. M. Stonebraker, P. Brown, D. Zhang, and J. Becla. Scidb: A database management system for applications with complex analytics. Computing in Science Engineering, 15(3):54–62, May 2013.
38. A. Thusoo, Z. Shao, S. Anthony, D. Borthakur, N. Jain, J. Sen Sarma, R. Murthy, and H. Liu. Data warehousing and analytics infrastructure at facebook. In Proceedings of the 2010 ACM SIGMOD International Conference on Management of Data, SIGMOD’10, pages 1013–1020, New York, NY, USA, 2010. ACM.
39. R. S. Xin, J. Rosen, M. Zaharia, M. J. Franklin, S. Shenker, and I. Stoica. Shark: Sql and rich analytics at scale. In Proceedings of the 2013 ACM SIGMOD International Conference on Management of Data, SIGMOD’13, pages 13–24, New York, NY, USA, 2013. ACM.
40. M. Zaharia, M. Chowdhury, T. Das, A. Dave, J. Ma, M. McCauley, M. J. Franklin, S. Shenker, and I. Stoica. Resilient distributed datasets: A fault-tolerant abstraction for in-memory cluster computing. In Proceedings of the 9th USENIX conference on Networked Systems Design and Implementation, pages 2–2. USENIX Association, 2012.