Global analytics in the face of bandwidth and regulatory constraints

Proceedings of the 12th USENIX Symposium on Networked Systems Design and Implementation, NSDI 2015

Volumn , Issue , 2015, Pages 323-336

  Vulimiri, Ashish ^a   Curino, Carlo ^b   Godfrey, Brighten ^a   Jungblut, Thomas ^b   Padhye, Jitu ^b   Varghese, George ^b  

^a UIUC   (United States)

^b MICROSOFT   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

BANDWIDTH; SYSTEMS ANALYSIS; WIDE AREA NETWORKS;

CENTRAL LOCATIONS; DISTRIBUTED DATA; NETWORK-CENTRIC; PRODUCTION WORKLOADS; QUERY EXECUTION; QUERY OPTIMIZATION; RESEARCH ISSUES; TRANSOCEANIC LINKS;

BIG DATA;

EID: 84967163393     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (45)

1. H. Abu-Libdeh, L. Princehouse, and H. Weatherspoon. RACS: a case for cloud storage diversity. In SoCC 2010.
2. S. Agarwal, J. Dunagan, N. Jain, S. Saroiu, A. Wolman, and H. Bhogan. Volley: Automated data placement for geo-distributed cloud services. In NSDI 2010.
3. A. Auradkar, C. Botev, S. Das, D. D. Maagd, A. Feinberg, P. Ganti, L. Gao, B. Ghosh, K. Gopalakrishna, B. Harris, J. Koshy, K. Krawez, J. Kreps, S. Lu, S. Nagaraj, N. Narkhede, S. Pachev, I. Perisic, L. Qiao, T. Quiggle, J. Rao, B. Schulman, A. Sebastian, O. Seeliger, A. Silberstein, B. Shkolnik, C. Soman, R. Sumbaly, K. Surlaker, S. Topiwala, C. Tran, B. Varadarajan, J. Westerman, Z. White, D. Zhang, and J. Zhang. Data infrastructure at LinkedIn. 2014 IEEE 30th International Conference on Data Engineering, 0:1370-1381, 2012.
4. C. Ballinger. Born to be parallel: Why parallel origins give Teradata database an enduring performance edge. http://www.teradata.com/white-papers/born-to-be-parallel-teradata-database.
5. P. A. Bernstein, N. Goodman, E. Wong, C. L. Reeve, and J. B. Rothnie, Jr. Query processing in a system for distributed databases (SDD-1). ACM Transactions on Database Systems, 1981.
6. The Big Data Benchmark. https://amplab.cs.berkeley.edu/benchmark/.
7. P. Cao and Z. Wang. Efficient top-k query calculation in distributed networks. In PODC '04, PODC '04.
8. W. W. Chu and P. Hurley. Optimal query processing for distributed database systems. IEEE Trans. Comput.
9. R. Cole, F. Funke, L. Giakoumakis, W. Guy, A. Kemper, S. Krompass, H. Kuno, R. Nambiar, T. Neumann, M. Poess, K.-U. Sattler, M. Seibold, E. Simon, and F. Waas. The mixed workload chbenchmark. In DBTest '11, DBTest '11, pages 8:1-8:6, New York, NY, USA, 2011. ACM.
10. B. F. Cooper, R. Ramakrishnan, U. Srivastava, A. Silberstein, P. Bohannon, H.-A. Jacobsen, N. Puz, D. Weaver, and R. Yerneni. PNUTS: Yahoo!' s hosted data serving platform. VLDB, 2008.
11. B. F. Cooper, A. Silberstein, E. Tam, R. Ramakrishnan, and R. Sears. Benchmarking cloud serving systems with ycsb. In SoCC '10.
12. J. C. Corbett, J. Dean, M. Epstein, A. Fikes, C. Frost, J. Furman, S. Ghemawat, A. Gubarev, C. Heiser, P. Hochschild, W. Hsieh, S. Kanthak, E. Kogan, H. Li, A. Lloyd, S. Melnik, D. Mwaura, D. Nagle, S. Quinlan, R. Rao, L. Rolig, Y. Saito, M. Szymaniak, C. Taylor, R. Wang, and D. Woodford. Spanner: Google's globally-distributed database. In OSDI 2012.
13. D. DeWitt and J. Gray. Parallel database systems: the future of high performance database systems. Commun. ACM.
14. A. Dey. Yahoo cross data-center data movement. http://yhoo.it/1nPRImNl, 2010.
15. C. Dwork. Differential privacy: A survey of results. In M. Agrawal, D. Du, Z. Duan, and A. Li, editors, Theory and Applications of Models of Computation, volume 4978 of Lecture Notes in Computer Science, pages 1-19. Springer Berlin Heidelberg, 2008.
16. European Commission press release. Commission to pursue role as honest broker in future global negotiations on internet governance. http://europa.eu/rapid/press-release-IP-14-142-en.htm.
17. L. Gao. On inferring autonomous system relationships in the internet. IEEE/ACM Trans. Netw., 9(6):733-745, Dec. 2001.
18. A. Ghazal, T. Rabl, M. Hu, F. Raab, M. Poess, A. Crolotte, and H.-A. Jacobsen. Bigbench: Towards an industry standard benchmark for big data analytics. In Proceedings of the 2013 ACM SIGMOD International Conference on Management of Data, SIGMOD '13, pages 1197-1208, New York, NY, USA, 2013. ACM.
19. K. Glerum, K. Kinshumann, S. Greenberg, G. Aul, V. Orgovan, G. Nichols, D. Grant, G. Loihle, and G. Hunt. Debugging in the (very) large: Ten years of implementation and experience. In Proceedings of the ACM SIGOPS 22Nd Symposium on Operating Systems Principles, SOSP '09, pages 103-116, New York, NY, USA, 2009. ACM.
20. G. Graefe. Query evaluation techniques for large databases. ACM Comput. Surv.
21. A. Greenberg, J. Hamilton, D. A. Maltz, and P. Patel. The cost of a cloud: Research problems in data center networks. ACM CCR, 39(1), 2008.
22. A. Gupta, F. Yang, J. Govig, A. Kirsch, K. Chan, K. Lai, S. Wu, S. Dhoot, A. Kumar, A. Agiwal, S. Bhansali, M. Hong, J. Cameron, M. Siddiqi, D. Jones, J. Shute, A. Gubarev, S. Venkataraman, and D. Agrawal. Mesa: Geo-replicated, near realtime, scalable data warehousing. In VLDB, 2014.
23. J.-H. Hwang, Y. Xing, U. Cetintemel, and S. Zdonik. A cooperative, self-configuring highavailability solution for stream processing. In Data Engg. Workshop 2007.
24. T. Inc. Global Internet Geography. http://www.telegeography.com/research-services/global-internet-geography/, 2014.
25. K. Jeon, S. Chandrashekhara, F. Shen, S. Mehra, O. Kennedy, and S. Y. Ko. Pigout: Making multiple hadoop clusters work together. In Big Data, 2014 IEEE International Conference on, pages 100-109, Oct 2014.
26. F. Kerschbaum. Privacy-preserving computation. In B. Preneel and D. Ikonomou, editors, Privacy Technologies and Policy, volume 8319 of Lecture Notes in Computer Science, pages 41-54. Springer Berlin Heidelberg, 2014.
27. D. Kossmann. The state of the art in distributed query processing. ACM Computing Surveys, 32(4):422-469, Dec. 2000.
28. N. Laoutaris, M. Sirivianos, X. Yang, and P. Rodriguez. Inter-datacenter bulk transfers with netstitcher. In SIGCOMM 2011.
29. G. Lee, J. Lin, C. Liu, A. Lorek, and D. Ryaboy. The unified logging infrastructure for data analytics at Twitter. PVLDB, 2012.
30. G. Lohman. Is query optimization a "solved" problem? http://wp.sigmod.org/?p=1075, April 2014.
31. S. Madden. Database abstractions for managing sensor network data. Proc. of the IEEE, 98(11):1879-1886, 2010.
32. C. Olston, B. Reed, U. Srivastava, R. Kumar, and A. Tomkins. Pig Latin: a not-so-foreign language for data processing. In SIGMOD 2008.
33. M. T. Özsu and P. Valduriez. Principles of distributed database systems. Springer, 2011.
34. A. Rabkin, M. Arye, S. Sen, V. S. Pai, and M. J. Freedman. Aggregation and degradation in Jet- Stream: Streaming analytics in the wide area. In NSDI 2014.
35. M. Rost and K. Bock. Privacy by design and the new protection goals. DuD, January, 2011.
36. N. Shrivastava, C. Buragohain, D. Agrawal, and S. Suri. Medians and beyond: New aggregation techniques for sensor networks. In Proceedings of the 2Nd International Conference on Embedded Networked Sensor Systems, SenSys '04, pages 239-249, New York, NY, USA, 2004. ACM.
37. A. Swami and A. Gupta. Optimization of large join queries. In Proceedings of the 1988 ACM SIGMOD International Conference on Management of Data, SIGMOD '88, pages 8-17, New York, NY, USA, 1988. ACM.
38. A. L. Tatarowicz, C. Curino, E. P. C. Jones, and S. Madden. Lookup tables: Fine-grained partitioning for distributed databases. In Proceedings of the 2012 IEEE 28th International Conference on Data Engineering, ICDE '12, pages 102-113, Washington, DC, USA, 2012. IEEE Computer Society.
39. A. Thusoo, J. S. Sarma, N. Jain, Z. Shao, P. Chakka, N. Zhang, S. Antony, H. Liu, and R. Murthy. Hive - a petabyte scale data warehouse using hadoop. In ICDE 2010.
40. A. Thusoo, Z. Shao, S. Anthony, D. Borthakur, N. Jain, J. Sen Sarma, R. Murthy, and H. Liu. Data warehousing and analytics infrastructure at Facebook. SIGMOD, 2010.
41. A. Tridgell and P. Mackerras. The rsync algorithm, 1996.
42. M. Vrable, S. Savage, and G. M. Voelker. Cumulus: Filesystem backup to the cloud. Trans. Storage, 5(4):14:1-14:28, Dec. 2009.
43. A. Vulimiri, C. Curino, P. B. Godfrey, K. Karanasos, and G. Varghese. WANalytics: Analytics for a Geo-Distributed Data-Intensive World. In Conference on Innovative Data Systems Research (CIDR 2015), January 2015.
44. J. Wiener and N. Boston. Facebook's top open data problems. https://research.facebook.com/blog/1522692927972019/facebook-s-top-open-data-problems/, 2014.
45. M. Zaharia, M. Chowdhury, T. Das, A. Dave, J. Ma, M. McCauly, M. J. Franklin, S. Shenker, and I. Stoica. Resilient distributed datasets: a fault-tolerant abstraction for in-memory cluster computing. In NSDI 2012.