Exploring data parallelism and locality in wide area networks

2008 Workshop on Many-Task Computing on Grids and Supercomputers, MTAGS 2008

Volumn , Issue , 2008, Pages

  Gu, Yunhong ^a   Grossman, Robert ^a  

^a UNIVERSITY OF ILLINOIS AT CHICAGO   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

APPLICATION DEVELOPERS; CLOUD COMPUTING; COMMODITY CLUSTERS; DATA CENTERS; DATA LOCATION; DATA PARALLEL; DATA PARALLELISM; DATA SETS; DISTRIBUTED DATA; DISTRIBUTED DATA PROCESSING; GLOBAL SCALE; LARGE DATASETS; LOAD-BALANCING; PROGRAMMING STRUCTURES;

APPLICATION PROGRAMMING INTERFACES (API); DATA PROCESSING; FAULT TOLERANCE; FAULT TOLERANT COMPUTER SYSTEMS; PARALLEL ARCHITECTURES; PARALLEL PROCESSING SYSTEMS; QUALITY ASSURANCE; SPHERES; SUPERCOMPUTERS;

WIDE AREA NETWORKS;

EID: 67649208936     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/MTAGS.2008.4777906     Document Type: Conference Paper

References (18)

1. M. D. Beynon, R. Ferreira, T. Kurc, A. Sussman, and J. Saltz. DataCutter: Middleware for filtering very large scientific datasets on archival storage systems. Mass Storage Systems Conference, College Park, MD, March 2000.
2. Fay Chang, Jeffrey Dean, Sanjay Ghemawat, Wilson C. Hsieh, Deborah A. Wallach, Mike Burrows, Tushar Chandra, Andrew Fikes, and Robert E. Gruber, Bigtable: A Distributed Storage System for Structured Data, OSDI'06, Seattle, WA, November, 2006.
3. Liang Chen, K. Reddy, and Gagan Agrawal. GATES: A Grid-Based Middleware for Processing Distributed Data Streams. 13th IEEE International Symposium on High Performance Distributed Computing (HPDC) 2004, Honolulu, Hawaii.
4. Jeffrey Dean and Sanjay Ghemawat, MapReduce: Simplified Data Processing on Large Clusters, OSDI'04: Sixth Symposium on Operating System Design and Implementation, San Francisco, CA, December, 2004.
5. Sanjay Ghemawat, Howard Gobioff, and Shun-Tak Leung, The Google File System, pub. 19th ACM Symposium on Operating Systems Principles, Lake George, NY, October, 2003.
6. Jim Gray. Sort benchmark home page. http://research.microsoft.com/barc/ SortBenchmark/.
7. Yunhong Gu, Robert L. Grossman, UDT: UDP-based Data Transfer for High-Speed Wide Area Networks. Computer Networks (Elsevier). Volume 51, Issue 7. May 2007.
8. Yunhong Gu, Robert L. Grossman, Alex Szalay and Ani Thakar, Distributing the Sloan Digital Sky Survey Using UDT and Sector, Proceedings of e-Science 2006.
9. D. R. Musser, "Introspective Sorting and Selection Algorithms", Software Practice and Experience 27(8):983, 1997.
10. John D. Owens, David Luebke, Naga Govindara Ju, Mark Harris, Jens Kruger, Aaron E. Lefohn, and Timothy J.Purcell, A Survey of General-Purpose Computation on Graphics Hardware, Eurographics 2005, pages 21-51, 2005.
11. Ioan Raicu, Zhao Zhang, Mike Wilde, Ian Foster, Pete Beckman, Kamil Iskra, Ben Clifford. "Toward Loosely Coupled Programming on Petascale Systems", to appear at IEEE/ACM Supercomputing 2008.
12. Y. Zhao, M. Hategan, B. Clifford, I. Foster, G. von Laszewski, I. Raicu, T. Stef-Praun, and M. Wilde, Swift: Fast, Reliable, Loosely Coupled Parallel Computation IEEE International Workshop on Scientific Workflows 2007.
13. Jim Wyllie. Spsort: How to sort a terabyte quickly. http://alme1.almaden. ibm.com/cs/spsort.pdf.
14. The Condor DAGMan (Directed Acyclic Graph Manager), http://www.cs.wisc. edu/condor/dagman, 2007.
15. Hadoop, http://hadoop.apache.org/core.
16. Sector, http://sector.sf.net.
17. The Teraflow Testbed, http://www.teraflowtestbed.net
18. The Open Cloud Testbed, http://www.opencloudconsortium.org