SDQuery DSI: Integrating data management support with a wide area data transfer protocol

International Conference for High Performance Computing, Networking, Storage and Analysis, SC

Volumn , Issue , 2013, Pages

  Su, Yu ^a   Wang, Yi ^a   Agrawal, Gagan ^a   Kettimuthu, Rajkumar ^b  

^a OHIO STATE UNIVERSITY   (United States)

^b ARGONNE NATIONAL LABORATORY   (United States)

Author keywords

Data management; I O performance tuning; Indexing; Query processing; Wide area networks

Indexed keywords

DATA TRANSFER; DYNAMIC LOADS; INDEXING (OF INFORMATION); INFORMATION MANAGEMENT; QUERY PROCESSING; TOOLS; WIDE AREA NETWORKS;

CURRENT LIMITATION; DATA SET SIZE; DATA TRANSFER PROTOCOLS; MANAGEMENT SUPPORT; NETWORK ENVIRONMENTS; PERFORMANCE MODEL; PERFORMANCE TUNING; RAPID GROWTH;

DIGITAL STORAGE;

EID: 84899697283     PISSN: 21674329     EISSN: 21674337     Source Type: Conference Proceeding    
DOI: 10.1145/2503210.2503270     Document Type: Conference Paper

References (33)

1. Ioannis Alagiannis, Renata Borovica, Miguel Branco, Stratos Idreos, and Anastasia Ailamaki. NoDB: efficient query execution on raw data files. In Proceedings of the 2012 ACM SIGMOD International Conference on Management of Data, pages 241-252, 2012.
2. W. E. Allcock, I. Foster, and R. Madduri. Reliable Data Transport: A Critical Service for the Grid. In Proceedings of the Workshop on Building Service Based Grids, 2004.
3. G. Antoshenkov. Byte-aligned bitmap compression. In Data Compression Conference (DCC), page 476. IEEE, 1995.
4. Andrew Baranovski, Keith Beattie, Shishir Bharathi, Joshua Boverhof, John Bresnahan, Ann Chervenak, Ian Foster, Tim Freeman, Dan Gunter, Kate Keahey, Carl Kesselman, Rajkumar Kettimuthu, Nick Leroy, Michael Link, Miron Livny, Ravi Madduri, Gene Oleynik, Laura Pearlman, Robert Schuler, and Brian Tierney. Enabling petascale science: Data management, troubleshooting, and scalable science services. Journal of Physics: Conference Series, 125, 2008.
5. J. Buck, N. Watkins, J. LeFevre, K. Ioannidou, C. Maltzahn, N. Polyzotis, and S. Brandt. Scihadoop: Array-based query processing in hadoop. In International Conference for High Performance Computing, Networking, Storage and Analysis (SC), 2011.
6. Jerry Chou, Kesheng Wu, O Rubel, Mark Howison, Ji Qiang, Brian Austin, E Wes Bethel, Rob D Ryne, Arie Shoshani, et al. Parallel index and query for large scale data analysis. In International Conference for High Performance Computing, Networking, Storage and Analysis (SC), pages 1-11. IEEE, 2011.
7. P. Cornillon, J. Gallagher, and T. Sgouros. Opendap: Accessing data in a distributed, heterogeneous environment. Data Science Journal, 2(0):164-174, 2003.
8. Ian Foster and Carl Kesselman. The globus toolkit. The grid: blueprint for a new computing infrastructure, pages 259-278, 1999.
9. Thomas J Hacker, Brian D Athey, and Brian Noble. The end-to-end performance effects of parallel tcp sockets on a lossy wide-area network. In 16th International Parallel and Distributed Processing Symposium (IPDPS), pages 434-443. IEEE, 2002.
10. Hans-Christian Hege, Andrei Hutanu, Ralf Kähler, André Merzky, Thomas Radke, Edward Seidel, and Brygg Ullmer. Progressive retrieval and hierarchical visualization of large remote data. Scalable Computing: Practice and Experience, 6(3), 2001.
11. PW Jones, PH Worley, Y. Yoshida, JB White III, and J. Levesque. Practical performance portability in the parallel ocean program (pop). Concurrency and Computation: Practice and Experience, 17(10):1317-1327, 2005.
12. Rajkumar Kettimuthu, Michael Link, John Bresnahan, and William Allcock. Globus data storage interface (dsi)-enabling easy access to grid datasets. In First DIALOGUE Workshop: Applications-Driven Issues in Data Grids, 2005.
13. Rajkumar Kettimuthu, Alex Sim, Dan Gunter, Bill Allcock, Peer-Timo Bremer, John Bresnahan, Andrew Cherry, Lisa Childers, Eli Dart, Ian Foster, Kevin Harms, Jason Hick, Jason Lee, Michael Link, Jeff Long, Keith Miller, Vijaya Natarajan, Valerio Pascucci, Ken Raffenetti, David Ressman, Dean Williams, Loren Wilson, and Linda Winkler. Lessons learned from moving earth system grid data sets over a 20 gbps wide-area network. In Proceedings of the 19th ACM International Symposium on High Performance Distributed Computing (HPDC 2010), Jun 2010.
14. Ezra Kissel, D. Martin Swany, and Aaron Brown. Improving GridFTP performance using the Phoebus session layer. In Proceedings of SC, November 2009.
15. Scott Klasky, Hasan Abbasi, Jeremy Logan, Manish Parashar, Karsten Schwan, Arie Shoshani, Matthew Wolf, Sean Ahern, Ilkay Altintas, Wes Bethel, et al. In situ data processing for extreme-scale computing. In Scientific Discovery through Advanced Computing Program (SciDAC'11), 2011.
16. T. Kosar and M. Livny. Stork: Making Data Placement a First Class Citizen in the Grid. In Proceedings of International Conference on Distributed Computing Systems (ICDCS), 2004.
17. Wantao Liu, Brian Tieman, Rajkumar Kettimuthu, and Ian Foster. A data transfer framework for large-scale science experiments. In 19th ACM International Symposium on High Performance Distributed Computing (HPDC), pages 717-724. ACM, 2010.
18. D. Lu, Y. Qiao, P. A. Dinda, and F. E. Bustamante. Modeling and Taming Parallel TCP on Wide Area Networks. In Proceedings of the 12th International Parallel and Distributed Processing Symposium (IPDPS), April 2005.
19. B. Ludäscher, I. Altintas, C. Berkley, D. Higgins, E. Jaeger, M. Jones, E. A. Lee, J. Tao, and Y. Zhao. Scientific workflow management and the kepler system. Concurrency and Computation: Practice and Experience, 18(10):1039-1065, 2006.
20. S Narayanan, TJ Madden, AR Sandy, Raj Kettimuthu, and Michael Link. Gridftp based real-time data movement architecture for x-ray photon correlation spectroscopy at the advanced photon source. In 8th IEEE International Conference on E-Science (e-Science), pages 1-8. IEEE, 2012.
21. P. O'Neil and D. Quass. Improved query performance with variant indexes. In ACM Sigmod Record, volume 26, pages 38-49. ACM, 1997.
22. Lili Qiu, Yin Zhang, and Srinivasan Keshav. On individual and aggregate tcp performance. In Seventh International Conference on Network Protocols (ICNP), pages 203-212. IEEE, 1999.
23. Alberto Sánchez, María S Pérez, Pierre Gueant, Jesús Montes, and Pilar Herrero. A parallel data storage interface to gridftp. In Proceedings of the 2006 Confederated international conference on On the Move to Meaningful Internet Systems: CoopIS, DOA, GADA, and ODBASE-Volume Part II, pages 1203-1212. Springer, 2006.
24. Michael Stonebraker, Jacek Becla, David Dewitt, Kian-Tat Lim, David Maier, Oliver Ratzesberger, and Stan Zdonik. Requirements for science data bases and scidb. In Conference on Innovative Data Systems Research (CIDR), January 2009.
25. Yu Su and Gagan Agrawal. Supporting user-defined subsetting and aggregation over parallel netcdf datasets. In 12th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid), pages 212-219. IEEE, 2012.
26. Yu Su, Gagan Agrawal, and Jonathan Woodring. Indexing and parallel query processing support for visualizing climate datasets. In 41th IEEE/ACM International Conference on Parallel Processing (ICPP), pages 249-258. IEEE, 2012.
27. Daniel L Wang, Charles S Zender, and Stephen F Jenks. Clustered workflow execution of retargeted data analysis scripts. In 8th IEEE International Symposium on Cluster, Cloud and Grid Computing (CCGrid), pages 449-458. IEEE, 2008.
28. Yi Wang, Yu Su, and Gagan Agrawal. Supporting a light-weight data management layer over hdf5. In 13th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid). IEEE, 2013.
29. Li Weng, Gagan Agrawal, Umit Catalyurek, Tahsin Kurc, Sivaramakrishnan Narayanan, and Joel Saltz. An approach for automatic data virtualization. In Proceedings of the Conference on High Performance Distributed Computing (HPDC), June 2004.
30. K. Wu, W. Koegler, J. Chen, and A. Shoshani. Using bitmap index for interactive exploration of large datasets. In 15th International Conference on Scientific and Statistical Database Management, pages 65-74. IEEE, 2003.
31. K. Wu, E. J. Otoo, and A. Shoshani. Compressing bitmap indexes for faster search operations. In 14th International Conference on Scientific and Statistical Database Management, pages 99-108. IEEE, 2002.
32. K. Wu, K. Stockinger, and A. Shoshani. Breaking the curse of cardinality on bitmap indexes. In 20th International Conference on Scientific and Statistical Database Management, pages 348-365. Springer, 2008.
33. Kesheng Wu, W. Koegler, J. Chen, and A. Shoshani. Using bitmap index for interactive exploration of large datasets. In 15th International Conference on Scientific and Statistical Database Management, 2003, pages 65-74. IEEE, July 2003.