Extending I/O through high performance data services

Proceedings - IEEE International Conference on Cluster Computing, ICCC

Volumn , Issue , 2009, Pages

  Abbasi, Hasan ^a   Lofstead, Jay ^a   Zheng, Fang ^a   Schwan, Karsten ^a   Wolf, Karsten ^a   Klasky, Scott ^b  

^a GEORGIA INSTITUTE OF TECHNOLOGY   (United States)

^b OAK RIDGE NATIONAL LABORATORY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ASYNCHRONOUS I/O; CONTROL ASPECTS; CONTROL PLANES; DATA MOVEMENTS; DATA PLANES; DATA SERVICES; DATA VOLUME; I/O PERFORMANCE; INFORMATION CONTENTS; OUTPUT DATA; PERFORMANCE DATA; PROCESSING CORE; RUNTIMES; SERVICE FUNCTIONS;

CLUSTER COMPUTING; COMPUTER SCIENCE; DATA REDUCTION;

TECHNICAL PRESENTATIONS;

EID: 72049132104     PISSN: 15525244     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/CLUSTR.2009.5289167     Document Type: Conference Paper

References (36)

1. H. Abbasi, M. Wolf, and K. Schwan. LIVE Data Workspace: A Flexible, Dynamic and Extensible Platform for Petascale Applications. Cluster Computing, 2007. IEEE International, Sept. 2007.
2. Hasan Abbasi, Greg Eisenhauer, Matthew Wolf, and Karsten Schwan. A high performance infrastructure for composable event overlays. To appear in the Proceedings of HPDC 2009.
3. Sandip Agarwala, Greg Eisenhauer, and Karsten Schwan. Morphable messaging: Efficient support for evolution in distributed applications. In Challenges of Large Applications in Distributed Environments (CLADE), June 2004.
4. Michael Beynon, Renato Ferreira, Tahsin M. Kurc, Alan Sussman, and Joel H. Saltz. Datacutter: Middleware for filtering very large scientific datasets on archival storage systems. In IEEE Symposium on Mass Storage Systems, pages 119-134, 2000.
5. Fabian E. Bustamante, Greg Eisenhauer, Karsten Schwan, and Patrick Widener. Efficient wire formats for high performance computing. In Proceedings of the 2000 ACM/IEEE conference on Supercomputing (CDROM), page 39. IEEE Computer Society, 2000.
6. Z. Cai, V. Kumar, and K. Schwan. Iq-paths: Self-regulating data streams across network overlays. In Proceedings of the IEEE Symposium on High Performance Distributed Computing, Paris, France, 2006.
7. Philip H. Carns, Walter B. Ligon III, Robert B. Ross, and Rajeev Thakur. PVFS: A parallel file system for linux clusters. In Proceedings of the 4th Annual Linux Showcase and Conference, pages 317-327, Atlanta, GA, 2000. USENIX Association.
8. Lustre: A scalable, high-performance file system. Cluster File Systems Inc. white paper, version 1.0, November 2002. http://www.lustre.org/docs/ whitepaper.pdf.
9. Jeffrey Dean and Sanjay Ghemawat. Mapreduce: simplified data processing on large clusters. In OSDI'04: Proceedings of the 6th conference on Symposium on Opearting Systems Design & Implementation, pages 10-10, Berkeley, CA, USA, 2004. USENIX Association.
10. Ewa Deelman, James Blythe, Yolanda Gil, Carl Kesselman, Gaurang Mehta, Karan Vahi, Kent Blackburn, Albert Lazzarini, Adam Arbree, Richard Cavanaugh, and Scott Koranda. Mapping abstract complex workflows onto grid environments. J. Grid Comput., 1(1):25-39, 2003.
11. C. Docan, M. Parashar, and S. Klasky. High speed asynchronous data transfers on the cray xt3. In Cray User Group Conference, 2007.
12. Greg Eisenhauer. The evpath library. http://www.cc.gatech.edu/systems/ projects/EVPath.
13. Greg Eisenhauer. Portable binary input/output. http://www.cc.gatech.edu/ systems/projects/PBIO.
14. Greg Eisenhauer. The ECho Event Delivery System. Technical Report GIT-CC-99-108, College of Computing, Georgia Institute of Technology, 1999. (http://www.cc.gatech.edu/tech reports).
15. Greg Eisenhauer, Fabian Bustamente, and Karsten Schwan. Event services for high performance computing. In Proceedings of High Performance Distributed Computing (HPDC-2000), 2000.
16. I. Foster, J. Vockler, M. Wilde, and Y. Zhao. Chimera: A virtual data system for representing, querying, and automating data derivation. In Scientific and Statistical Database Management, 2002. Proceedings. 14th International Conference on, pages 37-46, 2002.
17. David Kotz. Disk-directed i/o for mimd multiprocessors. ACM Trans. Comput. Syst., 15(1):41-74, 1997.
18. Rob Latham, Neil Miller, Robert Ross, and Phil Carns. A next-generation parallel file system for linux clusters. Linux- World, 2(1), January 2004.
19. Jay Lofstead, Fang Zheng, Scott Klasky, and Karsten Schwan. Input/output apis and data organization for high performance scientific computing. In In Proceedings of Petascale Data Storage Workshop 2008 at Supercomputing 2008, 2008.
20. Jay Lofstead, Fang Zheng, Scott Klasky, and Karsten Schwan. Adaptable, metadata rich io methods for portable high performance io. In In Proceedings of IPDPS'09, May 25-29, Rome, Italy, 2009.
21. Jay F. Lofstead, Scott Klasky, Karsten Schwan, Norbert Podhorszki, and Chen Jin. Flexible io and integration for scientific codes through the adaptable io system (adios). In CLADE '08: Proceedings of the 6th international workshop on Challenges of large applications in distributed environments, pages 15-24, New York, NY, USA, 2008. ACM.
22. Bertram Ludscher, Ilkay Altintas, Chad Berkley, Dan Higgins, Efrat Jaeger, Matthew Jones, Edward A. Lee, Jing Tao, and Yang Zhao. Scientific workflow management and the kepler system: Research articles. Concurr. Comput. : Pract. Exper., 18(10):1039-1065, 2006.
23. O. E. B. Messer, S. W. Bruenn, J. M. Blondin, W. R. Hix, A. Mezzacappa, and C. J. Dirk. Petascale supernova simulation with CHIMERA. Journal of Physics Conference Series, 78(1):012049-+, July 2007.
24. Ron A. Oldfield, Arthur B. Maccabe, Sarala Arunagiri, Todd Kordenbrock, Rolf Rie sen, Lee Ward, and Patrick Widener. Lightweight I/O for Scientific Applications. In Proc. 2006 IEEE Conference on Cluster Computing, Barcelona, Spain, September 2006.
25. Ron A. Oldfield, Patrick Widener, Arthur B. Maccabe, Lee Ward, and Todd Kordenbrock. Efficient Data Movement for Lightweight I/O. In Proc. 2006 Workshop on highperformance I/O techniques and deployment of Very-Large Scale I/ O Systems (HiPerI/O 2006), Barcelona, Spain, September 2006.
26. th International Conference on Very Large Databases, August 1998.
27. Erik Riedel, Christos Faloutsos, Garth A. Gibson, and David Nagle. Active disks for large-scale data processing. IEEE Computer, 34(6):68-74, June 2001.
28. F. Schmuck and R. Haskin. GPFS: A Shared-Disk File System for Large Computing Clusters. Proceedings of the 1st USENIX Conference on File and Storage Technologies, 2002.
29. K. E. Seamons, Y. Chen, P. Jones, J. Jozwiak, and M. Winslett. Server-directed collective i/o in panda. In Supercomputing '95: Proceedings of the 1995 ACM/IEEE conference on Supercomputing (CDROM), page 57, New York, NY, USA, 1995. ACM.
30. Balasubramanian Seshasayee and Karsten Schwan. Mobile service overlays: reconfigurable middleware for manets. In MobiShare '06: Proceedings of the 1st international workshop on Decentralized resource sharing in mobile computing and networking, pages 30-35, New York, NY, USA, 2006. ACM.
31. NTB Stone, D. Balog, B. Gill, B. Johan-SON, J. Marsteller, P. Nowoczynski, D. Porter, R. Reddy, JR Scott, D. Simmel, et al. PDIO: High-performance remote file I/O for Portals enabled compute nodes. Proceedings of the 2006 Conference on Parallel and Distributed Processing Techniques and Applications, Las Vegas, NV, June, 2006.
32. Douglas Thain, Todd Tannenbaum, and Miron Livny. Distributed computing in practice: the condor experience. Concurrency - Practice and Experience, 17(2-4):323-356, 2005.
33. WX Wang, Z. Lin, WM Tang, WW Lee, S. Ethier, JLV Lewandowski, G. Rewoldt, TS Hahm, and J. Manickam. Gyro-kinetic simulation of global turbulent transport properties in tokamak experiments. Physics of Plasmas, 13:092505, 2006.
34. Patrick M. Widener, Matthew Wolf, Hasan Abbasi, Matthew Barrick, Jay Lofstead, Jack Pullikottil, Greg Eisenhauer, Ada Gavrilovska, Scott Klasky, Ron Oldfield, Patrick G. Bridges, Arthur B. Maccabe, and Karsten Schwan. Structured streams: Data services for petascale science environments. Technical Report TR-CS-2007-2017, University of New Mexico, Albuquerque, NM, November 2007.
35. Matthew Wolf, Hasan Abbasi, Benjamin Collins, David Spain, and Karsten Schwan. Service augmentation for high end interactive data services. In IEEE International Conference on Cluster Computing (Cluster 2005), September 2005.
36. Jia Yu and Rajkumar Buyya. A taxonomy of scientific workflow systems for grid computing. SIGMOD Rec., 34(3):44-49, 2005. (Pubitemid 43152531)