Deep scientific computing requires deep data

IBM Journal of Research and Development

Volumn 48, Issue 2, 2004, Pages 209-232

  Kramer, W T C ^a   Shoshani, A ^a   Agarwal, D A ^a   Draney, B R ^a   Jin, G ^a   Butler, G F ^a   Hules, J A ^a  

^a LAWRENCE BERKELEY NATIONAL LABORATORY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTER NETWORKS; COMPUTER SIMULATION; DATA STORAGE EQUIPMENT; DATABASE SYSTEMS; ONLINE SYSTEMS; RESPONSE TIME (COMPUTER SYSTEMS); SECURITY OF DATA; SENSOR DATA FUSION;

DATA REPOSITORIES; DEEP SCIENTIFIC COMPUTING; DEEP STORAGE RESOURCES;

DISTRIBUTED COMPUTER SYSTEMS;

EID: 2542471673     PISSN: 00188646     EISSN: None     Source Type: Journal    
DOI: 10.1147/rd.482.0209     Document Type: Article

References (50)

1. "Towards Globus Toolkit 3.0: Open Grid Services Architecture," http://www.globus.org/ogsa/.
2. M. Alcubierre, B. Bruegmann, P. Diener, M. Koppitz, D. Pollney, E. Seidel, and R. Takahashi, "Gauge Conditions for Long-Term Numerical Black Hole Evolutions Without Excision," Phys. Rev. D 67, 084023 (2003).
3. "Lawrence Berkeley National Lab Leads International Team to Win High-Performance Computing Bandwidth Challenge," http://www.lbl.gov/CS/Archive/othernews112502.html.
4. J. Shalf and E. W. Bethel, "Cactus and Visapult: An Ultra-High Performance Grid-Distributed Visualization Architecture Using Connectionless Protocols," IEEE Computer Graph. & Appl. 23, no. 2, 51-59 (2003).
5. "NERSC Helps Climate Scientists Complete First Ever 1,000-Year Run of Nation's Leading Climate Change Modeling Application," http://www.lbl.gov/CS/Archive/headlines090402.html.
6. Community Climate System Model, http://www.ccsm.ucar.edu/.
7. The Nearby Supernova Factory, http://snfactory.lbl.gov/.
8. U.S. DOE Coupled Climate Model Data Archive, http://www.nersc.gov/projects/gcm_data/.
9. "The Earth System Grid II: Turning Climate Model Datasets into Community Resources," http://www.earthsystemgrid.org/.
10. "Analysis of Fugu rubripes Genome Predicts Nearly 1,000 New Human Genes," http://www.jgi.doe/gov/News/news_7_25_02.html.
11. "Computational Analysis of Genomic Sequence Data," http://www.nersc.gov/research/annrep01/sh_BER_06.html.
12. "High Performance Networks for High Impact Science," Report of the August 13-15, 2002 Workshop on Networking, conducted by the DOE Office of Science, http://doecollaboratory.pnl.gov/meetings/hpnpw/finalreport/ high-impact_science.pdf.
13. Fibre Channel Industry Association, http://www.fibrechannel.org/.
14. "NERSC Strategic Proposal FY2002-FY2006," http://www.nersc.gov/aboutnersc/pubs/Strategic_Proposal_final.pdf.
15. High Performance Storage System (HPSS); see http://www4.clearlake.ibm.com/hpss/.
16. IBM Tivoli® Storage Manager, http://www-3.ibm.com/software/tivoli/.
17. Storage Archive Manager File System, http://www.sun.com/storage/software/data_mgmt/utilization/.
18. VERITAS Software Corporation, http://www.veritas.com/.
19. Cray Inc. Data Migration Facility, http://www.cray.com/products/software/dmf.html.
20. "Reference Model for Open Storage System Interconnection - Mass Storage System Reference Model Version 5," developed by the IEEE Storage System Standards Working Group (Project 1244), Sept. 8, 1994, R. Garrison, Martin Marietta Corporation, Ed.; http://www.arl.mil/IEEE/ssswg.html.
21. 3PARdata, Inc., 4209 Technology Drive, Fremont, CA 94538; Phone: (510) 413-5999, http://www.3pardata.com/.
22. Panasas, Inc.; see http://www.panasas.com/.
23. YottaYotta Inc., 6020 - 104 Street, Edmonton, Alberta, T6H 5S4; Phone: (780) 989-6800; http://www.yottayotta.com/.
24. Julian Satran, Kalman Meth, Costa Sapuntzakis, Mallikarjun Chadalapaka, and Efri Zeidner, "iSCSI Internet Draft 19-January-03," http://www.ietf.org/internet-drafts/draft-ietf-ips-iscsi-20.txt
25. InfiniBand Trade Association, http://www.infinibandta.org/home/
26. Gregory F. Butler, Rei Chi Lee, and Michael L. Welcome, "The Global Unified Parallel File System (GUPFS) Project: FY 2002 Activities and Results," Lawrence Berkeley National Laboratory Technical Report LBNL-52456 (2003), http://www.nersc.gov/aboutnersc/pubs/GUPFS_02.pdf
27. Peter J. Braam and Rumi Zahir, "Lustre Technical Project Summary," Version 2, July 29, 2001; Cluster File Systems, Inc., Mountain View, CA, http://www.lustre.org/docs/lustre-sow-dist.pdf.
28. "General Parallel Life System (GPFS) for AIX®," http://www-1.ibm.com/servers/eserver/pseries/software/sp/gpfs.html.
29. Lustre; see http://www.lustre.org/.
30. Christos Karamanolis, Mallik Mahalingam, Dan Muntz, and Zheng Zhang, "DiFFS: A Scalable Distributed File System," Hewlett-Packard Laboratories Technical Report HPL-2001-19 (2001), http://www.hpl.hp.com/techreports/2001/HPL-2001-19.pdf.
31. Maximum Throughput InfinARRAY, http://www.max-t.com/html/products/infinarray.html.
32. Ibrix, Inc., http://www.ibrix.com/.
33. The Parallel Virtual File System (PVFS), http://www.parl.clemson.edu/pvfs/.
34. "Storage Resource Managers: Middleware Components for Grid Storage," presented at the Nineteenth IEEE Symposium on Mass Storage Systems (MSS02), 2002, http://sdm.lbl.gov/-arie/papers/srm.mss02.pdf.
35. Arie Shoshani, Alexander Sim, and Junmin Gu, "Storage Resource Management: Essentiall Components for the Grid," Grid Resource Management: State of the Art and Future Trends, J. Nabrzyski, J. M. Schopf, and J. Weglarz, Eds., Kluwer Academic Publishers, New York, 2003.
36. Particle Physics Data Grid, http://www.ppdg.net/.
37. Fermilab Mass Storage System, http://hppc.fnal.gov/enstore/.
38. JASMine - Jefferson Lab Asynchronous Storage Manager, http://cc.jlab.org/scicomp/JASMine/.
39. The CASTOR Project (CERN Advanced STORage Manager), http://castor.web.cern.ch/castor/.
40. Myricom®, Inc., http://www.myri.com/.
41. Quadrix Solutions, Inc., http://www.quadrix.com/.
42. G. Jin and B. Tierney, "System Capability Effects on Algorithms for Network Bandwidth Measurement," Proceedings of the Internet Measurement Conference, Miami, October 27-29, 2003; Lawrence Berkeley National Laboratory Report LBNL-48556, http://dsd.lbl.gov/DIDC/papers/imc-2003.pdf.
43. C. William McCurdy, Rick Stevens, Horst Simon, William Kramer, David Bailey, William Johnston, Charlie Catlett, Rusty Lusk, Thomas Morgan, Juan Meza, Michael Banda, James Leighton, and John Hules, "Creating Science-Driven Computer Architecture: A New Path to Scientific Leadership," Lawrence Berkeley National Laboratory Report LBNL/PUB-5483 (2002), http://www.nersc.gov/news/ArchDevProposal.5.01.pdf.
44. T. Dunigan, M. Mathis, and B. Tierney, "A TCP Tuning Daemon," Proceedings of the IEEE/ACM SC2002 Conference, November 2002, http://www-didc.lbl.gov/papers/net100.sc02.final.pdf.
45. S. Floyd, "High Speed TCP for Large Congestion Windows," Internet draft, 2003, http://www.icir/org/floyd/papers/draft-floyd-tcp-highspeed-03.txt
46. B. Tierney, "TCP Tuning Guide," http://www-didc.lbl.gov/TCP-tuning/TCP-tuning.html.
47. C. Jin, D. Wei, S. H. Low, G. Buhrmaster, J. Bunn, D. H. Choe, R. L. A. Cottrell, J. C. Doyle, H. Newman, F. Paganini, S. Ravot, and S. Singh, "FAST Kernel: Background Theory and Experimental Results," presented at the First International Workshop on Protocols for Fast Long-Distance Networks, February 3-4, 2003, CERN, Geneva, Switzerland, http://netlab.caltech.edu/pub/papers/pfldnet.pdf.
48. V. Paxson, "Bro: A System for Detecting Network Intruders in Real-Time," Computer Networks 31, 2435-2463 (1999).
49. D. Gunter, B. Tierney, C. E. Tull, and V. Virmani, "On Demand Grid Application Tuning and Debugging with the NetLogger Activation Service," presented at the Fourth International Workshop on Grid Computing (Grid2003), Phonix, AZ, November 17, 2003.
50. D. Agarwal, J. M. González, G. Jin, and B. Tierney, "An Infrastructure for Passive Network Monitoring of Application Data Streams," presented at the 2003 Passive and Active Measurement Workshop, San Diego, April 2003.