Challenges of scale: When all computing becomes grid computing

Advances in Parallel Computing

Volumn 16, Issue , 2008, Pages 186-206

  Fowler, Robert J ^a   Gamblin, Todd ^a   Kandaswamy, Gopi ^a   Mandal, Anirban ^a   Porterfield, Allan K ^a   Ramakrishnan, Lavanya ^a   Reed, Daniel A ^a  

^a UNIVERSITY OF NORTH CAROLINA   (United States)

Author keywords

bioinformatics; mesoscale weather; multidisciplinary science; performance analysis; Petascale; reliability; workflows

Indexed keywords

EID: 60349114552     PISSN: 09275452     EISSN: None     Source Type: Book Series    
DOI: None     Document Type: Conference Paper

References (51)

1. M. Eastwood and K. Cayton, "Quad-Core Processors Bring Higher Performance and Lower Cost to Mainstream Computing," Intel White Paper, April 2007
2. E. Korpela, D Werthimer, D. Anderson, J. Cobb and M. Lebofsky, "SETI@home-Massively Distributed Computing for SETI," IEEE Multimedia , Vol. 3, No. 1 pp. 78-93, January 1996
3. M. Shirts and V. Pande, "Screen Savers of the World Unite!," Science, Vol. 290 no. 5498, 8 December 2000
4. Y. Zhao, M. Wilde, I. Foster, J. Voeckler, J. Dobson, E. Glibert, T. Jordan and E. Quigg, "Virtual Data Grid Middleware Services for Data-Intensive Science," Concurrency and Computation: Practice and Experience, Vol 18(6), May 2006
5. J. Michalakes, J. Dudhia, D. Gill, T. Henderson, J. Klemp, W. Skamarock, and W. Wang, "The Weather Research and Forecast Model: Software Architecture and Performance," Proceedings of the 11th ECMWF Workshop on the Use of High Performance Computing in Meteorology, October 2004
6. I. Foster, C. Kesselman, J. Nick and S. Tuecke, "Grid Services for Distributed System Integration," Computer, 35(6), 2002
7. Renaissance Computing Institute, www.renci.org, 2007
8. K. K. Droegemeier, D. Gannon, D. Reed, B. Plale, J. Alameda, T. Baltzer, K. Brewster, R. Clark, B. Domenico, S. Graves, E. Joseph, D. Murray, R. Ramachandran, M. Ramamurthy, L. Ramakrishnan, J. A. Rushing, D. Weber, R. Wilhelmson, A. Wilson, M. Xue and S. Yalda, "Service-Oriented Environments for Dynamically Interacting with Mesoscale Weather," IEEE Computational Science and Engineering, Vol. 7, No. 6, pp. 12-29, December 2005
9. "Leadership-Class System Acquisition-Creating a Petascale Computing Environment for Science and Engineering (program solicitation)," http://www.nsf.gov/pubs/2006/nsf06573/nsf06573.html, 2006
10. J. Dongarra et al, "Top500 Supercomputer Sites," www.top500.org, 2007
11. C. W. McCurdy, R. Stevens, H. Simon, W. Kramer, D. Bailey, W. Johnston, C. Catlett, R. Lusk, T. Morgan, J. Meza, M. Banda, J. Leighton and J. Hules, "Creating Science-Driven Computer Architecture: A New Path to Scientific Leadership," Lawrence Berkeley National Laboratory, LBNL/PUB-5483, October 2002
12. D. A. Reed, C-D. Lu and C. L. Mendes, "Reliability Challenges in Large Systems," Future Generation Computer Systems, Vol. 22, No. 3, pp. 293-302, February 2006
13. E. Pinheiro, W. Weber and L.A. Barroso, "Failure Trends in a Large Disk Population," Proceedings of the 5th USENIX Conference on File and Storage Technologies (FAST07), February 2007
14. C-D. Lu and D. A. Reed, "Assessing Fault Sensitivity in MPI Applications," Proceedings of SC2004, Pittsburgh, November 2004
15. O. Khalili, J. He, C. Olschanowsky, A. Snavely, and H. Casanova, "Measuring the Performance and Reliability of Production Computational Grids," 7th IEEE/ACM International Conference on Grid Computing, 2006
16. G. E. Fagg, E. Gaberiel, G, Bosilca, T. Spring, Z. Angskun, and J. Hayes, "The Network Weather Service: A Distributed Resource Performance Forecasting Service for Metacomputing," Future Generation Computer Systems, Vol 15(5-6), pp. 757-768, 1999
17. E. N. Elnozahy, L. Alvisi, Y. M. Wang and D. B. Johnson, "A Survey of Rollback-Recovery Protocols in Message Passing Systems," ACM Computing Surveys, Vol. 34 issue 3, September 2002
18. T. Andrews, F. Curbera, H. Dholakia, Y. Goland, J. Klein, F.Leymann, K. Liu, D. Roller, D. Smith, S. Thatte, I. Trickovic and S. Weerawarana, "Business Process Execution Language for Web Services," http://www.ibm.com/developerworks/webservices/library/ws-bpel/, May 2003
19. M. Feller, I. Foster and S. Martin, "GT4 GRAM: A Functionality and Performance Study," TeraGrid 2007 Conference, 2007
20. K. Czajkowski, S. Fitzgerald, I. Foster and C. Kesselman. "Grid Information Services for Distributed Resource Sharing," Proceedings of the Tenth International Symposium on High-Performance Distributed Computing, 2001
21. A. Bayucan, R. Henderson, C. Lesiak, B. Mann, T. Proett and D. Tweten, "Portable Batch System: External Reference Specification," MRJ Technology Solutions, November 1999
22. K. Czajkowski, I. Foster, N. Karonis, C. Kesselman, S. Martin, W. Smith and S. Tuecke, "A Resource Management Architecture for Metacomputing Systems," 4th Workshop on Job Scheduling Strategies for Parallel Processing, Springer-Verlag, 1998
23. I. Foster, C. Kesselman, G. Tsudik and S. Tuecke, "A Security Architecture for Computational Grids," Proceedings of the 5th ACM Conference on Computer and Communications Security Conference, 1998
24. W. Allcock, J. Bresnahan, R. Kettimuthu, M. Link, C. Dumitrescu, I. Raicu and I. Foster, "The Globus Striped GridFTP Framework and Server," Proceedings of Super Computing 2005 (SC05), November 2005
25. B. Plale, D. Gannon, J. Brotzge, K. Droegemeier, J. Kurose, D. Mclaughlin, R. Wilhelmson, S. Graves, M. Ramamurhty, R. D. Clark, S. Yalda, D.A. Reed, E. Joseph and V. Chandraeskar, "CASA and LEAD: Adaptive Cyberinfrastructure for Real-Time Multiscale Weather Forecasting," IEEE Computer, Vol. 39, No. 11, pp. 56-64, November 2006
26. Y. Kee, D. Logothetis, R. Huang, H. Casanova and A. Chien, "Efficient Resource Description and High Quality Selection for Virtual Grids," Proceedings of the 5th IEEE Symposium on Cluster Computing and the Grid (CCGrid'05), 2005
27. President's IT Advisory Committee (PITAC), Computational Science: Ensuring America's Competitiveness, June 2005
28. C. L. Mendes and D. A. Reed, "Monitoring Large Systems via Statistical Sampling," The International Journal of High Performance Computing Applications, Vol.18, No.2, Summer 2004, pp.267-277
29. T. Gamblin, R. Fowler and D. A. Reed, "Scalable Methods for Monitoring and Detecting Behavioral Classes in Scientific Codes," in preparation, 2007
30. F. Petrini, D. Kerbyson and S. Pakin, "The Case of the Missing Supercomputer Performance: Achieving Optimal Performance on the 8,192 Processors of ASCI Q," Proceedings of SC03, Phoenix Arizona, November 2003
31. R. A. Luettich, J. J. Westerink and N. W. Scheffner. "ADCIRC: An Advanced Three-Dimensional Circulation Model of Shelves, Coasts and Estuaries, Report 1: Theory and Methodology of ADCIRC-2DDI and ADCIRC-3DL," Dredging Research Program Technical Report DRP-92-6, U.S. Army Engineer Research and Development Center, Vicksburg, MS, 1992
32. R. A. Luettich and J. J. Westerink, "Theory Report: Formulation and Numerical Implementation of the 2D/3D ADCIRC Finite Element Model Version 43.XX," http://www.marine.unc/C-CATS/adcirc/adcirc.htm, March 2003
33. J. Mellor-Crummey, R. Fowler, G.l Marin, and N. Tallent, "HPCView: A Tool for Top-down Analysis of Node Performance," The Journal of Supercomputing, 23:81-104, 2002
34. N. Froyd, J. Mellor-Crummey and R. Fowler, "Efficient Call-Stack Profiling of Unmodified, Optimized Code," Proceedings of the International Conference on Supercomputing (ICS2005), pages 81-90, Cambridge, MA, June 2005
35. J. Dongarra, K. London, S. Moore, P. Mucci, D. Terpstra, H. You and M. Zhou, "Experiences and Lessons Learned with a Portable Interface to Hardware Performance Counters," PADTAD Workshop, IPDPS, 2003
36. International Committee on Information Technology Standards, "397-2005(1532d) AT Attachment with Packet Interface-7"
37. "Advanced Configuration and Power Interface, www.acpi.info/, October 2006
38. "IPMI-Intelligent Platform Management Interface Specification, Second Generation V2.0," Intel Hewlett-Packard NEC Dell, February 2004
39. R. Wolski, N. Pjesivac-Grbovic, K. London and J. Dongarra. "Extending the MPI Specification for Process Fault Tolerance on High Performance Computing Systems," International Supercomputing Conference (ICS) Heidelberg Germany, June 2004
40. J. Brevik, D. Nurmi and R. Wolski, "Predicting Bounds on Queuing Delay for Batch-scheduled Parallel Machines," Proceedings of the Eleventh ACM SIGPLAN Symposium on Principle and Practice of Parallel Programming, 2006
41. "XBaya: A Graphical Workflow Composer for Web Services," Indiana University, http://www.extreme.indiana.edu/xgws/xbaya/, 2007
42. L. Pearlman, C. Kesselman, S. Gullapalli, B. F. Spencer, J. Futrelle, K. Ricker, I. Foster, P. Hubbard, and C. Severance, "Distributed Hybrid Earthquake Engineering Experiments: Experiences with a Ground-Shaking Grid Application," Proceedings of the 13th IEEE international Symposium on High Performance Distributed Computing, pp. 14-23, June 2004
43. L. Ramakrishnan, M. S.C Reed, J. L. Tilson and D. A. Reed, "Grid Portals for Bioinformatics," Second International Workshop on Grid Computing Environments (GCE), Workshop at SC|06, November 2006, Tampa, Florida
44. Atkins et al, "Revolutionizing Science and Engineering Through Cyberinfrastructure," Report of the National Science Foundation Blue-Ribbon Advisory Panel on Cyberinfrastructure, January 2003
45. C. Catlett, "The Philosophy of TeraGrid: Building an Open, Extensible, Distributed Terascale Facility," Second IEEE/ACM International Symposium on Cluster Computing and the Grid (CGRID2002), p. 5, 2002
46. J. Alameda, M. Christie, G. Fox, J. Futrelle, D. Gannon, M. Hategan, G. Kandaswamy, G. V. Laszewski, M. A. Nacar, M. Pierce, E. Roberts, C. Severance, and M. Thomas, "The Open Grid Computing Environments Collaboration: Portlets and Services for Science Gateways," Concurrency and Computation : Practice & Experience, Vol 19, 6, pp 921-942, April 2007
47. C. Letondal, "A Web Interface Generator for Molecular Biology Programs in Unix," Bioinformatics, 17(1), pp 73-82, 2001
48. I. Foster, "Globus Toolkit Version 4: Software for Service-Oriented Systems," IFIP International Conference on Network and Parallel Computing, Springer-Verlag LNCS 3779, pp 2-13, 2006
49. T. Oinn, M. Addis, J. Ferris, D. Marvin, M. Greenwood, T. Carver, M. R. Pocock, A. Wipat and P. Li, "Taverna: A Tool for the Composition and Enactment of Bioinformatics Workflows," Bioinformatics, 20, pp. 3045-3054
50. J. Novotny, S. Tuecke and V. Welch, "An Online Credential Repository for the Grid: MyProxy," Proceedings of the Tenth IEEE International Symposium on High Performance Distributed Computing (HPDC-10), August 2001
51. L. Ramakrishnan and D. A. Reed, "Performability Modeling for Scheduling and Fault Tolerance Strategies for Grid Workflows," submitted to SC07, November 2007