PANORAMA: An approach to performance modeling and diagnosis of extreme-scale workflows

International Journal of High Performance Computing Applications

Volumn 31, Issue 1, 2017, Pages 4-18

  Deelman, Ewa ^a   Carothers, Christopher ^b   Mandal, Anirban ^c   Tierney, Brian ^d   Vetter, Jeffrey S ^e   Baldin, Ilya ^c   Castillo, Claris ^c   Juve, Gideon ^a   Król, Dariusz ^a,f   Lynch, Vickie ^e   Mayer, Ben ^e   Meredith, Jeremy ^e   Proffen, Thomas ^e   Ruth, Paul ^c   Ferreira da Silva, Rafael ^a  

^a UNIVERSITY OF SOUTHERN CALIFORNIA   (United States)

^b RENSSELAER POLYTECHNIC INSTITUTE   (United States)

^c UNIVERSITY OF NORTH CAROLINA   (United States)

^d LAWRENCE BERKELEY NATIONAL LABORATORY   (United States)

^e OAK RIDGE NATIONAL LABORATORY   (United States)

^f AGH UNIVERSITY OF SCIENCE AND TECHNOLOGY   (Poland)

Author keywords

extreme scale; Performance modeling; scientific workflow

Indexed keywords

COMPUTER SIMULATION LANGUAGES; DATA HANDLING; ENTERPRISE RESOURCE MANAGEMENT; INFORMATION ANALYSIS; WORK SIMPLIFICATION;

COMPUTATIONAL SCIENCE; EXTREME SCALE; PERFORMANCE MODEL; PERFORMANCE OPTIMIZATIONS; RUN-TIME PERFORMANCE; SCIENTIFIC WORKFLOWS; TESTBED EXPERIMENTATIONS; WORKFLOW MANAGEMENT SYSTEMS;

INFORMATION MANAGEMENT;

EID: 85008474111     PISSN: 10943420     EISSN: 17412846     Source Type: Journal    
DOI: 10.1177/1094342015594515     Document Type: Article

References (68)

1. Alexandrov A, Ionescu MF, Schauser KE, et al. (1995) LogGP: Incorporating long messages into the LogP model. In: 7th annual ACM symposium on parallel algorithms and architectures, pp. 95-105.
2. Arnold O, Bilheux JC, Borreguero JM, et al. (2014) Mantid: Data analysis and visualization package for neutron scattering and μ SR experiments. Nuclear Instruments and Methods in Physics Research Section A 764: 156-166.
3. Baldin I, Xin Y, Evans D, et al. (2012) Exogeni: A multi-domain infrastructure-as-a-service testbed. In: 8th international ICST conference on testbeds and research infrastructures for the development of networks and communities, pp. 97-113.
4. Barnes PD Jr, Carothers CD, Jefferson D, et al. (2013) Warp speed: executing time warp on 1,966,080 cores. In: ACM SIGSIM conference on principles of advanced discrete simulation, pp. 327-336. DOI: 10.1145/2486092.2486134.
5. Bauer DW Jr, Carothers CD, Holder A, (2009) Scalable time warp on blue gene supercomputers. In: ACM/IEEE/SCS 23rd workshop on principles of advanced and distributed simulation, pp. 35-44.
6. Bharathi S, Chervenak A, Deelman E, et al. (2008) Characterization of scientific workflows. In: 3rd workshop on workflows in support of large-scale science, pp. 1-10. DOI: 10.1109/WORKS.2008.4723958
7. Carothers CD, Perumalla KS, (2010) On deciding between conservative and optimistic approaches on massively parallel-platforms. In: Winter simulation conference'10, pp. 678-687.
8. Carothers CD, Perumalla KS, Fujimoto RM, (1999) Efficient optimistic parallel simulations using reverse computation. ACM Transactions on Modeling and Computer Simulation 9 (3): 224-253. DOI: 10.1145/347823.347828.
9. Case D, Berryman JT, Betz RM, et al. (2015) AMBER 15. San Francisco, CA: University of California.
10. Chen W, Ferreira da Silva R, Deelman E, et al. (2014) Using imbalance metrics to optimize task clustering in scientific workflow executions. Future Generation Computer Systems. DOI:10.1016/j.future.2014.09.014.
11. Culler D, Karp R, Patterson DA, et al. (1993) LogP: Towards a realistic model of parallel computation. ACM SIGPLAN Notices 28 (7): 1-12.
12. Czechowski K, Battaglino C, McClanahan C, et al. (2012) On the communication complexity of 3D FFTs and its implications for exascale. In: 26th ACM international conference on supercomputing, pp. 205-214. DOI: 10.1145/2304576.2304604.
13. da Cruz S, da Silva F, Gadelha LMR Jr, et al. (2008) A lightweight middleware monitor for distributed scientific workflows. In: 8th IEEE international symposium on cluster computing and the grid, pp. 693-698. DOI: 10.1109/CCGRID.2008.89.
14. Deelman E, Blythe J, Gil Y, et al. (2002) Pegasus: Planning for execution in grids. GriPhyN technical report 20: 1-6.
15. Deelman E, Callaghan S, et al. (2006) Managing large-scale workflow execution from resource provisioning to provenance tracking: The Cybershake example. In: 2nd IEEE international conference on e-Science and grid computing, pp. 4-6.
16. Deelman E, Chervenak A, (2008) Data management challenges of data-intensive scientific workflows. IEEE international symposium on cluster computing and the grid, pp. 687-692. DOI: 10.1109/CCGRID.2008.24.
17. Deelman E, Gannon D, Shields M, et al. (2009) Workflows and e-science: An overview of workflow system features and capabilities. Future Generation Computer Systems 25 (5): 528-540.
18. Deelman E, Singh G, Su M, et al. (2005) Pegasus: A framework for mapping complex scientific workflows onto distributed systems. Scientific Programming 13 (3): 219-237.
19. Deelman E, Vahi K, Juve G, et al. (2015) Pegasus, a workflow management system for science automation. Future Generation Computer Systems 46: 17-35. DOI: 10.1016/j.future.2014.10.008
20. Ferreira da Silva R, Chen W, et al. (2014a) Community resources for enabling and evaluating research on scientific workflows. In: 10th IEEE international conference on e-Science, pp. 177-184. DOI: 10.1109/eScience.2014.44.
21. Ferreira da Silva R, Glatard T, (2013) A science-gateway workload archive to study pilot jobs, user activity, bag of tasks, task sub-steps, and workflow executions. In: Euro-Par 2012: Parallel Processing Workshops (Lecture Notes in Computer Science, vol. 7640). New York: Springer, pp. 79-88. DOI: 10.1007/978-3-642-36949-0-10.
22. Ferreira da Silva R, Glatard T, et al. (2013a) Self-healing of workflow activity incidents on distributed computing infrastructures. Future Generation Computer Systems 29 (8): 2284-2294. DOI: 10.1016/j.future.2013.06.012.
23. Ferreira da Silva R, Glatard T, et al. (2014 b) Controlling fairness and task granularity in distributed, online, non-clairvoyant workflow executions. Concurrency and Computation: Practice and Experience 26 (14): 2347-2366. DOI: 10.1002/cpe.3303.
24. Ferreira da Silva R, Juve G, Deelman E, et al. (2013b) Toward fine-grained online task characteristics estimation in scientific workflows. In: 8th workshop on workflows in support of large-scale science, pp. 58-67. DOI: 10.1145/2534248.2534254.
25. Ferreira da Silva R, Rynge M, Juve G, et al. (2015) Characterizing a high throughput computing workload: The Compact Muon Solenoid (CMS) experiment at LHC. In: 2015 international conference on computational science (ICCS 2015).
26. Gahvari H, Gropp W, (2010) An introductory exascale feasibility study for FFTs and multigrid. In: IEEE international symposium on parallel distributed processing, pp. 1-9. DOI: 10.1109/IPDPS.2010.5470417.
27. Germain-Renaud C, Cady A, Gauron P, et al. (2011) The grid observatory. In: IEEE international symposium on cluster computing and the grid, pp. 114-123.
28. Gunter D, Deelman E, et al. (2011) Online workflow management and performance analysis with stampede. In: 7th international conference on network and service management, pp. 1-10.
29. Hart DL, (2011) Measuring teragrid: workload characterization for a high-performance computing federation. International Journal of High Performance Computing Applications 25 (4): 451-465.
30. Iosup A, Epema D, (2011) Grid computing workloads. IEEE Internet Computing 15 (2): 19-26. DOI:10.1109/MIC. 2010.130.
31. Iosup A, Li H, Jan M, et al. (2008) The grid workloads archive. Future Generation Computer Systems 24 (7): 672-686. DOI: 10.1016/j.future.2008.02.003.
32. Janssen CL, Adalsteinsson H, Kenny JP, (2011) Using simulation to design extremescale applications and architectures: Programming model exploration. SIGMETRICS Performance Evaluation Review 38 (4): 4-8.
33. Juve G, Chervenak A, Deelman E, et al. (2013) Characterizing and profiling scientific workflows. Future Generation Computer Systems 29 (3): 682-692. DOI:10.1016/j.future. 2012.08.015.
34. Juve G, Tovar B, et al. (2014) Practical resource monitoring for robust high throughput computing. Technical report, University of Southern California.
35. Kephart J, Chess D, (2003) The vision of autonomic computing. Computer 36 (1): 41-50. DOI:10.1109/MC.2003. 1160055.
36. Kondo D, Javadi B, Iosup A, et al. (2010) The failure trace archive: Enabling comparative analysis of failures in diverse distributed systems. In: 10th IEEE/ACM international conference on cluster, cloud and grid computing, pp. 398-407.
37. Lee S, Meredith JS, Vetter JS, (2015) Zermatt: A framework for automated performance modeling and online performance prediction. In: ACM international conference on supercomputing, Newport Beach, CA.
38. Lindner B, Smith J, (2012) Sassena: X-ray and neutron scattering calculated from molecular dynamics trajectories using massively parallel computers. Computer Physics Communications 183 (7): 1491-1501.
39. Liu N, Cope J, Carns P, et al. (2012) On the role of burst buffers in leadership-class storage systems. In: IEEE 28th symposium on mass storage systems and technologies, pp. 1-11. DOI: 10.1109/MSST.2012.6232369.
40. Madougou S, Shahand S, Santcroos M, et al. (2013) Characterizing workflow-based activity on a production e-infrastructure using provenance data. Future Generation Computer Systems 29 (8): 1931-1942. DOI: 10.1016/j.future.2013.04.019.
41. Mahambre S, Kulkarni P, Bellue U, et al. (2012) Workload characterization for capacity planning and performance management in IAAS cloud. In: IEEE international conference on cloud computing in emerging markets, pp. 1-7. DOI: 10.1109/CCEM.2012.6354624.
42. Mandal A, Baldin I, Xin Y, et al. (2014) Enabling persistent queries for cross-aggregate performance monitoring. IEEE Communications Magazine 52 (5): 157-164.
43. Mandal A, Ruth P, Baldin I, et al. (2013) Evaluating I/O aware network management for scientific workflows on networked clouds. In: 3rd international workshop on network-aware data management, pp. 2:1-2:10. DOI: 10.1145/2534695.2534698.
44. Mao M, Humphrey M, (2011) Auto-scaling to minimize cost and meet application deadlines in cloud workflows. In: International conference for high performance computing, networking, storage and analysis, pp. 49:1-49:12. DOI: 10.1145/2063384.2063449.
45. Mason T, Abernathy D, Anderson I, et al. (2006) The Spallation Neutron Source in Oak Ridge: A powerful tool for materials research. Physica B: Condensed Matter 385: 955-960.
46. Mathis M, Semke J, Mahdavi J, et al. (1997) The macroscopic behavior of the TCP congestion avoidance algorithm. SIGCOMM Computer Communication Review 27 (3): 67-82. DOI: 10.1145/263932.264023.
47. Mubarak M, Carothers CD, Ross R, et al. (2012) Modeling a million-node dragonfly network using massively parallel discrete event simulation. In: 3rd international workshop on performance modeling, benchmarking and simulation of high performance computer systems, pp. 366-376. DOI: 10.1109/SC.Companion.2012.56.
48. Ostermann S, Plankensteiner K, Prodan R, et al. (2009) Workflow monitoring and analysis tool for askalon. In: Grid and Services Evolution. New York: Springer, pp. 1-14. DOI: 10.1007/978-0-387-85966-8-6.
49. Ostermann S, Prodan R, et al. (2008a) On the characteristics of grid workflows. In: CoreGRID Symposium-Euro-Par 2008, pp. 1-12.
50. Ostermann S, Prodan R, et al. (2008b) A trace-based investigation of the characteristics of grid workflows. In: Priol T, Vanneschi M, (eds.) From Grids to Service and Pervasive Computing. New York: Springer, pp. 191-203. DOI: 10.1007/978-0-387-09455-7-14.
51. Ostermann S, Prodan R, et al. (2010) Dynamic cloud provisioning for scientific grid workflows. In: 11th IEEE/ACM international conference on grid computing, pp. 97-104. DOI: 10.1109/GRID.2010.5697953.
52. Phillips JC, Braun R, Wang W, et al. (2005) Scalable molecular dynamics with NAMD. Journal of Computational Chemistry 26 (16): 1781-1802.
53. Ramakrishnan L, Gannon D, (2008) A survey of distributed workflow characteristics and resource requirements. Technical Report, Indiana University. Available at: http://www.cs.indiana.edu/pub/techreports/TR671.pdf.
54. Ren Z, Xu X, Wan J, et al. (2012) Workload characterization on a production Hadoop cluster: A case study on Taobao. In: IEEE international symposium on workload characterization, pp. 3-13. DOI: 10.1109/IISWC.2012.6402895.
55. Rodrigues AF, Hemmert KS, Barrett BW, et al. (2011) The structural simulation toolkit. SIGMETRICS Performance Evaluation Review 38 (4): 37-42.
56. Russell N, Aalst W, et al. (2006) Workflow exception patterns. In: Dubois E, Pohl K, (eds.) Advanced Information Systems Engineering (Lecture Notes in Computer Science, vol. 4001). Berlin: Springer, pp. 288-302. DOI: 10.1007/11767138-20.
57. Samak T, Gunter D, Goode M, Deelman E, Juve G, Mehta G, Silva F, Vahi K, (2011 a) Online fault and anomaly detection for large-scale scientific workflows. In: 2011 IEEE 13th international conference on high performance computing and communications (HPCC). IEEE, pp. 373-381.
58. Samak T, Gunter D, Goode M, Deelman E, Mehta G, Silva F, Vahi K, (2011 b) Failure prediction and localization in large scientific workflows. In: 6th workshop on workflows in support of large-scale science. ACM, pp. 107-116. DOI: 10.1145/2110497.2110510.
59. Singh G, Kesselman C, et al. (2006) Application-level resource provisioning on the grid. In: 2nd IEEE international conference on e-science and grid computing. DOI: 10.1109/E-SCIENCE.2006.261167.
60. Spafford K, Vetter JS, (2012) Aspen: A domain specific language for performance modeling. In: ACM/IEEE international conference for high performance computing, networking, storage, and analysis, pp. 84:1-84:11.
61. Spafford K, Vetter JS, Benson T, et al. (2013) Modeling synthetic aperture radar computation with Aspen. International Journal of High Performance Computing Applications 27 (3): 255-262.
62. Srinivasan S, Juve G, et al. (2014) A cleanup algorithm for implementing storage constraints in scientific workflow executions. In: 9th workshop on workflows in support of large-scale science, pp. 41-49. DOI:10.1109/WORKS. 2014.8.
63. Tierney B, Boote J, Boyd E, et al. (2009) Instantiating a Global Network Measurement Framework. Technical Report LBNL-1452E, Lawrence Berkeley National Lab.
64. Truong HL, Fahringer T, (2004) Scalea-g: A unified monitoring and performance analysis system for the grid. In: Dikaiakos M, (ed.) Grid Computing (Lecture Notes in Computer Science, vol. 3165). Berlin: Springer, pp. 202-211. DOI: 10.1007/978-3-540-28642-4-24.
65. Vahi K, Harvey I, Samak T, Gunter D, Evans K, Rogers D, Taylor I, Goode M, Silva F, Al-Shakarchi E, et al. (2013a) A case study into using common real-time workflow monitoring infrastructure for scientific workflows. Journal of Grid Computing 11 (3): 381-406.
66. Vahi K, Rynge M, et al. (2013b) Rethinking data management for big data scientific workflows. In: Workshop on BigData and science: infrastructure and services, pp. 1-9.
67. Valiant LG, (1990) A bridging model for parallel computation. Communications of the ACM 33 (8): 103-111.
68. Vöeckler JS, Mehta G, et al. (2006) Kickstarting remote applications. In: 2nd international workshop on grid computing environments, pp. 1-8.