Machine learning based online performance prediction for runtime parallelization and task scheduling

ISPASS 2009 - International Symposium on Performance Analysis of Systems and Software

Volumn , Issue , 2009, Pages 89-100

  Li, Jiangtian ^a   Ma, Xiaosong ^a,b   Singh, Karan ^c   Schulz, Martin ^d   De Supinski, Bronis ^d   McKee, Sally A ^e  

^a North Carolina State University   (United States)

^b OAK RIDGE NATIONAL LABORATORY   (United States)

^c Cornell University Ithaca ^*  (United States)

^d LAWRENCE LIVERMORE NATIONAL LABORATORY   (United States)

^e CHALMERS UNIVERSITY OF TECHNOLOGY   (Sweden)

Author keywords

Artificial Neural Networks; Automatic Task Scheduling; Performance Prediction; Scripting Languages

Indexed keywords

APRIORI; ARTIFICIAL NEURAL NETWORKS; AUTOMATIC TASK SCHEDULING; COST ESTIMATE; DEPENDENT TASKS; EXECUTION TIME; FUNDAMENTAL PROBLEM; LOAD IMBALANCE; LOAD-BALANCING; MACHINE LEARNING TECHNIQUES; MACHINE-LEARNING; MANY-CORE; ON-LINE PERFORMANCE; PARALLELIZATIONS; PERFORMANCE MODEL; PERFORMANCE PREDICTION; PROGRAM EXECUTION; REAL-WORLD APPLICATION; RUNTIME SYSTEMS; RUNTIMES; SCIENTIFIC APPLICATIONS; SCRIPTING LANGUAGES; SEQUENTIAL APPLICATIONS; SYNTHETIC TESTS; TASK PARTITIONING; TASK-SCHEDULING; TEST SCRIPTS;

BACKPROPAGATION; COMPUTER SOFTWARE; COST ACCOUNTING; LEARNING ALGORITHMS; LINGUISTICS; MULTITASKING; NEURAL NETWORKS; PARALLEL PROGRAMMING; QUERY LANGUAGES; ROBOT LEARNING;

SCHEDULING ALGORITHMS;

EID: 70349185990     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ISPASS.2009.4919641     Document Type: Conference Paper

References (46)

1. J. Gummaraju, J. Coburn, Y. Turner, and M. Rosenblum, "Streamware: Programming General-purpose Multicore Processors Using Streams," in Architectural Support for Programming Languages and Operating Systems (ASPLOS XIII), 2008.
2. M. D. Linderman, J. D. Collins, H. Wang, and T. H. Meng, "Merge: AProgramming Model for Heterogeneous Multi-core Systems," in Architectural Support for Programming Languages and Operating Systems (ASPLOS XIII), 2008.
3. D. August, "Automatic Parallelization is Key for Manycore Success," in Manycore Computing Workshop, 2007.
4. W. Hwu, S. Ryoo, S. Ueng, J. H. Kelm, I. Gelado, S. S. Stone, R. E. Kidd, S. S. Baghsorkhi, A. A. Mahesri, S. C. Tsao, N. Navarro, S. S. Lumetta, M. I. Frank, and S. J. Patel, "Implicitly Parallel Programming Models for Thousand-core Microprocessors," in Design and Automation Conference (DAC '07), 2007.
5. F. Putze, P. Sanders, and J. Singler, "MCSTL: The Multi-core Standard Template Library," in Principles and Practice of Parallel Programming (PPoPP), 2007.
6. S. Rus, M. Pennings, and L. Rauchwerger, "Sensitivity Analysis for Automatic Parallelization on Multi-cores," in ICS '07: Proceedings of the 21st annual international conference on Supercomputing, 2007.
7. R. Choy and A. Edelman, "Parallel MATLAB: Doing It Right," Proceedings of the IEEE, vol. 93, no. 2, 2005.
8. X. Ma, J. Li, and N. F. Samatova, "Automatic Parallelization of Scripting Languages: Toward Transparent Desktop Parallel Computing," in Proceedings of the IEEE International Parallel and Distributed Processing Symposium (IPDPS), 2007.
9. R Development Core Team, R: A Language and Environment for Statistical Computing, R Foundation for Statistical Computing, Vienna, Austria, 2005, ISBN 3-900051-07-0. [Online]. Available: http://www.Rproject.org
10. Y. Kwok and I. Ahmad, "Static Scheduling Algorithms for Allocating Directed Task Graphs to Multiprocessors," ACM Comput. Surv., 1999.
11. M. Y. Wu and D. D. Gajski, "Hypertool: A Programming Aid for Message-Passing Systems," IEEE Trans. Parallel Distrib. Syst., vol. 1, no. 3, pp. 330-343, 1990.
12. Bioconductor Core, An Overview of Projects in Computing for Genomic Analysis, 2002. [Online]. Available: http://www.bioconductor.org/
13. T. Mitchell, Machine Learning. Boston, MA:WCB/McGraw Hill, 1997.
14. A. Zell and et. al., SNNS: Stuttgart Neural Network Simulator., University of Stuttgart, User Manual, Version 4.2.
15. D. Culler, R. Karp, D. Patterson, A. Sahay, K. E. Schauser, E. Santos, R. Subramonian, and T. von Eicken, "LogP: Towards a Realistic Model of Parallel Computation," SIGPLAN Not., 1993.
16. Y. Kwok and I. Ahmad, "Benchmarking and Comparison of the Task Graph Scheduling Algorithms," J. Parallel Distrib. Comput., 1999.
17. M. Y. Wu, W. Shu, and Y. Chen, "Runtime Parallel Incremental Scheduling of DAGs," in ICPP '00: Proceedings of the Proceedings of the 2000 International Conference on Parallel Processing, 2000.
18. D. R. Barr and T. Davidson, "A Kolmogorov-Smirnov Test for Censored Samples," Technometrics, 1973.
19. W. J. Conover and R. L. Iman, "Rank Transformations as a Bridge Between Parametric and Nonparametric Statistics," The American Statistician, 1981.
20. K. J. Levy, "Pairwise Comparisons Associated with the K Independent Sample Median Test," The American Statistician, 1979.
21. M. Schulz, J. Galarowicz, D. Maghrak, W. Hachfeld, D. Monotya, and S. Cranford, "Open SpeedShop: An Open Source Infrastructure for Parallel Performance Analysis," Scientific Programming, Special Issue on Large-Scale Programming Tools and Environments, vol. 16, no. 2,3, pp. 105-121, 2008.
22. R. Bell, A. Malony, and S. Shende, "ParaProf: A Portable, Extensible, and Scalable Tool for Parallel Performance Profile Analysis," in Proceedings of the International Conference on Parallel and Distributed Computing (Euro-Par 2003), Aug. 2003, pp. 17-26.
23. V. Pillet, J. Labarta, T. Cortes, and S. Girona, "PARAVER: A Tool to Visualise and Analyze Parallel Code," in Proceedings of WoTUG- 18: Transputer and Occam Developments, ser. Transputer and Occam Engineering, vol. 44, Apr. 1995, pp. 17-31.
24. L. DeRose and D. A. Reed, "SvPablo: A Multi-Language Architecture- Independent Performance Analysis System," in Proceedings of the International Conference on Parallel Processing (ICPP'99), Sep. 1999.
25. M. Müller, H. Brunst, M. Jurenz, A. Knüpfer, M. Lieber, H. Mix, and W. Nagel, "Developing Scalable Applications with Vampir, VampirServer and VampirTrace," in Proceedings of the Minisymposium on Scalability and Usability of HPC Programming Tools at PARCO 2007, Sep. 2007.
26. K. A. Huck, A. D. Malony, and A. Morris, "Design and Implementation of a Parallel Performance Data Management Framework," in International Conference on Parallel Processing (ICPP '05), 2005.
27. K. L. Karavanic, J. May, K. Mohror, B. Miller, K. A. Huck, R. Knapp, and B. Pugh, "Integrating Database Technology with Comparison-based Parallel Performance Diagnosis: The PerfTrack Performance Experiment Management Tool," in Supercomputing '05: Proceedings of the 2005 ACM/IEEE conference on Supercomputing (CDROM), 2005.
28. X. Wu, V. E. Taylor, J. Geisler, X. Li, Z. Lan, R. Stevens, M. Hereld, and I. R. Judson, "Design and Development of Prophesy Performance Database for Distributed Scientific Applications," in Proc. the 10th SIAM Conference on Parallel Processing for Scientific Computing, 2001.
29. S. Yu, M. Winslett, J. Lee, and X. Ma, "Automatic and Portable Performance Modeling for Parallel I/O: A Machine-Learning Approach," ACM SIGMETRICS Performance Evaluation Review, 2002.
30. E. Ipek, B. R. de Supinski, M. Schulz, and S. A. McKee, "An Approach to Performance Prediction for Parallel Applications," in Proceedings of the International Conference on Parallel and Distributed Computing (Euro-Par 2005), Aug 2005, pp. 196-205.
31. B. C. Lee, D. M. Brooks, B. R. de Supinski, M. Schulz, K. Singh, and S. A. McKee, "Methods of Inference and Learning for Performance Modeling of Parallel Applications," in Principles and Practices of Parallel Programming (PPOPP), 2007.
32. E. Ipek, S. A. McKee, R. Caruana, B. R. de Supinski, and M. Schulz, "Efficiently Exploring Architectural Design Spaces via Predictive Modeling," in Architectural Support for Programming Languages and Operating Systems (ASPLOS-XII), 2006.
33. B. Lee and J. M. Schopf, "Run-Time Prediction of Parallel Applications on Shared Environments," in CLUSTER, 2003.
34. L. Carrington, N. Wolter, A. Snavely, and C. B. Lee, "Applying an Automated Framework to Produce Accurate Blind Performance Predictions of Full-scale HPC Applications," in Proceedings of the 2004 Department of Defense Users Group Conference, 2004.
35. J. Yang, A. Khokhar, S. Sheikh, and A. Ghafoor, "Estimating Execution Time for Parallel Tasks in Heterogeneous Processing (HP) Environment," in Proceedings of the Heterogeneous Computing Workshop, 1994.
36. M. Curtis-Maury, K. Singh, S. A. McKee, F. Blagojevic, D. S. Nikolopoulos, B. R. de Supinski, and M. Schulz, "Identifying Energy-Efficient Concurrency Levels Using Machine Learning," in Proc. of the International Workshop on Green Computing, Sep. 2007.
37. J. K. Lenstra and A. H. G. R. Kan, "Complexity of Scheduling under Precedence Constraints," Operation Research, 1978.
38. G. C. Sih and E. A. Lee, "A Compile-Time Scheduling Heuristic for Interconnection-Constrained Heterogeneous Processor Architectures," IEEE Trans. Parallel Distrib. Syst., vol. 4, no. 2, pp. 175-187, 1993.
39. S. Ramaswamy, S. S. Sapatnekar, and P. Banerjee, "A Framework for Exploiting Task and Data Parallelism on Distributed Memory Multicomputers," IEEE Trans. Parallel Distrib. Syst., 1997.
40. S. Bansal, P. Kumar, and K. Singh, "An Improved Two-step Algorithm for Task and Data Parallel Scheduling in Distributed Memory Machines," Parallel Computing, 2006.
41. A. Mandal, K. Kennedy, C. Koelbel, G. Marin, J. Mellor-Crummey, B. Liu, and L. Johnsson, "Scheduling Strategies for Mapping Application Workflows onto the Grid," in IEEE Symposium on High Performance Distributed Computing (HPDC 2005), 2005.
42. K. P. Belkhale and P. Banerjee, "An Approximate Algorithm for the Partitionable Independent Task Scheduling Problem," in International Conference on Parallel Processing(ICPP), Vol. 1, 1990.
43. J. Turek, J. L. Wolf, and P. S. Yu, "Approximate Algorithms Scheduling Parallelizable Tasks," in SPAA '92: Proceedings of the Fourth Annual ACM Symposium on Parallel Algorithms and Architectures, 1992.
44. S. Chakrabarti, J. Demmel, and K. A. Yelick, "Models and Scheduling Algorithms for Mixed Data and Task Parallel Programs," J. Parallel Distrib. Comput., 1997.
45. J. Wildstrom, P. Stone, E. Witchel, R. J. Mooney, and M. Dahlin, "Towards Self-Configuring Hardware for Distributed Computer Systems," in ICAC '05: Proceedings of the Second International Conference on Automatic Computing, 2005.
46. F. Gomez, D. Burger, and R. Miikkulainen, "A Neuroevolution Method for Dynamic Resource Allocation on a Chip Multiprocessor," in Proceedings of the International Joint Conference on Neural Networks (IJCNN-01), 2001.