Quantifying the potential benefit of overlapping communication and computation in large-scale scientific applications

Proceedings of the 2006 ACM/IEEE Conference on Supercomputing, SC'06

Volumn , Issue , 2006, Pages

  Sancho, José Carlos ^a   Barker, Kevin J ^a   Kerbyson, Darren J ^a   Davis, Kei ^a  

^a LOS ALAMOS NATIONAL LABORATORY   (United States)

Author keywords

Overlapping communication and computation; Performance modeling; Scientific applications

Indexed keywords

OVERLAPPING COMMUNICATION; OVERLAPPING COMPUTATION; PERFORMANCE MODELING; SCIENTIFIC APPLICATIONS;

BANDWIDTH; CODES (SYMBOLS); COMPUTATION THEORY; COST EFFECTIVENESS; LARGE SCALE SYSTEMS; RESOURCE ALLOCATION;

TELECOMMUNICATION NETWORKS;

EID: 34548214244     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1188455.1188585     Document Type: Conference Paper

References (26)

1. ALEXANDROV, A., IONESCU, M. F., SCHAUSER, K. E., AND SCHEIMAN, C. 1995. LogGP: Incorporating long messages into the LogP model - One step closer towards a realistic model for parallel computation. In SPAA: Proceedings of the Seventh Annual ACM Symposium on Parallel Algorithms and Architectures, ACM Press, Santa Barbara, California, pp. 95-105.
2. BARKER, K. J., AND KERBYSON, D. J. 2005. A performance model and scalability analysis of the HYCOM ocean simulation application. In IASTED PDCS, IASTED, Phoenix, Arizona.
3. BELL, C., BONACHEA, D., COTE, Y., DUELL, J., HARGROVE, P., HUSBANDS, P., IANCU, C., WELCOME, M., AND YELICK, K. 2003. An evaluation of current high performance networks. In International Parallel and Distributed Processing Symposium, IEEE Press, Nice, France, pp. 10.
4. BELL, C., BONACHEA, D., NISHTALA, R., AND YELICK, K. 2006. Optimizing bandwidth limited problems using one-sided communication and overlap. In International Parallel and Distributed Processing Symposium, ACM/IEEE Press, Rhodes Island, Greece, pp. 10.
5. CHEN, W., IANCU, C., AND YELICK, K. 2005. Communication optimizations for fine-grained UPC applications. In International Conference on Parallel Architectures and Compilation Techniques, ACM/IEEE Press, Saint Louis, Missouri, pp. 267-278.
6. COARFA, C., DOTSENKO, Y., ECKHARDT, J., AND MELLOR-CRUMMEY, J. 2003. Co-Array Fortran performance and potential: An NPB experimental study. In International Workshop on Languages and Compilers for Parallel Computing, Springer Berlin / Heidelberg, College Station, Texas, vol. 2958 of Lecture Notes in Computer Science, pp. 177-193.
7. DANALIS, A., KIM, K., POLLOCK, L., AND SWANY, M. 2005. Transformations to parallel codes for communication-computation overlap. In Supercomputing Conference, ACM/IEEE Press, Seattle, Washington, pp. 58.
8. DUNIGAN, T. H., VETTER, J. S., WHITE, J. B., AND WORLEY, P. H. 2005. Performance evaluation of the Cray X1 distributed shared-memory architecture. IEEE Micro vol. 25, no. 1, pp. 30-40.
9. HALLIWELL, G. R. 2004. Evaluation of vertical coordinate and vertical mixing algorithms in the hybrid coordinate ocean model (HYCOM). Ocean Modelling vol. 7, no. 3-4, pp. 285-322.
10. HIRANANDANI, S., KENNEDY, K., AND TSENG, C. 1992. Compiling Fortran D for MIMD distributed-memory machines. Communications of the ACM vol. 35, no. 8, pp. 66-80.
11. HOISIE, A., LUBECK, O., AND WASSERMAN, H. 2000. Performance and scalability analysis of teraflop-scale parallel architectures using multidimensional wavefront applications. International Journal of High Performance Applications vol. 14, no. 4, pp. 330-346.
12. HPCS. High Productivity Computing Systems initiative in DARPA. See http://www.darpa.mil/ipto/programs/hpcs/.
13. IANCU, C., HUSBANDS, P., AND HARGROVE, P. 2005. HUNTing the overlap. In International Conference on Parallel Architectures and Compilation Techniques, ACM/IEEE Press, Saint Louis, Missouri, pp. 279-290.
14. JONES, P. W., WORLEY, P. H., YOSHIDA, Y., J. B. WHITE III, AND LEVESQUE, J. 2005. Practical performance portability in the parallel ocean program (POP). Concurrency and Computation: Practice and Experience vol. 17, no. 10, pp. 1317-1327.
15. KE, J., BURTSCHER, M., AND SPEIGHT, E. 2005. Reducing communication time through message prefetching. In International Conference on Parallel and Distributed Processing Techniques and Applications, CSREA Press, Las Vegas, Nevada, pp. 557-563.
16. KE, J., BURTSCHER, M., AND SPEIGHT, E. 2005. Tolerating message latency through the early release of blocked receives. In Euro-Par, Springer Berlin / Heidelberg, Lisbon, Portugal, vol. 3648 of Lecture Notes in Computer Science, pp. 19-29.
17. KERBYSON, D. J., AND JONES, P. W. 2005. A performance model of the parallel ocean program. International Journal of High Performance Computing Applications vol. 35, no. 3, pp. 261-276.
18. KERBYSON, D. J., ALME, H. J., HOISIE, A., PETRINI, F., WASSERMAN, H. J., AND GITTINGS, M. 2001. Predictive performance and scalability modeling of a large-scale application. In Supercomputing Conference, ACM/IEEE Press, Denver, Colorado, pp. 39.
19. KING, C. T., CHOU, W. H., AND NI, L. M. 1990. Pipelined data parallel algorithms-I: Concept and modeling. IEEE Transactions on Parallel and Distributed Systems vol. 1, no. 4, pp. 486-499.
20. KOCH, K. R., BAKER, R. S., AND ALCOUFFE, R. E. 1992. Solution of the first-order form of the 3-D discrete ordinates equation on a massively parallel processor. Transactions of the American Nuclear Society vol. 65, pp. 198-192.
21. LAWRY, W., WILSON, C., MACCABE, A. B., AND BRIGHTWELL, R. 2002. COMB: A portable benchmark suite for assessing MPI overlap. In IEEE International Conference on Cluster Computing, IEEE Press, Chicago, Illinois, pp. 472.
22. LEU, J., AGRAWAL, D. P., AND MAUNEY, J. 1987. Modeling of parallel software for efficient computation-communication overlap. In Fall Joint Computer Conference, IEEE Press, Washington DC, pp. 569-575.
23. LIU, G., AND ABDELRAHMAN, T. S. 1998. Computation-communication overlap on network-of-workstation multiprocessors. In International Conference on Parallel and Distributed Processing Techniques and Applications, CSREA Press, Las Vegas, Nevada, pp. 1635-642.
24. MVAPICH/MVAPICH2. See http://nowlab.cse.ohio-state.edu/projects/mpi-iba/.
25. QUINN, M. J., AND HATCHER, P. J. 1996. On the utility of communication-computation overlap in dataparallel programs. Journal of Parallel and Distributed Computing vol. 33, no. 2, pp. 197-204.
26. SOHN, A., KU, J., KODAMA, Y., SATO, M., SAKANE, H., YAMANA, H., SAKAI, S., AND YAMAGUCHI, Y. 1996. Identifying the capability of overlapping computation with communication. In International Conference on Parallel Architectures and Compilation Techniques, ACM/IEEE Press, Boston, Massachusetts, pp. 133.