Enabling event tracing at leadership-class scale through I/O forwarding middleware

HPDC '12 - Proceedings of the 21st ACM Symposium on High-Performance Parallel and Distributed Computing

Volumn , Issue , 2012, Pages 49-60

  Ilsche, Thomas ^a,c   Schuchart, Joseph ^a   Cope, Jason ^b   Kimpe, Dries ^b   Jones, Terry ^c   Knüpfer, Andreas ^a   Iskra, Kamil ^b   Ross, Robert ^b   Nagel, Wolfgang E ^a   Poole, Stephen ^c  

^a DRESDEN UNIVERSITY OF TECHNOLOGY   (Germany)

^b ARGONNE NATIONAL LABORATORY   (United States)

^c OAK RIDGE NATIONAL LABORATORY   (United States)

Author keywords

Atomic append; Event tracing; I O forwarding

Indexed keywords

ARTIFICIAL PERTURBATIONS; ATOMIC APPEND; BUFFERING CAPABILITIES; COMPUTING SYSTEM; EVENT TRACING; FILE SYSTEMS; LOG DATA; PARALLEL APPLICATION; PARALLEL FILE SYSTEM; STORAGE SYSTEMS; SYSTEM SOFTWARES; TOOLSETS;

FILE ORGANIZATION;

MIDDLEWARE;

EID: 84863906363     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2287076.2287085     Document Type: Conference Paper

References (32)

1. Abbasi, H., Wolf, M., Eisenhauer, G., Klasky, S., Schwan, K., and Zheng, F. DataStager: Scalable data staging services for petascale applications. In Proceedings of the 18th ACM International Symposium on High Performance Distributed Computing (HPDC) (2009), pp. 39-48.
2. Ali, N., Carns, P., Iskra, K., Kimpe, D., Lang, S., Latham, R., Ross, R., Ward, L., and Sadayappan, P. Scalable I/O forwarding framework for high-performance computing systems. In Proceedings of the 11th IEEE International Conference on Cluster Computing (CLUSTER) (2009).
3. Bent, J., Gibson, G., Grider, G., McClelland, B., Nowoczynski, P., Nunez, J., Polte, M., and Wingate, M. PLFS: A checkpoint filesystem for parallel applications. In Proceedings of 21st ACM/IEEE International Conference for High Performance Computing, Networking, Storage and Analysis (SC) (2009).
4. Bland, A., Kendall, R., Kothe, D., Rogers, J., and Shipman, G. Jaguar: The world's most powerful computer. In Proceedings of the 51st Cray User Group Meeting (CUG) (2009).
5. Carns, P., Ligon III, W., Ross, R., and Wyckoff, P. BMI: A network abstraction layer for parallel I/O. In Proceedings of the 19th IEEE International Parallel and Distributed Processing Symposium, Workshop on Communication Architecture for Clusters (CAC) (2005).
6. Chen, J. H., Choudhary, A., de Supinski, B., DeVries, M., Hawkes, E. R., Klasky, S., Liao, W. K., Ma, K. L., Mellor-Crummey, J., Podhorszki, N., Sankaran, R., Shende, S., and Yoo, C. S. Terascale direct numerical simulations of turbulent combustion using S3D. Computational Science & Discovery 2, 1 (2009), 015001.
7. Ching, A., Choudhary, A., Coloma, K., Liao, W., Ross, R., and Gropp, W. Noncontiguous I/O access through MPI-IO. In Proceedings of the 3rd IEEE/ACM International Symposium on Cluster Computing and the Grid (CCGrid) (2003), pp. 104-111.
8. Docan, C., Parashar, M., and Klasky, S. DART: A substrate for high speed asynchronous data IO. In Proceedings of the 17th International Symposium on High Performance Distributed Computing (HPDC) (2008).
9. Frings, W., Wolf, F., and Petkov, V. Scalable massively parallel I/O to task-local files. In Proceedings of 21st ACM/IEEE International Conference for High Performance Computing, Networking, Storage and Analysis (SC) (2009).
10. Ghemawat, S., Gobioff, H., and Leung, S. The Google File System. SIGOPS Operating Systems Review 37 (Oct. 2003), 29-43.
11. Gygi, F., Duchemin, I., Donadio, D., and Galli, G. Practical algorithms to facilitate large-scale first-principles molecular dynamics. Journal of Physics: Conference Series 180, 1 (2009).
12. Hildebrand, D., and Honeyman, P. Exporting storage systems in a scalable manner with pNFS. In Proceedings of the 22nd IEEE / 13th NASA Goddard Conference on Mass Storage Systems and Technologies (MSST) (2005), pp. 18-27.
13. IEEE POSIX Standard 1003.1 2004 Edition. http://www.opengroup.org/ onlinepubs/000095399/functions/write.html.
14. Iskra, K., Romein, J. W., Yoshii, K., and Beckman, P. ZOID: I/O-forwarding infrastructure for petascale architectures. In Proceedings of the 13th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming (PPoPP) (2008), pp. 153-162.
15. Jagode, H., Dongarra, J., Alam, S., Vetter, J., Spear, W., and Malony, A. D. A holistic approach for performance measurement and analysis for petascale applications. In Proceedings of the 9th International Conference on Computational Science (ICCS) (2009), vol. 2, pp. 686-695.
16. Jones, T., Dawson, S., Neely, R., Tuel, W., Brenner, L., Fier, J., Blackmore, R., Caffrey, P., and Maskell, B. Improving the scalability of parallel jobs by adding parallel awareness. In Proceedings of the 15th ACM/IEEE International Conference on High Performance Networking and Computing (SC) (2003).
17. Knüpfer, A., Brunst, H., Doleschal, J., Jurenz, M., Lieber, M., Mickler, H., Müller, M. S., and Nagel, W. E. The Vampir performance analysis tool-set. In Tools for High Performance Computing (2008), M. Resch, R. Keller, V. Himmler, B. Krammer, and A. Schulz, Eds., Springer Verlag, pp. 139-155.
18. Lofstead, J. F., Klasky, S., Schwan, K., Podhorszki, N., and Jin, C. Flexible IO and integration for scientific codes through the adaptable IO system (ADIOS). In Proceedings of the 6th International Workshop on Challenges of Large Applications in Distributed Environments (CLADE) (2008), pp. 15-24.
19. MPI Forum. MPI-2: Extensions to the Message-Passing Interface. http://www.mpi-forum.org/docs/docs.html, 1997.
20. Muelder, C., Gygi, F., and Ma, K.-L. Visual analysis of inter-process communication for large-scale parallel computing. IEEE Transactions on Visualization and Computer Graphics 15, 6 (2009), 1129-1136.
21. Muelder, C., Sigovan, C., Ma, K.-L., Cope, J., Lang, S., Iskra, K., Beckman, P., and Ross, R. Visual analysis of I/O system behavior for high-end computing. In Proceedings of the 3rd International Workshop on Large-Scale System and Application Performance (LSAP) (2011).
22. Nisar, A., Liao, W., and Choudhary, A. Scaling parallel I/O performance through I/O delegate and caching system. In Proceedings of 20th ACM/IEEE International Conference for High Performance Computing, Networking, Storage and Analysis (SC) (2008).
23. Ohta, K., Kimpe, D., Cope, J., Iskra, K., Ross, R., and Ishikawa, Y. Optimization techniques at the I/O forwarding layer. In Proceedings of the 12th IEEE International Conference on Cluster Computing (CLUSTER) (2010).
24. Pedretti, K., Brightwell, R., and Williams, J. Cplant™ runtime system support for multi-processor and heterogeneous compute nodes. In Proceedings of the 4th IEEE International Conference on Cluster Computing (CLUSTER) (2002), pp. 207-214.
25. Petascale Data Storage Institute. http://www.pdsi-scidac.org/.
26. Peterka, T., Goodell, D., Ross, R., Shen, H.-W., and Thakur, R. A configurable algorithm for parallel image-compositing applications. In Proceedings of 21st ACM/IEEE International Conference for High Performance Computing, Networking, Storage and Analysis (SC) (2009).
27. Romein, J. Fcnp: Fast I/O on the Blue Gene/P. In Parallel and Distributed Processing Techniques and Applications (PDPTA'09) (2009).
28. Shipman, G., Dillow, D., Oral, S., and Wang, F. The Spider center wide file system; from concept to reality. In Proceedings of the 51st Cray User Group Meeting (CUG) (2009).
29. Vishwanath, V., Hereld, M., Iskra, K., Kimpe, D., Morozov, V., Papka, M., Ross, R., and Yoshii, K. Accelerating I/O forwarding in IBM Blue Gene/P systems. In Proceedings of 22nd ACM/IEEE International Conference for High Performance Computing, Networking, Storage and Analysis (SC) (2010).
30. Wylie, B. J. N., Geimer, M., Mohr, B., Böhme, D., Szebenyi, Z., and Wolf, F. Large-scale performance analysis of Sweep3D with the Scalasca toolset. Parallel Processing Letters 20, 4 (2010), 397-414.
31. Yoshii, K., Iskra, K., Naik, H., Beckman, P., and Broekema, P. C. Performance and scalability evaluation of 'Big Memory' on Blue Gene Linux. International Journal of High Performance Computing Applications 25, 2 (2011), 148-160.
32. Yu, H., Sahoo, R. K., Howson, C., Almási, G., Castaños, J. G., Gupta, M., Moreira, J. E., Parker, J. J., Engelsiepen, T. E., Ross, R. B., Thakur, R., Latham, R., and Gropp, W. D. High performance file I/O for the Blue Gene/L supercomputer. In Proceedings of the 12th International Symposium on High-Performance Computer Architecture (HPCA) (2006), pp. 187-196.