Compiler-based I/O prefetching for out-of-core applications

ACM Transactions on Computer Systems

Volumn 19, Issue 2, 2001, Pages 111-170

  Brown, Angela Demke ^a   Mowry, Todd C ^a   Krieger, Orran ^b  

^a Carnegie Mellon University   (United States)

^b IBM T J WATSON RESEARCH CENTER   (United States)

Author keywords

Compiler optimization; D.3.4 Programming Languages : Processors Compilers; D.4.2 Operating Systems : Storage Management Virtual memory; D.4.8 Operating Systems : Performance; Design; Experimentation; Optimization; Performance; Virtual memory

Indexed keywords

EID: 0012609370     PISSN: 07342071     EISSN: None     Source Type: Journal    
DOI: 10.1145/377769.377774     Document Type: Article

References (43)

1. ARUNACHALAM, M., CHOUDHARY, A., AND RULLMAN, B. 1995. A prefetching prototype for the parallel file system on the Paragon. In Proceedings of the 1995 ACM SIGMETRICS Joint International Conference on Measurement and Modeling of Computer Systems (SIGMETRICS '95/PERFORMANCE '95, Ottawa, Ontario, Canada, May 15-19), B. D. Gaither, Ed. ACM Press, New York, NY, 321-323. Extended abstract.
2. BAILEY, D., BARTON, J., LASINSKI, T., AND SIMON, H. 1991. The NAS parallel benchmarks. RNR-91-002.
3. BORDAWEKAR, R., CHOUDHARY, A., AND RAMANUJAM, J. 1996. Automatic optimization of communication in compiling out-of-core stencil codes. In Proceedings of the 1996 international conference on Supercomputing (ICS '96, Philadelphia, PA, May 25-28), P. C. Yew, Chair. ACM Press, New York, NY, 366-373.
4. BROWN, A. D. AND MOWRY, T. C. 2000. Taming the memory hogs: Using compiler-inserted releases to manage physical memory intelligently. In Proceedings of the 4th Symposium on Operating Systems Design and Implementation (San Diego, CA). 31-44.
5. CAO, P., FELTEN, E. W., KARLIN, A. R., AND LI, K. 1995. A study of integrated prefetching and caching strategies. In Proceedings of the 1995 ACM SIGMETRICS Joint International Conference on Measurement and Modeling of Computer Systems (SIGMETRICS '95/PERFORMANCE '95, Ottawa, Ontario, Canada, May 15-19), B. D. Gaither, Ed. ACM Press, New York, NY, 188-197.
6. CHANG, F. AND GIBSON, G. 1999. Automatic I/O hint generation through speculative execution. In Proceedings of the 3rd USENIX Symposium on Operating Systems Design and Implementation (OSDI '99, New Orleans, LA., Feb.). USENIX Assoc., Berkeley, CA.
7. CHEN, P. M., LEE, E. K., GIBSON, G. A., KATZ, R. H., AND PATTERSON, D. A. 1994. RAID: High-performance, reliable secondary storage. ACM Comput. Surv. 26, 2 (June), 145-185.
8. COLVIN, A. AND CORMEN, T. H. 1998. ViC*: A preprocessor for virtual-memory C*. In Proceedings of the Third International Workshop on High-Level Parallel Programming Models and Supportive Environments (HIPS'98, Orlando, FL, Mar.).
9. CRANDALL, P. E., AYDT, R. A., CHIEN, A. A., AND REED, D. A. 1995. Input/output characteristics of scalable parallel applications. In Proceedings of the 1995 Conference on Supercomputing (CD-ROM) (San Diego, CA, Dec. 3-8), S. Karin, Chair. ACM Press, New York, NY.
10. CUREWITZ, K., KRISHNAN, P., AND VITTER, J. 1993. Practical prefetching via data compression. In Proceedings of the 1993 ACM SIGMOD International Conference on Management of Data (SIGMOD '93, Washington, DC, May 26-28), P. Buneman and S. Jajodia, Eds. ACM Press, New York, NY, 43-53.
11. DEL ROSARIO, J. M. AND CHOUDHARY, A. N. 1994. High-performance I/O for massively parallel computers: Problems and prospects. IEEE Computer 27, 3 (Mar.), 59-68.
12. GANNON, D., JALBY, W., AND GALLIVAN, K. 1988. Strategies for cache and local memory management by global program transformation. J. Parallel Distrib. Comput. 5, 5 (Oct.), 587-616.
13. GRIFFIOEN, J. AND APPLETON, R. 1994. Reducing file system latency using a predictive approach. In Proceedings of the Winter Conference on USENIX (Jan.). USENIX Assoc., Berkeley, CA, 197-208.
14. GRIMSHAW, A. S. AND LOYOT, E. C. JR. 1991. ELFS: Object-oriented extensible file systems. In Proceedings of the First International Conference on Parallel and Distributed Information Systems (Miami Beach, FL, Dec.). 510-513.
15. HUBER, J. V., CHIEN, A. A., ELFORD, C. L., BLUMENTHAL, D. S., AND REED, D. A. 1995. PPFS: A high performance portable parallel file system. In Proceedings of the 9th ACM International Conference on Supercomputing (ICS '95, Barcelona, Spain, July 3-7), M. Valero, Chair. ACM Press, New York, NY, 385-394.
16. IEEE. 1992. Threads extension for portable operating systems (Draft 7).
17. KENNEDY, K., KOELBEL, C., AND PALECZNY, M. 1993. Scalable I/O for out-of-core structures. CRPC-TR93357-S. Center for Research on Parallel Computation, Rice University, Houston, TX.
18. KIMBREL, T., TOMKINS, A., PATTERSON, R., BERSHAD, B., CAO, P., FELTEN, E., GIBSON, G., KARLIN, A., AND LI, K. 1996. A trace-driven comparison of algorithms for parallel prefetching and caching. In Proceedings of the 2nd USENIX Symposium on Operating Systems Design and Implementation (Seattle, WA, Oct.). 19-34.
19. KOTZ, D. AND ELLIS, C. S. 1990. Prefetching in file systems for MIMD multiprocessors. IEEE Trans. Parallel Distrib. Syst. 1, 2 (Apr.), 218-230.
20. KOTZ, D. AND ELLIS, C. S. 1993. Practical prefetching techniques for multiprocessor file systems. Distrib. Parallel Databases 1, 1 (Jan.), 33-51.
21. KRIEGER, O. AND STUMM, M. 1997. HFS: A performance-oriented flexible file system based on building-block compositions. ACM Trans. Comput. Syst. 15, 3, 286-321.
22. KRIEGER, O., STUMM, M., AND UNRAU, R. 1992. Exploiting the advantages of mapped files for stream I/O. In Proceedings of the 1992 Winter USENIX Conference (San Francisco, CA, Jan.). USENIX Assoc., Berkeley, CA, 27-42.
23. KROEGER, T. M. AND LONG, D. D. E. 1996. Predicting file system actions from prior events. In Proceedings of the 1996 Technical Conference on USENIX (San Diego, CA, Jan.). USENIX Assoc., Berkeley, CA, 319-328.
24. LAM, M. S. 1988. Software pipelining: An effective scheduling technique for VLIW machines. In Proceedings of the ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI '88, Atlanta, GA, June 22-24), R. L. Wexelblat, Ed. ACM Press, New York, NY, 318-328.
25. LAUDON, J. AND LENOSKI, D. 1997. The SGI Origin2000: A ccNUMA highly scalable server. In Proceedings of the 24th International Symposium on Computer Architecture (ISCA '97, Denver, CO, June 2-4), A. R. Pleszkun and T. Mudge, Chairs. ACM Press, New York, NY, 241-251.
26. MALKAWI, M. AND PATEL, J. 1985. Compiler directed management policy for numerical programs. In Proceedings of the 10th ACM Symposium on Operating Systems Principles (Orcas Island, Washington, Dec.). 97-106.
27. MOWRY, T. C. 1994. Tolerating latency through software-controlled data prefetching. Ph.D. Dissertation. Stanford University, Stanford, CA.
28. MOWRY, T. C., LAM, M. S., AND GUPTA, A. 1992. Design and evaluation of a compiler algorithm for prefetching. In Proceedings of the 5th International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS-V, Boston, MA, Oct. 12-15), S. Eggers, Chair. ACM Press, New York, NY, 62-73.
29. PALECZNY, M., KENNEDY, K., AND KOELBEL, C. 1995. Compiler support for out-of-core arrays on data parallel machines. In Proceedings of the Fifth Symposium on Frontiers of Massively Parallel Computation (McLean, VA, Feb.). 110-118.
30. PARK, Y., SCOTT, R., AND SACHREST, S. 1996. Virtual memory versus file interfaces for large, memory-intensive scientific applications. In Proceedings of the Conference on Supercomputing (Pittsburgh, PA, Nov.). 17-22.
31. PATTERSON, R. H., GIBSON, G. A., GINTING, E., STODOLSKY, D., AND ZELENKA, J. 1995. Informed prefetching and caching. In Proceedings of the 15th ACM Symposium on Operating System Principles (SOSP, Copper Mountain Resort, Colorado, U.S., 3-6 Dec.). ACM Press, New York, NY, 79-95.
32. POOLE, J. T. 1994. Preliminary survey of I/O intensive applications. CCSF-38.
33. SINGH, T. AND CHOUDHARY, A. 1994. ADOPT: A dynamic scheme for optimal prefetching in parallel file systems.
34. SONG, I. AND CHO, Y. 1993. Page prefetching based on fault history. In Proceedings of the Third Mach Symposium on USENIX (Santa Fe, NM, Apr.). 203-213.
35. SWEENEY, A., DOUCETTE, D., HU, W., ANDERSON, C., NISHIMOTO, M., AND PECK, G. 1996. Scalability in the XFS file system. In Proceedings of the 1996 Technical Conference on USENIX (San Diego, CA, Jan.). USENIX Assoc., Berkeley, CA, 1-14.
36. THAKUR, R., BORDAWEKAR, R., AND CHOUDHARY, A. 1994. Compilation of out-of-core data parallel programs for distributed memory machines. In Proceedings of IPPS '94 Workshop on Input/Output in Parallel Computer Systems (IPPS '94, Cancun, Mexico, Apr.). Syracuse University, Syracuse, NY, 54-72.
37. THAKUR, R., BORDAWEKAR, R., CHOUDHARY, A., PONNUSAMY, R., AND SINGH, T. 1993. PASSION runtime library for parallel I/O. In Proceedings of the Conference on Scalable Parallel Libraries (Mississippi State University, Oct.), A. Skjellum, Ed. IEEE Computer Society, Washington, DC, 119-128.
38. TJIANG, S. W. K. AND HENNESSY, J. L. 1992. Sharlit: A tool for building optimizers. In Proceedings of the 5th ACM SIGPLAN Conference on Programming Language Design and Implementation (SIGPLAN '92, San Francisco, CA, June 17-19), R. L. Wexelblat, Ed. ACM Press, New York, NY.
39. TRIVEDI, K. 1977. On the paging performance of array algorithms. IEEE Trans. Comput. C-26, 10 (Oct.), 938-947.
40. UNRAU, R. C., KRIEGER, O., GAMSA, B., AND STUMM, M. 1995. Hierarchical clustering: A structure for scalable multiprocessor operating system design. J. Supercomput. 9, 1/2 (), 105-134.
41. VRANESIC, Z. G., STUMM, M., LEWIS, D. M., AND WHITE, R. 1991. Hector: A hierarchically structured shared-memory multiprocessor. IEEE Computer 24, 1 (Jan.), 72-79.
42. WOLF, M. E. AND LAM, M. S. 1991. A data locality optimization algorithm. In Proceedings of the ACM Conference on Programming Language Design and Implementation (SIGPLAN '91, Toronto, Ontario, Canada, June 26-28), D. S. Wise, Chair. ACM Press, New York, NY, 30-44.
43. WOMBLE, D., GREENBERG, D., RIESEN, R., AND WHEAT, S. 1993. Out of core, out of mind: Practical parallel I/O. In Proceedings of the Conference on Scalable Parallel Libraries (Mississippi State University, Oct.), A. Skjellum, Ed. IEEE Computer Society, Washington, DC, 10-16.