Long term distributed file reference tracing: Implementation and experience

Software - Practice and Experience

Volumn 26, Issue 6, 1996, Pages 705-736

  Mummert, L ^a   Satyanarayanan, M ^a  

^a Carnegie Mellon University   (United States)

Author keywords

Distributed file systems; Evaluation; File reference tracing; Measurement

Indexed keywords

COMPUTER AIDED SOFTWARE ENGINEERING; COMPUTER WORKSTATIONS; DATA ACQUISITION; DISTRIBUTED COMPUTER SYSTEMS; FILE ORGANIZATION; INFORMATION RETRIEVAL SYSTEMS; ONLINE SYSTEMS; PROGRAM DEBUGGING; UNIX; USER INTERFACES;

COLLECTION SERVERS; DATA EXTRACTION AGENT; FILE REFERENCE TRACING; KERNEL BUFFER; KERNEL HOOKS;

DISTRIBUTED DATABASE SYSTEMS;

EID: 0030169103     PISSN: 00380644     EISSN: None     Source Type: Journal    
DOI: 10.1002/(sici)1097-024x(199606)26:6<705::aid-spe29>3.0.co;2-o     Document Type: Article

References (71)

1. Mary G. Baker, John H. Hartmann, Michael D. Kupfer, Ken W. Shirriff and John K. Ousterhout, 'Measurement of a distributed file system', Proceedings of the Thirteenth ACM Symposium on Operating Systems Principles, October 1991, pp. 198-212.
2. John K. Ousterhout, Hervé Da Costa, David Harrison, John A. Kunze, Mike Knupfer and James G. Thompson, 'A trace-driven analysis of the UNIX 4.2 BSD file system', Proceedings of the Tenth ACM Symposium on Operating Systems Principles, December 1985.
3. Juan M. Porcar, 'File migration in distributed computer systems', Ph.D. dissertation, University of California, Berkeley, July 1982.
4. Edwin P. Stritter, 'File migration', Ph.D. dissertation, Stanford University, March 1977.
5. Alan Jay Smith, 'Analysis of long term file reference patterns for application to file migration algorithms', IEEE Transactions on Software Engineering, 7, (4), 403-417 (1981).
6. Alan Jay Smith, 'Long term file migration: development and evaluation of algorithms', Communications of the ACM, 24, (8), 521-532 (1981).
7. D.H. Lawrie, J.M. Randal and R.R. Barton, 'Experiments with automatic file migration', IEEE Computer, 15, (7), 45-55 (1982).
8. M. Satyanarayanan, James J. Kistler, Puneet Kumar, Maria E. Okasaki, Ellen H. Siegel and David C. Steere, 'Coda: a highly available file system for a distributed workstation environment', IEEE Transactions on Computers, 39, (4), 447-459 (1990).
9. M. Satyanarayanan, 'Scalable, secure, and highly available distributed file access', IEEE Computer, 23, (5), 9-21 (1990).
10. Rick Floyd, 'Short-term file reference patterns in a UNIX environment', Technical Report TR 177, Department of Computer Science, University of Rochester, March 1986.
11. Shikharesh Majumdar and Richard B. Bunt, 'Measurement and analysis of locality phases in file referencing behaviour', Proceedings of Performance '86 and ACM SIGMETRICS 1986 Joint Conference on Computer Performance Modelling, Measurement and Evaluation, May 1986.
12. M. Satyanarayanan, John H. Howard, David A. Nichols, Robert N. Sidebotham, Alfred Z. Spector and Michael J. West, 'The ITC distributed file system: principles and design', Proceedings of the Tenth ACM Symposium on Operating Systems Principles, December 1-4, 1985, pp. 35-50.
13. Russel Sandberg, David Goldberg, Steve Kleiman, Dan Walsh and Bob Lyon, 'Design and implementation of the Sun network file system', USENIX Summer Conference Proceedings, USENIX Association, June 1985.
14. Marshall K. McKusick, William N. Joy, Samuel J. Leffler and Robert S. Fabry, 'A fast file system for Unix', ACM Transactions on Computer Systems, 2, (3), 181-197 (1984).
15. Gene McDaniel, 'METRIC: a kernel instrumentation system for distributed environments', Proceedings of the Sixth ACM Symposium on Operating Systems Principles, November 1977, pp. 93-99.
16. Department of Electrical Engineering Computer Systems Research Group, Computer Science Division and Berkeley Computer Science, University of Californa, Unix Programmer's Manual Reference Guide, USENIX Association, April 1986.
17. Mike Accetta, Robert V. Baron, William Bolosky, David B. Golub, Richard F. Rashid, Avadis Tevanian, Jr and Michael Wayne Young, 'Mach: a new kernel foundation for UNIX development', Proceedings of the Summer 1986 USENIX Conference, Atlanta, GA, July 1986, pp. 93-113.
18. S. R. Kleiman, 'Vnodes: an architecture for multiple file system types in Sun UNIX', USENIX Summer Conference Proceedings. USENIX Association, 1986.
19. Richard A. Floyd and Carla Schlauer Ellis, 'Directory reference patterns in hierarchical file systems', IEEE Transactions on Knowledge and Data Engineering, 1, (2), 238-247 (1989).
20. Samual J. Leffler, Marshall Kirk McKusick, Michael J. Karels and John S. Quarterman, The Design and Implementation of the 4.3BSD UNIX Operating System, Addison-Wesley, Reading, Massachusetts, 1989.
21. M. Satyanarayanan (ed.), RPC2 User Guide and Reference Manual, School of Computer Science, Carnegie Mellon University, Pittsburgh PA 15213, October 1991.
22. John H. Howard, Michael L. Kazar, Sherri G. Menees, David A. Nichols, M. Satyanarayanan, Robert N. Sidebotham and Michael J. West, 'Scale and performance in a distributed file system', ACM Transactions on Computer Systems, 6, (1), 51-81 (1988).
23. Puneet Kumar and M. Satyanarayanan, 'Log-based directory resolution in the Coda file system', Proceedings of the Second International Conference on Parallel and Distributed Information Systems, January 1993, pp. 202-213. Also available as Technical Report CMU-CS-91-164, School of Computer Science, Carnegie Mellon University.
24. Puneet Kumar and M. Satyanarayanan, 'Log-based directory resolution in the Coda file system', Proceedings of the Second International Conference on Parallel and Distributed Information Systems, January 1993, pp. 202-213. Also available as Technical Report CMU-CS-91-164, School of Computer Science, Carnegie Mellon University.
25. Michael Blair Jones, 'Transparently interposing user code at the system interface', Ph.D. dissertation, Carnegie Mellon University, September 1992.
26. Michael B. Jones, 'Interposition agents: transparently interposing user code at the system interface', Proceedings of 14th ACM Symposium on Operating Systems Principles. Association for Computing Machinery SIGOPS, December 1993, pp. 80-93.
27. IBM, 'OS/VS2 system programming library: service aids', Technical Report GC28-0674-1, IBM Corporation, (1976).
28. S. W. Sherman and J. C. Browne, 'Trace driven modeling: review and overview', Proceedings of the ACM-SIGSIM Symposium on the Simulation of Computer Systems, June 1973, pp. 201-207.
29. James J. Kistler and M. Satyanarayanan, 'Disconnected operation in the Coda file system', ACM Transactions on Computer Systems, 10, (1), 3-25 (1992).
30. James J. Kistler, 'Disconnected operation in a distributed file system', Ph.D. dissertation, School of Computer Science, Carnegie Mellon University, April 1993.
31. Brian D. Noble and M. Satyanarayanan, 'An empirical study of a highly available file system', Proceedings of the 1994 ACM SIGMETRICS Conference on Measurement and Modeling of Computer Systems, Nashville, TN, May 1994, pp. 138-149.
32. R. Hugo Patterson, Garth A. Gibson and M. Satyanarayanan, 'A status report on research in transparent informed prefetching', Operating Systems Review, 27, (2), 21-34 (1993).
33. R. Hugo Patterson and Garth A. Gibson, 'Exposing I/O concurrency with informed prefetching', Proceedings of the Third International Conference on Parallel and Distributed Information Systems, September 1994.
34. L. Mummert, M. Ebling and M. Satyanarayanan, 'Exploiting weak connectivity for mobile file access', Proceedings of the Fifteenth ACM Symposium on Operating Systems Principles, December 1995.
35. M. Satyanarayanan, Henry H. Mashburn, Puneet Kumar, David C. Steere and James J. Kistler, 'Lightweight recoverable virtual memory', ACM Transactions on Computer Systems, 12, (1), 33-57 (1994). Corrigendum: 12, (2), 165-172 (1994).
36. D.L. Mills, 'Internet time synchronization: the network time protocol', IEEE Transactions on Communications, 39, (10), 1482-93 (1991).
37. Maria R. Ebling and M. Satyanarayanan, 'SynRGen: an extensible file reference generator', Proceedings of the 1994 ACM SIGMETRICS Conference on Measurement and Modeling of Computer Systems, Nashville, TN, May 1994, pp. 108-117.
38. Steven Tsao, 'Disk geometry and performance characteristic', Data Storage Systems Center, Research for Undergraduate Students Program, July 1992.
39. Mark Holland and Rachad Youssef, Personal communication, October 1994.
40. Raj Jain, The Art of Computer Systems Performance Analysis: Techniques for Experimental Design, Measurement, Simulation, and Modeling, John Wiley & Sons, Inc, New York, NY, 1991.
41. R. Hugo Patterson, Personal communication, November 1993.
42. Gordon Irlam, 'A static analysis of Unix file systems circa 1993', ftp ://cs.dartmouth.edu/pub/file-sizes/ufs93b.tar.gz, (October 1993).
43. Steven Strange, 'Analysis of long-term Unix file access patterns for application to automatic file migration strategies', Technical Report UCB/CSD 92/700, University of California, Berkeley, Computer Science Division, (August 1992).
44. M. Satyanarayanan, 'A study of file sizes and functional lifetimes', Proceedings of the Eighth ACM Symposium on Operating Systems Principles, December 1981, pp. 96-108.
45. Ron Revelle, 'An empirical study of file reference patterns', Technical Report RJ 1557, IBM, April 1975.
46. Jim Griffioen and Randy Appleton, 'Reducing file system latency using a predictive approach', USENIX Summer Conference Proceedings. USENIX Association, June 1994, pp. 197-207.
47. Ethan L. Miller and Randy H. Katz, 'An analysis of file migration in a Unix supercomputing environment', Technical Report UCB/CSD 92/712, University of California, Berkeley, Computer Science Division, (November 1992).
48. G.P. Bozman, H.H. Ghannad and E.D. Weinberger, 'A trace-driven study of CMS file references', IBM Journal of Research and Development, 35, (5-6), 815-828 (1991).
49. David W. Jensen and Daniel A. Reed, 'File archive activity in a supercomputer environment', Technical Report UIUCDCS-R-91-1672, University of Illinois at Urbana-Champaign, Department of Computer Science, April 1991.
50. Keith Muller and Joseph Pasquale, 'A high performance multi-structured file system design', Proceedings of the Thirteenth ACM Symposium on Operating Systems Principles, October 1991, pp. 56-67.
51. Carl D. Tait and Dan Duchamp, 'Detection and exploitation of file working sets', Proceedings of the 11th International Conference on Distributed Computing Systems, May 1991, pp. 2-9.
52. Prabuddha Biswas and K.K. Ramakrishan, 'File access characterization of VAX/VMS environments', Proceedings of the 10th International Conference on Distributed Systems, May 1990, pp. 227-234.
53. K. K. Ramakrishnan, P. Biswas and R. Karedla, 'Analysis of file I/O traces in commercial computing environments', Proceedings of the 1992 ACM SIGMETRICS and Performance '92 International Conference on Measurement and Modeling of Computer Systems, June 1992, pp. 78-90.
54. Prabuddha Biswas, K. K. Ramakrishnan and Don Towsley, 'Trace driven analysis of write caching policies for disks', Proceedings of the 1993 ACM SIGMETRICS Conference on Measurement and Modeling of Computer Systems, June 1993, pp. 13-23.
55. Kim Korner, 'Intelligent caching for remote file service', Proceedings of the 10th International Conference on Distributed Systems, May 1990, pp. 220-226.
56. Carl Staelin and Hector Garcia-Molina, 'File system design using large memories', Technical Report CS-TR-246-90, Princeton University, Department of Computer Science, February 1990. Appeared in 5th Jerusalem Conference on Information Technology, Jerusalem, Israel, October 1990.
57. Michael Burrows, 'Efficient data sharing', Ph.D. dissertation, University of Cambridge, December 1988.
58. Rick Floyd, 'Directory reference patterns in a UNIX environment', Technical Report TR 179, Department of Computer Science, University of Rochester, August 1986.
59. Songnian Zhou, Hervé Da Costa and Alan Jay Smith, 'A file system tracing package for Berkeley UNIX', USENIX Summer Conference Proceedings, USENIX Association, June 1985.
60. Geoffrey H. Kuenning, Gerald J. Popek and Peter L. Reiher, 'An analysis of trace data for predictive file caching in mobile computing', USENIX Summer Conference Proceedings, USENIX Association, June 1994.
61. Michael Dahlin, Clifford Mather, Randolph Wang, Thomas Anderson and David Patterson, 'A quantitative analysis of cache policies for scalable network file systems', Proceedings of the 1994 ACM SIGMETRICS Conference on Measurement and Modeling of Computer Systems, Nashville, TN, May 1994, pp. 150-160.
62. Matt Blaze, 'NFS tracing by passive network monitoring', USENIX Winter Conference Proceedings, USENIX Association, January 1992, pp. 333-343.
63. Ken W. Shirriff and John K. Ousterhout, 'A trace-driven analysis of name and attribute caching in a distributed system', USENIX Winter Conference Proceedings, USENIX Association, January 1992, pp. 315-31.
64. Alan B. Sheltzer, Robert Lindell and Gerald J. Popek, 'Name service locality and cache design in a distributed operating system', Proceedings of the Sixth International Conference on Distributed Computing Systems, May 1986, pp. 515-522.
65. M. Satyanarayanan, Distributed File Systems, chapter 14, pp. 353-383, Distributed Systems, (2nd ed.) Addison-Wesley, Reading, Massachusetts, 1993.
66. B. Walker, G. Popek, R. English, C. Kline and G. Thiel, 'The LOCUS distributed operating system', Proceedings of the Ninth Symposium on Operating System Principles, October 1983.
67. C. Thacker, L. Stewart and E. Satterthwaite, 'Firefly: a multiprocessor workstation', IEEE Transactions on Computers, 37, (8) 909-920 (1988).
68. Michael N. Nelson, Brent B. Welch and John K. Ousterhout, 'Caching in the Sprite network file system', ACM Transactions on Computer Systems, 6, (1), 134-154 (1988).
69. IBM, OS/VS2 MVS system programming library: system management facilities (SMF)', Technical Report GN28-2903, IBM Corporation, May 1978.
70. Randy Appleton, Personal communication, July 1994.
71. J.D. Simonetti, 'A system call trace facility', Technical Report 85/13, State University of New York at Stony Brook, 1985.