Analysis of workload behavior in scientific and historical long-term data repositories

ACM Transactions on Storage

Volumn 8, Issue 2, 2012, Pages

  Adams, Ian F ^a   Storer, Mark W ^b   Miller, Ethan L ^a  

^a UNIVERSITY OF CALIFORNIA   (United States)

^b Network Appliance Inc   (United States)

Author keywords

Archival storage; Tertiary storage; Trace analysis

Indexed keywords

ACCESS PATTERNS; ARCHIVAL STORAGE SYSTEMS; ARCHIVAL SYSTEMS; COST EFFICIENCY; DATA REPOSITORIES; DATA STORAGE SYSTEMS; HARD DRIVES; HISTORICAL ARCHIVE; INDEXING SERVICE; KNOWLEDGE GAPS; MANAGEMENT PROCESS; MEDIA TYPES; MEDICAL DATA; STORAGE SPACES; WRITE ONCE;

DATA STORAGE EQUIPMENT; INFORMATION MANAGEMENT; TRACE ANALYSIS;

BEHAVIORAL RESEARCH;

EID: 84863626117     PISSN: 15533077     EISSN: 15533093     Source Type: Journal    
DOI: 10.1145/2180905.2180907     Document Type: Article

References (53)

1. AGRAWAL, N, BOLOSKY, W. J., DOUCEUR, J. R., and LORCH, J. R. 2007. A five-year study of file-system metadata. In Proceedings of the 5th USENIX Conference on File and Storage Technologies (FAST). 31-45.
2. AGRAWAL, N, ARPACI-DUSSEAU, A. C, and ARPACI-DUSSEAU, R. H. 2009. Generating realistic impressions for file-system benchmarking. In Proceedings of the 7th USENIX Conference on File and Storage Technologies (FAST). 125-138.
3. ALASKA STATE. 2010. Alaska's digital archives. vilda.alaska.edu.
4. AMAZON. 2011. Amazon's simple storage service. http://aws.amazon. com/s3/.
5. ANDERSON, E. 2009. Capture, conversion, and analysis of an intense NFS workload. In Proceedings of the 7th USENIX Conference on File and Storage Technologies.
6. ANDERSON, E., ARLITT, M., CHARLES B. MORREY, I., and VEITCH, A. 2009. DataSeries: An efficient, flexible data format for structured serial data. ACM SIGOPS Operat. Syst. Rev. 43, 1, 70-75.
7. BAIRAVASUNDARAM, L. N, GOODSON, G. R., PASUPATHY, S., and SCHINDLER, J. 2007. An analysis of latent sector errors in disk drives. In Proceedings of the SIGMETRICS Conference on Measurement and Modeling of Computer Systems.
8. BAIRAVASUNDARAM, L. N, GOODSON, G. R., SCHROEDER, B., ARPACI-DUSSEAU, A. C, and ARPACI-DUSSEAU, R. H. 2008. An analysis of data corruption in the storage stack. In Proceedings of the 6th USENIX Conference on File and Storage Technologies (FAST). 223-238.
9. BAKER, M., KEETON, K., and MARTIN, S. 2005. Why traditional systems don't help us save stuff forever. In Proceedings of 1st IEEE Workshop on Hot Topics in System Dependendability.
10. BAKER, M., SHAH, M., ROSENTHAL, D. S. H., ROUSSOPOULOS, M., MANIATIS, P., GIULI, T., and BUNGALE, P. 2006. A fresh look at the reliability of long-term digital storage. In Proceedings of EuroSys'06. 221-234.
11. BENT, J., GIBSON, G., GRIDER, G., MCCLELLAND, B., NOWOCZYNSKI, P., NUNEZ, J., POLTE, M., and WINGATE, M. 2009. PLFS: A checkpoint filesystem for parallel applications. In Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis.
12. aLIFORNIA DWR. 2010. California Department of Water Resources water reports. http://www.water.ca.gov/waterdatalibrary/docs/Hydstra/index.cfm.
13. CHRONICLES. 2011. Chronicles of life: Save your memories forever. http://www.chronicleoflife. com/.
14. COLARELLI, D. and GRUNWALD, D. 2002. Massive arrays of idle disks for storage archives. In Proceedings of the ACM/IEEE Conference on Supercomputing (SC'02).
15. CORNELL UNIVERSITY LIBRARY. 2010. Cornell University Library arXiv. http://arxiv.org/.
16. DAMOULAKIS, J. 2007. Opinion: Tape backup is WORN (write once, read never). http://www.computerworld.com/s/article/9026619/Opinion-Tape-backup-is- WORN-write-once-read-never-.
17. DAYAL, S. 2008. Characterizing HEC Storage Systems at Rest. Tech. rep. CMU-PDL-08-109, Carnegie Mellon University.
18. DROPBOX. 2011. Dropbox. http://www.dropbox.com/.
19. GIBSON, T, MILLER, E. L., and LONG, D. D. E. 1998. Long-term file activity and inter-reference patterns. In Proceedings of the 24th International Conference for the Resource Management and Performance and Performance Evaluation of Enterprise Computing Systems (CMG'98). CMG, Anaheim, CA, 976-987.
20. GIBSON, T. J. and MILLER, E. L. 1998. Long-term file activity patterns in a UNIX workstation environment. In Proceedings of the 6th Goddard Conference on Mass Storage Systems and Technologies/15th IEEE Symposium on Mass Storage Systems. 355-372.
21. HIPAA. 1996. Health Information Portability and Accountability Act.
22. IBM. 2010. IBM 3380 direct access storage device. http://www-03.ibm.com/ ibm/history/exhibits/storage/storage-3380e.html.
23. JAFFE, E. and KIRKPATRICK, S. 2009. Architecture of the Internet archive. In Proceedings of the Israeli Experimental Systems Conference (SYSTOR'09).
24. JENSEN, D. W. and REED, D. A. 1993. File archive activity in a supercomputing environment. In Proceedings of the 7th International Conference on Supercomputing (SuperComputing'93). 387-396.
25. LEUNG, A. W., PASUPATHY, S., GOODSON, G., and MILLER, E. L. 2008. Measurement and analysis of large-scale network file system workloads. In Proceedings of the USENIX Annual Technical Conference.
26. LILLIBRIDGE, M., ELNIKETY, S., BIRRELL, A., BURROWS, M., and ISARD, M. 2003. A cooperative Internet backup scheme. In Proceedings of the USENIX Annual Technical Conference. 29-42.
27. MANIATIS, P., ROUSSOPOULOS, M., GIULI, T. J., ROSENTHAL, D. S. H., and BAKER, M. 2005. The LOCKSS peer-to-peer digital preservation system. ACM Trans. Comput. Syst. 23, 1, 2-50. (Pubitemid 40479366)
28. MILLER, E. and KATZ, R. 1993. An analysis of file migration in a Unix supercomputing environment. In Proceedings of the Winter USENIX Technical Conference. 421-433.
29. MOORE, R. L., DAOUST, J., MCDONALD, R. H., and MINOR, D. 2007. Disk and tape storage cost models. In Archiving 2007.
30. NEW YORK STATE. 2010. New York State digital archives. http://www.archives.nysed.gov/aindex.shtml.
31. NOAA. 2010. National Climatic Data Center. http://www.ncdc.noaa.gov/oa/ ncdc.html.
32. ORNL. 2010. Distributed Active Archive Center. http://daac.ornl.gov/.
33. PINHEIRO, E. and BIANCHINI, R. 2004. Energy conservation techniques for disk array-based servers. In Proceedings of the 18th International Conference on Supercomputing.
34. PINHEIRO, E., WEBER, W.-D., and BARROSO, L. A. 2007. Failure trends in a large disk drive population. In Proceedings of the 5th USENIX Conference on File and Storage Technologies (FAST).
35. QUINLAN, S. and DORWARD, S. 2002. Venti: A new approach to archival storage. In Proceedings of the Conference on File and Storage Technologies (FAST). USENIX, 89-101.
36. ROSELLI, D., LORCH, J., and ANDERSON, T. 2000. A comparison of file system workloads. In Proceedings of the USENIX Annual Technical Conference. USENIX Association, 41-54.
37. SARBANES-OXLEY. 2002. Sarbanes-Oxley act 2002. www.soxlaw.com.
38. SCHROEDER, B. and GIBSON, G. A. 2007. Disk failures in the real world: What does an MTTF of 1, 000, 000 hours mean to you? In Proceedings of the 5th USENIX Conference on File and Storage Technologies (FAST). 1-16.
39. SMITH, A. J. 1981a. Analysis of long term file reference patterns for application to file migration algorithms. IEEE Trans. Softw. Engin. 7, 4, 403-417.
40. SMITH, A. J. 1981b. Long term file migration: Development and evaluation of algorithms. Comm. ACM 24, 8, 521-532.
41. STORER, M. W., GREENAN, K. M., MILLER, E. L., and VORUGANTI, K. 2007. POTSHARDS: Secure long-term storage without encryption. In Proceedings of the USENIX Annual Technical Conference. 143-156.
42. STORER, M. W., GREENAN, K. M., MILLER, E. L., and VORUGANTI, K. 2008. Pergamum: Replacing tape with energy efficient, reliable, disk-based archival storage. In Proceedings of the 6th USENIX Conference on File and Storage Technologies (FAST).
43. STRANGE, S. 1992. Analysis of long-term UNIX file access patterns for application to automatic file migration strategies. Tech. rep. UCB/CSD 92/700, University of California, Berkeley.
44. THERESKA, E., SALMON, B., STRUNK, J., WACHS, M., ABD-EL-MALEK, M., LOPEZ, J., and GRANGER, G. R. 2006. Stardust: Tracking activity in a distributed storage system. In Proceedings of the SIGMET RICS Conference on Measurement and Modeling of Computer Systems.
45. TRAEGER, A., ZADOK, E., JOUKOV, N., and WRIGHT, C. P. 2008. A nine year study of file system and storage benchmarking. ACM Trans. Storage 4, 2.
46. VOGELS, W. 1999. File system usage in Windows NT 4.0. In Proceedings of the 17th ACM Symposium on Operating Systems Principles (SOSP'99). 93-109. (Pubitemid 129548350)
47. WASHINGTON STATE. 2010. Washington State digital archives. http://www.digitalarchives.wa.gov/.
48. WILDANI, A. and MILLER, E. L. 2010. Semantic data placement for power management in archival storage. In Proceedings of the 5th International Workshop on Petascale Data Storage (PDSW10) (held in conjunction with SC2010).
49. WILDANI, A., SCHWARZ, T., MILLER, E. L., and LONG, D. D. E. 2009. Protecting against rare event failures in archival systems. In Proceedings of the 17th IEEE International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems (MASCOTS).
50. YOU, L. L., POLLACK, K. T., and LONG, D. D. E. 2005. Deep store: An archival storage system architecture. In Proceedings of the 21st International Conference on Data Engineering (ICDE'05).
51. ZHANG, Z., LIAN, Q., LIN, S., CHEN, W., CHEN, Y., and JIN, C. 2007. BitVault: A highly reliable distributed data retention platform. ACM SIGOPS Operat. Syst. Rev. 41, 2, 27-36.
52. ZHU, B., LI, K., and PATTERSON, H. 2008. Avoiding the disk bottleneck in the Data Domain deduplication file system. In Proceedings of the 6th USENIX Conference on File and Storage Technologies (FAST).
53. ZHU, Q., CHEN, Z., TAN, L., ZHOU, Y., KEETON, K., and WILKES, J. 2005. Hibernator: Helping disk arrays sleep through the winter. In Proceedings of the 20th ACM Symposium on Operating Systems Principles (SOSP'05). ACM.