FreeLoader: Scavenging desktop storage resources for scientific data

Proceedings of the ACM/IEEE 2005 Supercomputing Conference, SC'05

Volumn 2005, Issue , 2005, Pages

  Vazhkudai, Sudharshan S ^a   Ma, Xiaosong ^a,b   Frech, Vincent W ^b   Strickland, Jonathan W ^b   Tammineedi, Nandan ^b   Scott, Stephen L ^a  

^a OAK RIDGE NATIONAL LABORATORY   (United States)

^b North Carolina State University   (United States)

Author keywords

Distributed storage; Parallel I O; Scientific data management; Serverless storage system; Storage cache; Storage scavenging; Striped storage

Indexed keywords

DISTRIBUTED STORAGE; PARALLEL I/O; SCIENTIFIC DATA MANAGEMENT; SERVERLESS STORAGE SYSTEMS; STORAGE CACHE; STORAGE SCAVENGING; STRIPED STORAGE;

CACHE MEMORY; COMPUTER SIMULATION; COMPUTER WORKSTATIONS; PARALLEL PROCESSING SYSTEMS; PERSONAL COMPUTERS;

NATURAL SCIENCES COMPUTING;

EID: 33845399711     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/SC.2005.27     Document Type: Conference Paper

References (46)

1. Seti@home: The search for extraterrestrial intelligence. http://setiathome.ssl.berkeley.edu/, 2003.
2. A. Adya, W. Bolosky, M. Castro, R. Chaiken, G. Cermak, J.Douceur, J. Howell, J. Lorch, M. Theimer, and R. Wattenhofer. FARSITE: Federated, available, and reliable storage for an incompletely trusted environment. In Proceedings of the 5th Symposium on Operating Systems Design and Implementation, 2002.
3. S. Annapureddy, M. J. Freedman, and D. Mazires. Shark: Scaling file servers via cooperative caching. In Proceedings of 2nd USENIX/ACM Symposium on Networked Systems Design and Implementation (NSDI'05), May 2005.
4. M. Beck, T. Moore, and J. Plank. An end-to-end approach to globally scalable network storage. In Proceedings of the ACMSIGCOMM Conference on Applications, Technologies, Architectures, and Protocols for Computer Communication, 2002.
5. J. Bent, D. Thain, A. Arpaci-Dusseau, R. Arpaci-Dusseau, and M. Livny. Explicit control in a batch aware distributed file system. In Proceedings of the First USENIX/ACM Conference on Networked Systems Design and Implementation, March 2004.
6. J. Bent, V. Venkataramani, N. Leroy, A. Roy, J. Stanley, A. Arpaci-Dusseau, R. Arpaci-Dusseau, and M. Livny. Flexibility, manageability, and performance in a grid storage appliance. In Proceedings of the 11th High Performance Distributed Computing Symposium, 2002.
7. J. Bester, I. Foster, C. Kesselman, J. Tedesco, and S. Tuecke. GASS: A data movement and access service for wide area computing systems. In Proceedings of the Sixth Workshop on I/O in Parallel and Distributed Systems, 1999.
8. C. Blake and R. Rodrigues. High availability, scalable storage, dynamic peer networks: Pick two. In Proceedings of the 9th Workshop on Hot Topics in Operating Systems (HotOS '03), May 2003.
9. A. Butt, T. Johnson, Y. Zheng, and Y. Hu. Kosha: A peer-to-peer enhancement for the network file system. In Proceedings of Supercomputing, 2004.
10. P. Cams, W. Ligon III, R. Ross, and R. Thakur. PVFS: A Parallel File System For Linux Clusters. In Proceedings of the 4th Annual Linux Showcase and Conference, 2000.
11. A. Chien, B. Calder, S. Elbert, and K. Bhatia. Entropia: Architecture and performance of an enterprise desktop grid system. Journal of Parallel and Distributed Computing, 63(5), 2003.
12. I. Clarke, O. Sandberg, B. Wiley, and T. W. Hong. Freenet: A distributed anonymous information storage and retrieval system. Lecture Notes in Computer Science, 2000.
13. Cluster File Systems, Inc. Lustre: A scalable, high-performance file system, http://www.lustre.org/docs/-whitepaper.pdf, 2002.
14. B. Cohen. Incentives Build Robustness in BitTorrent. 2003.
15. R.A. Coyne and R.W. Watson. The parallel i/o architecture of the high-performance storage system (hpss). In Proceedings of the IEEE MSS Symposium, 1995.
16. J. Crowcroft and I. Pratt. Peer to Peer: peering into the future. In NETWORKING Tutorials, 2002.
17. M. Dahlin, R. Wang, T. E. Anderson, and D. A. Patterson. Cooperative caching: Using remote client memory to improve file system performance. In Operating Systems Design and Implementation, pages 267-280, 1994.
18. J. Douceur and W. Bolosky. A large-scale study of file-system contents. In Proceedings of SIGMETRICS, 1999.
19. P. Druschel and A. Rowstron. Storage management and caching in PAST, a large-scale, persistent peer-to-peer storage utility. In Proceedings of the 18th ACM Symposium on Operating System Principles, 2001.
20. J. Kubiatowicz et al. Oceanstore: An architecture for global-scale persistent storage. In the 9th International. Conference on Architectural Support for Programming Languages and Operating Systems, 2000.
21. M. J. Feeley, W. E. Morgan, F. H. Pighin, A. R. Karlin, H. M. Levy, and C. A. Thekkath. Implementing global memory management in a workstation cluster. In Proceedings of the 15th ACM Symposium on Operating Systems Principles, 1995.
22. S. Gadde, J. Chase, and M. Rabinovich. A taste of crispy squid. In Proceedings of the Workshop on Internet Server Performance, June 1998.
23. S. Ghemawat, H. Gobioff, and S. Leung. The Google file system. In Proceedings of the 19th Symposium on Operating Systems Principles, 2003.
24. J. Gray and A. S. Szalay. Scientific Data Federation. In I. Foster and C. Kesselman, editors, The Grid 2: Blueprint for a New Computing Infrastructure, pages 95-108, 2003.
25. P. Gummadi, R. Dunn, S. Saroiu, S. Gribble, H. Levy, and J. Zahorjan. Measurement, modeling, and analysis of a peer-to-peer file-sharing workload. In Proceedings of the 29th Symposium on Operating Systems Principles, 2003.
26. A. Gupta, B. Lin, and P. Dinda. Measuring and understanding user comfort with resource borrowing. In Proceedings of the 13th IEEE International Symposium on High Performance Distributed Computing, 2004.
27. J. Hartman and J. Ousterhout. The Zebra striped network file system. In Proceedings of the 14th Symposium on Operating Systems Principles, 1993.
28. J. H. Howard. An overview of the andrew file system. 1998.
29. http://www.coda.cs.cmu.edu. CODA File System, 1987.
30. A. Iamnitchi, M. Ripeanu, and I. Foster. Small-world file-sharing communities. In Infocom, 2004.
31. S. Iyer, A. Rowstron, and P. Druschel. Squirrel: a decentralized peer-to-peer web cache. In Proceedings of the 21st ACM Symposium on Principles of Distributed Computing, 2002.
32. E. Lee and C. Thekkath. Petal: Distributed virtual disks. In Proceedings of the 7th International Conference on Architectural. Support for Programming Languages and Operating Systems, 1996.
33. M. Litzkow, M. Livny, and M. Mutka. Condor - a hunter of idle workstations. In Proceedings of the 8th International. Conference on Distributed Computing Systems, 1988.
34. E. Markatos. Tracing a large-scale peer to peer system: An hour in the life of gnutella. In Proceedings of the 2nd IEEE/ACM International Symposium on Cluster Computing and the Grid, 2002.
35. National center for biotechnology information, http://-www.ncbi.nlm.nih. gov/, 2005.
36. SHARMAN NETWORKS. The kazaa media desktop. http://www.kazaa.com.
37. R. Novaes, P. Roisenberg, R. Scheer, C. Northfleet, J. Jornada, and W. Cirne. Non-dedicated distributed environment: A solution for safe and continuous exploitation of idle cycles. In Proceedings of the Workshop on Adaptive Grid Middleware, 2003.
38. B. Nowicki. NFS: Network File System Protocol Specification. Network Working Group RFC1094, 1989.
39. E. J. Otoo, D. Rotem, and A. Romosan. Optimal file-bundle caching algorithms for data-grids. In Proceedings of Supercomputing, 2004.
40. G. Popek and B. J. Walker. The LOCUS Distributed System Architecture. MIT Press, 1985.
41. P. Sarkar and J. Hartman. Efficient cooperative caching using hints. In Proceedings of the 2nd ACM Symposium on Operating Systems Design and Implementation (OSDI), 1996.
42. F. Schmuck and R. Haskin. GPFS: a shared-disk file system for large computing clusters. In Proceedings of the First Conference on File and Storage Technologies, 2002.
43. Sloan digital sky survey, http://www.sdss.org, 2005.
44. J. Strickland, V. Freeh, X. Ma, and S. Vazhkudai. Governor: Autonomic throttling for aggressive idle resource scavenging. In Proceedings of the 2nd IEEE International. Conference on Autonomic Computing, 2005.
45. A. Szalay and J. Gray. The world-wide telescope. Science, 293(14):2037-2040, 2001.
46. C. Thekkath, T. Mann, and E. Lee. Frangipani: A scalable distributed file system. In Proceedings of the 16th Symposium on Operating Systems Principles, 1997.