Block-based concurrent and storage-aware data streaming for grid applications with lots of small files

2009 9th IEEE/ACM International Symposium on Cluster Computing and the Grid, CCGRID 2009

Volumn , Issue , 2009, Pages 538-543

  Zhang, Wen ^a   Cao, Junwei ^a,b   Zhong, Yisheng ^a,b   Liu, Lianchen ^a,b   Wu, Cheng ^a,b  

^a TSINGHUA UNIVERSITY   (China)

^b TSINGHUA NATIONAL LABORATORY FOR INFORMATION SCIENCE AND TECHNOLOGY   (China)

Author keywords

[No Author keywords available]

Indexed keywords

DATA OVERFLOW; DATA STREAMING; DATA TRANSFERRING; GRID APPLICATIONS; GRID COMPUTING APPLICATIONS; GRIDFTP; MAKESPAN; NUMBER OF BLOCKS; OPTIMIZATION METHOD; SMALL FILES;

CLUSTER COMPUTING; COMPUTER SCIENCE; DATA REDUCTION; GRID COMPUTING; SCHEDULING;

DATA PROCESSING;

EID: 70349746795     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/CCGRID.2009.26     Document Type: Conference Paper

References (16)

1. D. A. Brown, P. R. Brady, A. Dietz, J. Cao, B. Johnson, and J. McNabb, "A Case Study on the Use of Workflow Technologies for Scientific Analysis: Gravitational Wave Data Analysis", in I. J. Taylor, D. Gannon, E. Deelman, and M. S. Shields (Eds.), Workflows for eScience: Scientific Workflows for Grids, Springer Verlag, pp. 39-59, 2007.
2. th IEEE Int. Symp. on High Performance Distributed Computing, pp. 225-234, 2002.
3. R. Pordes for the Open Science Grid Consortium, "The Open Science Grid", in Proc. Computing in High Energy and Nuclear Physics Conf., Interlaken, Switzerland, 2004.
4. th Int. Conf. on Distributed Computing Systems, pp. 104-111, 1988.
5. I. Foster and C. Kesselman, "Globus: A Metacomputing Infrastructure Toolkit", Int. J. Supercomputer Applications, Vol. 11, No. 2, pp. 115-128, 1997.0
6. B. Allcock, J. Bester, J. Bresnahan, A. L. Chervenak, I. Foster, C. Kesselman, S. Meder, V. Nefedova, D. Quesnal, and S. Tuecke, "Data Management and Transfer in High Performance Computational Grid Environments", Parallel Computing, Vol. 28, No. 5, pp. 749-771, 2002.
7. th IEEE/ACM Int. Conf. on Grid Computing, Tsukuba, Japan, 258-265, 2008.
8. A. Arasu, B. Babcock, S. Babu, M. Datar, K. Ito, I. Nishizawa, J. Rosenstein, and J. Widom, "STREAM: The Stanford Stream Data Manager", IEEE Data Eng. Bull., Vol. 26, No. 1, pp.19-26, 2003.
9. D. Carney, U. Cetinterne, M. Cherniack, et. al., "Monitoring Streams: A New Class of Data Management Applications", in Proc. Int. Conf. on Very Large Data Bases, pp. 215-225, 2002.
10. J. Chen, D. DeWitt, F. Tian, and Y. Wang, "NiagaraCQ: A Scalable Continuous Query System for Internet Databases", in Proc. ACM SIGMOD Int. Conf. on Management of Data, pp. 379-390, 2000.
11. C. Cranoe, T. Johnson, V. Shkapenyuk, and O. Spatscheck, "Gigascope: a Stream Database for Network Applications", in Proc. ACM SIGMOD Int. Conf. on Management of Data, pp. 379-390, 2003.
12. B. Agarwalla, N. Ahmed, D. Hilley, and U. Ramachandran, "Streamline: Scheduling Streaming Applications in a Wide Area Environment", Multimedia Systems, Vol. 13, No. 1, pp. 69-85, 2007.
13. th IEEE/ACM Int. Workshop on Grid Computing, pp. 243-250, 2004.
14. G. Fox, H. Gadgil, S. Pallickara, M. Pierce, R. L. Grossman, et. al., "High Performance Data Streaming in Service Architecture", Technical Report, Indiana University and University of Illinois at Chicago, July 2004.
15. th IEEE/ACM Int. Conf. on Grid Computing, pp. 243-250, 2007.
16. J. Bresnahan, M. Link, R. Kettimuthu, D. Fraser and I. Foster, "GridFTP Pipelining", in Proc. TeraGrid Conf., 2007.