Design and implementation of GXP make - A workflow system based on make

Proceedings - 2010 6th IEEE International Conference on e-Science, eScience 2010

Volumn , Issue , 2010, Pages 214-221

  Taura, Kenjiro ^a   Matsuzaki, Takuya ^a   Miwa, Makoto ^a   Kamoshida, Yoshikazu ^a   Yokoyama, Daisaku ^a   Dun, Nan ^a   Shibata, Takeshi ^a   Jun, Choi Sung ^a   Tsujii, Jun'ichi ^a  

^a UNIVERSITY OF TOKYO   (Japan)

Author keywords

[No Author keywords available]

Indexed keywords

BIOMEDICAL TEXT; DEPENDENCE ANALYSIS; DISTRIBUTED SYSTEMS; GNU MAKE; PROTEIN-PROTEIN INTERACTIONS; REAL APPLICATIONS; SPECIFIC DESIGN; WORK-FLOW SYSTEMS; WORK-FLOWS;

SUPERCOMPUTERS;

MANAGEMENT;

EID: 79951824258     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/eScience.2010.43     Document Type: Conference Paper

References (31)

1. K. Taura, "GXP: An interactive shell for the grid environment," in IWIA, 2004, pp. 59-67.
2. "Lustre," http://wiki.lustre.org/.
3. "IBM general parallel file system," http://www-03.ibm.com/ systems/clusters/software/gpfs/index.html.
4. "SSH filesystem," http://fuse.sourceforge.net/sshfs.html.
5. O. Tatebe, N. Soda, Y. Morita, S. Matsuoka, and S. Sekiguchi, "Gfarm v2: A grid file system that supports high-performance distributed and parallel data computing," in CHEP, 2004.
6. N. Dun, K. Taura, and A. Yonezawa, "GMount: An ad hoc and locality-aware distributed file system by using SSH and FUSE," in CCGRID, 2009, pp. 188-195.
7. D. Thain and M. Livny, "Parrot: Transparent user-level middleware for data-intensive computing," in Workshop on Adaptive Grid Middleware, 2003.
8. Y. Zhao, M. Hategan, B. Clifford, I. Foster, G. von Laszewski, I. Raicu, T. Stef-Praun, and M. Wilde, "Swift: Fast, reliable, loosely coupled parallel computation," in IEEE International Workshop on Scientific Workflows, 2007.
9. M. Isard, M. Budiu, Y. Yu, A. Birrell, , and D. Fetterly, "Dryad: Distributed data-parallel programs from sequential building blocks," in EuroSys, 2007.
10. T. Hiraishi, T. Abe, Y. Miyake, T. Iwashita, and H. Nakashima, "Xcrypt: Flexible and intuitive job-parallel script," in SACSIS, 2010, pp. 183-191, (in Japanese).
11. "Hadoop," http://hadoop.apache.org/.
12. T. Oinn, M. Greenwood, M. Addis, N. Alpdemir, J. Ferris, K. Glover, C. Goble, A. Goderis, D. Hull, D. Marvin, P. Li, P. Lord, M. Pocock, M. Senger, R. Stevens, A. Wipat, and C. Wroe, "Taverna: lessons in creating a workflow environment for the life sciences," Concurrency and Computation: Practice and Experience, vol. 18, no. 10, pp. 1067-1100, August 2006.
13. D. Churches, G. Gombas, A. Harrison, J. Maassen, C. Robinson, M. Shields, I. Taylor, and I. Wang, "Programming scientific and distributed workflow with Triana services," Concurrency and Computation: Practice and Experience, vol. 18, no. 10, pp. 1021-1037, August 2006.
14. B. Ludäscher, I. Altintas, C. Berkley, D. Higgins, E. Jaeger, M. Jones, E. A. Lee, J. Tao, and Y. Zhao, "Scientific workflow management and the Kepler system," Concurrency and Computation: Practice and Experience, vol. 18, no. 10, pp. 1039-1065, August 2006.
15. M. Tanaka and O. Tatebe, "Pwrake: A parallel and distributed flexible workflow management tool for wide-area data intensive computing (poster)," in HPDC, 2010.
16. L. Yi, C. Moretti, S. Emrich, J. Kenneth, and D. Thain, "Harnessing parallelism in multicore clusters with the all-pairs and wavefront abstractions," in HPDC, 2009, pp. 1-10.
17. "Grid Engine project home," http://gridengine.sunsource.net/.
18. P. Couvares, T. Kosar, A. Roy, J. Weber, and K. Wenger, Workflows for e-Science: Scientific Workflows for Grids. Springer Press, 2007, ch. Workflow in Condor.
19. E. Deelman, G. Singh, M.-H. Su, J. Blythe, Y. Gil, C. Kesselman, G. Mehta, K. Vahi, G. B. Berriman, J. Good, A. Laity, J. C. Jacob, and D. S. Katz., "Pegasus: a framework for mapping complex scientific workflows onto distributed systems," Scientific Programming Journal, vol. 13, no. 3, pp. 219-237, 2005.
20. I. Raicu, Y. Zhao, C. Dumitrescu, I. Foster, and M. Wilde, "Falkon: a Fast and Light-weight tasK executiON framework," in SC, 2007.
21. J. Dean and S. Ghemawat, "MapReduce: Simplified data processing on large clusters," in OSDI, 2004, pp. 137-150.
22. B. Folliot, "Implementation of a parallel and distributed MAKE on NFS with GATOS," in International Phoenix Conference on Computers and Communications, 1990, p. 871.
23. J. Buffenbarger, "A large-scale fault-tolerant distributed software-build process," in British Computer Society Configuration Management Specialist Group Conference, 2005.
24. A. Lih and E. Zadok, "PGMAKE: A portable distributed make system," Computer Science Department, Columbia University, Tech. Rep., 1994.
25. "GXP Grid and Cluster Shell," http://www.logos.ic.i.u-tokyo.ac. jp/gxp/.
26. D. Thain, T. Tannenbaum, and M. Livny, "Distributed computing in practice: the Condor experience." Concurrency - Practice and Experience, vol. 17, no. 2-4, pp. 323-356, 2005.
27. "InTrigger platform," http://www.intrigger.jp/.
28. M. Miwa, R. Sætre, J.-D. Kim, and J. Tsujii, "Event extraction with complex event classification using rich features," JBCB, vol. 8, no. 1, pp. 131-146, Feb. 2010.
29. T. Ninomiya, T. Matsuzaki, Y. Miyao, and J. Tsujii, "A log-linear model with an n-gram reference distribution for accurate HPSG parsing," in IWPT, 2007.
30. K. Sagae and J. Tsujii, "Dependency parsing and domain adaptation with LR models and parser ensembles," in The CoNLL 2007 Shared Task (EMNLP-CoNLL'07), 2007.
31. "Pubmed," http://www.ncbi.nlm.nih.gov/pubmed.