Task scheduling of massive spatial data processing across distributed data centers: What's new?

Proceedings of the International Conference on Parallel and Distributed Systems - ICPADS

Volumn , Issue , 2011, Pages 976-981

  Song, Weijing ^a   Yue, Shasha ^a   Wang, Lizhe ^a   Zhang, Wanfeng ^a   Liu, Dingsheng ^a  

^a CENTER FOR EARTH OBSERVATION AND DIGITAL EARTH   (China)

Author keywords

Massive data processing; Spatial data infrastructure; Task scheduling

Indexed keywords

DISTRIBUTED COMPUTING RESOURCES; DISTRIBUTED DATA; MASSIVE DATA SETS; MASSIVE SPATIAL DATA; MULTIPLE DATA; RESEARCH ISSUES; SCHEDULING MECHANISM; SPATIAL DATA INFRASTRUCTURE; TASK MODELS; TASK-SCHEDULING;

COMPUTER SYSTEMS; DATA PROCESSING; METADATA; MULTITASKING; SCHEDULING; SCHEDULING ALGORITHMS;

SPATIAL DISTRIBUTION;

EID: 84863018214     PISSN: 15219097     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICPADS.2011.134     Document Type: Conference Paper

References (68)

1. Shashi Shekhar, Sanjay Chawla. Spatial Database: A Tour. New York: Prentice Hall, 2003.
2. Dean Williams. The planet at their fingertips: Climate modeling data heats up, SciDAC Review Journal, 2009, 19:42-54.
3. Chaowei Yang, David Wong, Qianjun Miao, Ruixin Yang. Advanced Geoinformation Science, New York: CRC press, 2010.
4. A. Krishna Sinha. Geoinformatics: data to knowledge. Boulder: Geological Society of America, 2006.
5. NASA EOSDIS Web site. URL: http://www.esdis.eosdis.nasa.gov/.
6. Fang Huang. A research of cluster based parallel GIS key technology and prototype implementation. Ph.D. thesis, Chinese Academics and Science. 2008.
7. Fang Huang, Dingsheng Liu, Xiaowen Li, Lizhe Wang, Wenbo Xu. Preliminary Study of a Cluster-Based Open- Source Parallel GIS Based on the GRASS GIS. International Journal of Digital Earth.
8. Lizhe Wang, Marcel Kunze, Jie Tao: Performance evaluation of virtual machine-based Grid workflow system Concurrency and Computation: Practice and Experience 20(15): 1759-1771 (2008).
9. Ladislau Blni, Kyungkoo Jun, Krzysztof Palacz, Radu Sion, Dan C. Marinescu. The Bond Agent System and Applications. Second International Symposium on Agent Systems and Applications and Fourth International Symposium on Mobile Agents (ASA/MA) 2000: 99-112.
10. Prashant R. Chandra, etc. Darwin: customizable resource management for value-added network services. IEEE Network, 2001, 15(1): 22-35.
11. Geoff Coulson, etc. A retrospective on the design of the GOPI middleware platform. Multimedia System, 2002, 8(5): 340-352.
12. Saleem N. Bhatti, etc. Network QoS for Grid Systems. International Journal of High Performance Computing Applications, 2003, 17(3): 219-236.
13. Richard McClatchey, Ashiq Anjum, Heinz Stockinger, Arshad Ali, Ian Willers, Michael Thomas. Data Intensive and Network Aware (DIANA) Grid Scheduling. Journal of Grid Computing, 2007, 5(1):43-64. (Pubitemid 46909012)
14. Chao-Tung Yang, Fang-Yie Leu, Sung-Yi Chen. Network Bandwidth-aware job scheduling with dynamic information model for Grid resource brokers. The Journal of Supercomputing, 2010, 52(3): 199-223.
15. Xin Zuo, Alexey L. Lastovetsky: Experiments with a Software Component Enabling NetSolve with Direct Communications in a Non-Intrusive and Incremental Way. The 21th International Parallel and Distributed Processing Symposium (IPDPS), pp. 1-8, 2007.
16. Agustn C. Caminero, Omer Rana, Blanca Caminero, Carmen Carrin: Network-aware heuristics for inter-domain meta-scheduling in Grids. Journal of Computer System Science, 2011, 77(2): 262-281.
17. Davide Adami, etc. A Three-Dimensional Resource Scheduling Algorithm for a Network-Aware Grid. Proceedings of the Global Communications Conference, pp. 1-5, 2010.
18. Saman Taghavi Zargar, etc. RRNA: reliable soft real-time network aware grid scheduling algorithm using round trip time. Spring Simulation Multiconference, page 1-7, 2009.
19. Jing Taipiao, Jun Yan. A Network-aware Virtual Machine Placement and Migration Approach in Cloud Computing. International Conference on Grid and Cooperative Computing (GCC), pp. 87-92, 2010.
20. Francesco Palmieri. Network-aware scheduling for real-time execution support in data-intensive optical Grids. Future Generation Computer System, 25(7):794-803, 2009.
21. Luis Tomas, etc. Studying the Influence of Network- Aware Grid Scheduling on the Performance Received by Users. International Conference of On the Move to Meaningful Internet Systems, pp. 726-743, 2008.
22. Tevfik Kosar, Mehmet Balman. A new paradigm: Data-aware scheduling in grid computing. Future Generation Computer System, 2009, 25(4):406-413.
23. Kavitha Ranganathan, Ian T. Foster. Decoupling Computation and Data Scheduling in Distributed Data- Intensive Applications. International Symposium on High Performance Distributed Computing (HPDC), pp. 352-358, 2002.
24. Seonho Kim, Jon B. Weissman. A Genetic Algorithm Based Approach for Scheduling Decomposable Data Grid Applications. International Conference on Parallel Processing, pp. 406-413, 2004.
25. Atsuko Takefusa, Osamu Tatebe, Satoshi Matsuoka, Youhei Morita. Performance Analysis of Scheduling and Replication Algorithms on Grid Datafarm Architecture for High-Energy Physics Applications. International Symposium on High Performance Distributed Computing (HPDC), pp. 34-47, 2003.
26. Jihie Kim, Ewa Deelman, Yolanda Gil, Gaurang Mehta, Varun Ratnakar. Provenance trails in the Wings/Pegasus system. Concurrency and Computation: Practice and Experience, 2008, 20(5):587-597. (Pubitemid 351428371)
27. Young Choon Lee, Albert Y. Zomaya. Data Sharing Pattern Aware Scheduling on Grids. International Conference on Parallel Processing, pp. 365-372, 2006. (Pubitemid 47159063)
28. Srikumar Venugopal, Rajkumar Buyya. An SCP-based heuristic approach for scheduling distributed data-intensive applications on global grids. Journal of Parallel Distributed Computing, 68(4):471-487, 2008. (Pubitemid 351296392)
29. Leitao Guo, Hongwei Sun, Zhiguo Luo. A Data Distribution Aware Task Scheduling Strategy for MapReduce System. International Conference on Cloud Computing, pp. 694-699, 2009.
30. Ioan Raicu, Yong Zhao, Ian T. Foster, Alexander S. Szalay. Data Diffusion: Dynamic Resource Provision and Data-Aware Scheduling for Data Intensive Applications, Technical Report, CoRR abs/0808.3535, 2008.
31. USA Earth Science Grid Project. URL: http://www.earthsystemgrid.org.
32. Phase 3 of the Coupled Model Intercomparison Project (CMIP3). URL: http://cmip-pcmdi.llnl.gov/cmip3overview.html?submenuheader=1.
33. Michael Wilde, Ian T. Foster, Kamil Iskra, Peter H. Beckman, Zhao Zhang, Allan Espinosa, Mihael Hategan, Ben Clifford, Ioan Raicu. Parallel Scripting for Applications at the Petascale and Beyond. IEEE Computer, 2009, 42(11): 50-60.
34. Ioan Raicu, Yong Zhao, Catalin Dumitrescu, Ian T. Foster, Michael Wilde. Falkon: a FAst and Light-weight tasK executiON framework. International conference on Supercomputing, pp. 43-51, 2007.
35. Yong Zhao, etc. Swift: Fast, Reliable, Loosely Coupled Parallel Computation. IEEE International Conference on Services Computing, pp.199-206, 2007.
36. Kamer Kaya. Iterative-improvement-based heuristics for adaptive scheduling of tasks sharing files on heterogeneous master-slave environments. IEEE Trans. Parallel Distributed System, 2006, 17(8):883-896. (Pubitemid 44070155)
37. Henri Casanova, Dmitrii Zagorodnov, Francine Berman, and Arnaud Legrand. Heuristics for scheduling parameter sweep applications in grid environments. In 9th Heterogeneous Computing Workshop, pp 349-363, Cancun, Mexico, 2000. (Pubitemid 30592800)
38. Atsuko Takefusa, Osamu Tatebe, Satoshi Matsuoka, and Youhei Morita. Performance analysis of scheduling and replication algorithms on grid datafarm architecture for high-energy physics applications. The 12th IEEE International Symposium on High Performance Distributed Computing, pp. 34-41, 2003.
39. Arnaud Giersch, Yves Robert, and Frederic Vivien. Scheduling tasks sharing files from distributed repositories. The 10th International Euro-Par Conference, pp. 246-253, Sep. 2004.
40. Frederic Desprez and Antoine Vernois. Simultaneous Scheduling of Replication and Computation for Bioinformatic Applications on the Grid. In Workshop on Challenges of Large Applications in Distributed Environments, pp. 66-74, Research Triangle Park, NC, July 2005. (Pubitemid 44419196)
41. John Bent, etc. Coordination of Data Movement with Computation Scheduling on a Cluster. In Workshop on Challenges of Large Applications in Distributed Environments, pp. 25-34, Research Triangle Park, NC, July 2005. (Pubitemid 44419192)
42. Rocky Dunlap, etc. Earth system curator: metadata infrastructure for climate modeling. Earth Science Informatics, 2008, 1(3-4):131-149.
43. Jamie Shiers. The Worldwide LHC Computing Grid (worldwide LCG). Computer Physics Communications, 2007, 177(1-2): 219-223. (Pubitemid 46935627)
44. Tony Hey, Stewart Tansley, Kristin Tolle. The Fourth Paradigm: Data-Intensive Scientific Discovery. Seattle: Microsoft Research, Oct 2009.
45. Mao Xi, Li Qi, Zhang Junkui. Distributed modeling environment for global climate change research. The 18th International Conference on Geoinformatics, pp. 1-4, 2010.
46. Eric Clevede, David Weissenbach, Benoit Gotab. Distributed jobs on EGEE Grid infrastructure for an Earth science application: moment tensor computation at the centroid of an earthquake. Journal of Earth Science Informatics, 2009, 2(1-2):97-106.
47. Nataliia Kussul, Andrii Shelestov, Serhiy Skakun. Grid system for flood extent extraction from satellite images. Journal of Earth Science Informatics, 2008, 1(3-4):105-117.
48. Taezoon Park, Lan Zhao: Requirement Analysis of the Portal Site Serving Distributed Climate Data for e-Science. Lecture Notes in Computer Science, 2007, 4558:450-456, Springer.
49. Joe Loiacono, Andy Germain, Jeff Smith: Network performance measurements for NASA Earth Observation System. Journal of Computer Networks, 2004, 46(3):299-320.
50. Lizhe Wang, Gregor von Laszewski, Fang Huang, Jai Dayal, Tom Frulani, Geoffrey Fox: Task scheduling with ANN-based temperature prediction in a data center: a simulation-based study. Eng. Comput. (Lond.) 27(4): 381-391 (2011).
51. Oliver Sinnen. Task Scheduling for Parallel Systems, New York: Wiley, 2007.
52. Fang Huang, Discuss on High Performance Grid-Based GIS. Geomatics World, 2007, 5(4):33-39.
53. Yong Xue, Wei Wan, and Jianwen Ai. High Performance Geocomputation Developments. World Scientific Research and Development, 30 (3): 314-319, 2008.
54. Yi Liu, Research of remote sensing service based on cloud computing mode. Application Research of Computers, 26 (9):3428-3431, 2009.
55. Pam Frost Gorder. Multicore Processors for Science and Engineering. Computing in Science and Engineering, 2007, 9(2):3-7.
56. David B. Kirk, Wen-mei W. Hwu. Programming Massively Parallel Processors: A Hands-on Approach, New York: Morgan Kaufmann, 2010.
57. largGordon Bell, Jim Gray, Alexander S. Szalay: Petascale Computational Systems. IEEE Computer: 2006, 39(1):110-112. (Pubitemid 43191709)
58. Ron Sass, etc. Reconfigurable Computing Cluster (RCC) Project: Investigating the Feasibility of FPGA-Based Petascale Computing. IEEE Symposium on Field-Programmable Custom Computing Machines, pp. 127-140, 2007.
59. Yixin Wu, Suman Kumar, Seung-Jong Park. Measurement and performance issues of transport protocols over 10 Gbps high-speed optical networks. Journal of Computer Networks, 54(3):475-488 (2010).
60. Ian Gorton, etc. Data-Intensive Computing in the 21st Century. IEEE Computer: 41(4):30-32, 2008.
61. Maya Gokhale, etc. Hardware Technologies for High-Performance Data-Intensive Computing. IEEE Computer, 2008, 41(4):60-68.
62. Fernandez-Quiruelas, etc. Benefits and Requirements of Grid Computing for Climate Applications: An Example with the Community Atmospheric Model. Environmental Modeling, 2011 preprint. URL: http://www.meteo.unican.es/files/ pdfs/2010FernandezQEMSESGrid.pdf.
63. EELA: E-science grid facility for Europe and Latin America. URL: http://www.eu-eela.eu/.
64. EGEE: Enabling Grid for E-sciencE. URL: http://www.eu-egee.org/.
65. Hangjian Feng, Nan Liu and Renyi Liu. Research of Computers, 2004, 7:111-113.
66. Fengsheng Li and Xiaofeng Zhou. Research and Application of Spatial Data Processing Based on SDPG. Computer Engineering, 2006, 32(7):72-74.
67. Minyi Ke, Huazhong Le, Zhihua Liu. Research of Cluster-Based Job Management System Oriented to Spatial Data Processing. Control and Measurement, 2006, 7S: 221-223.
68. Kai Wang, Quanyuan Wu, and Shuqiang Yang. Design and Implementation of Job Scheduling Algorithm for Multi- User MapReduce Clusters. Computer and modernization, 2010, (10): 23-28.