SciMATE: A novel MapReduce-like framework for multiple scientific data formats

Proceedings - 12th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, CCGrid 2012

Volumn , Issue , 2012, Pages 443-450

  Wang, Yi ^a   Jiang, Wei ^a   Agrawal, Gagan ^a  

^a Ohio State University   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

CUSTOMIZABLE; DATA MINING APPLICATIONS; DATA SETS; DISTRIBUTED FILE SYSTEMS; FILE SYSTEMS; MAP-REDUCE; SCIENTIFIC DATA;

FILE ORGANIZATION; GRID COMPUTING;

MULTIPROCESSING SYSTEMS;

EID: 84863700233     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/CCGrid.2012.32     Document Type: Conference Paper

References (21)

1. HDF5. http:// hdf.ncsa.uiuc.edu/HDF5/.
2. NETCDF. http://www.unidata.ucar.edu/software/netcdf/.
3. David W. Aha, Dennis F. Kibler, and Marc K. Albert. Instance-based Learning Algorithms. Machine Learning, pages 6:37-66, 1991.
4. James Ahrens, Bruce Hendrickson, Gabrielle Long, Steve Miller, Rob Ross, and Dean Williams. Data-Intensive Science in the US DOE: Case Studies and Future Challenges. Computing in Science Engineering, 13(6):14-24, nov.-dec. 2011.
5. Joe B. Buck, Noah Watkins, Jeff LeFevre, Kleoni Ioannidou, Carlos Maltzahn, Neoklis Polyzotis, and Scott Brandt. Sci-Hadoop: Array-based Query Processing in Hadoop. In Proceedings of SC, 2011.
6. Cheng-Tao Chu, Sang Kyun Kim, Yi-An Lin, YuanYuan Yu, Gary R. Bradski, Andrew Y. Ng, and Kunle Olukotun. Map-Reduce for Machine Learning on Multicore. In Proceedings of the Twentieth Annual Conference on Neural Information Processing Systems (NIPS), pages 281-288. MIT Press, 2006.
7. Jeffrey Dean and Sanjay Ghemawat. MapReduce: Simplified Data Processing on Large Clusters. In Proceedings of OSDI, pages 137-150, 2004.
8. J. Ekanayake, S. Pallickara, and G Fox. MapReduce for Data Intensive Scientific Analyses. In IEEE Fourth International Conference on e-Science, pages 277-284, Dec 2008.
9. Jaliya Ekanayake, Hui Li, Bingjing Zhang, Thilina Gunarathne, Seung-Hee Bae, Judy Qiu, and Geoffrey Fox. Twister: A Runtime for Iterative MapReduce. In Proceedings of HPDC, pages 810-818, New York, NY, USA, 2010. ACM.
10. Dan Gillick, Arlo Faria, and John Denero. MapReduce: Distributed Computing for Machine Learning. 2008.
11. T. Gunarathne, Tak-Lon Wu, Judy Qiu, and Geoffrey C. Fox. MapReduce in the Clouds for Science. Indianapolis, IN, 11/2010 2010. IEEE.
12. Chou Jerry, Wu Kesheng, and Prabhat. FastQuery: A Parallel Indexing System for Scientific Data. In CLUSTER, pages 455-464. IEEE, 2011.
13. Wei Jiang and Gagan Agrawal. Ex-MATE: Data Intensive Computing with Large Reduction Objects and Its Application to Graph Mining. In Proceedings of CCGRID, pages 475-484, 2011.
14. Wei Jiang, Vignesh T. Ravi, and Gagan Agrawal. A Map-Reduce System with an Alternate API for Multi-core Environments. In Proceedings of CCGRID, pages 84-93, 2010.
15. Sarah Loebman, Dylan Nunley, YongChul Kwon, Bill Howe, Magdalena Balazinska, and Jeffrey P Gardner. Analyzing Massive Astrophysical Datasets: Can Pig/Hadoop or a Relational DBMS Help? pages 1-10. IEEE, 2009.
16. Yinian Qi and Mikhail J. Atallah. Efficient Privacy-Preserving k-Nearest Neighbor Search. In Proceedings of the 2008 The 28th International Conference on Distributed Computing Systems, pages 311-319, 2008.
17. Saba Sehrish, Grant Mackey, Jun Wang, and John Bent. M-RAP: A Novel MapReduce-based Framework to Support HPC Analytics Applications with Access Patterns. In Proceedings of HPDC, pages 107-118, 2010.
18. Wittawat Tantisiriroj, Seung Woo Son, Swapnil Patil, Samuel J. Lang, Garth Gibson, and Robert B. Ross. On the Duality of Data-Intensive File System Design: Reconciling HDFS and PVFS. In Proceedings of SC, pages 67:1-67:12. ACM, 2011.
19. Daniel L. Wang, Charles S. Zender, and Stephen F. Jenks. Clustered Workflow Execution of Retargeted Data Analysis Scripts. 2008.
20. Jianwu Wang, Daniel Crawl, and Ilkay Altintas. Kepler + Hadoop: A General Architecture Facilitating Data-Intensive Applications in Scientific Workflow Systems. In SC-WORKS'09, pages-1-1, 2009.
21. Hui Zhao, SiYun Ai, ZhenHua Lv, and Bo Li. Parallel Accessing Massive NetCDF Data Based on MapReduce. In Proceedings of the 2010 International Conference on Web Information Systems and Mining, WISM'10, pages 425-431, Berlin, Heidelberg, 2010. Springer-Verlag.