MapReduce for data intensive scientific analyses

Proceedings - 4th IEEE International Conference on eScience, eScience 2008

Volumn , Issue , 2008, Pages 277-284

  Ekanayake, Jaliya ^a   Pallickara, Shrideep ^a   Fox, Geoffrey ^a  

^a INDIANA UNIVERSITY   (United States)

Author keywords

MapReduce; Message passing; Parallel processing; Scientific data analysis

Indexed keywords

HIGH ENERGY PHYSICS; MESSAGE PASSING;

DATA INTENSIVES; HIGH ENERGIES; K-MEANS CLUSTERING; MAPREDUCE; PARALLEL DATUM; PARALLEL PROCESSING; PERFORMANCE REQUIREMENTS; PHYSICS DATUM; SCIENTIFIC ANALYSIS; SCIENTIFIC COMMUNITIES; SCIENTIFIC DATA ANALYSIS; TEXTUAL DATUM; VOLUMINOUS DATUM;

DATA PROCESSING;

EID: 62749127314     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/eScience.2008.59     Document Type: Conference Paper

References (21)

1. F. Darema, "SPMD model: past, present and future," Recent Advances in Parallel Virtual Machine and Message Passing Interface, 8th European PVM/MPI Users' Group Meeting, Santorini/Thera, Greece, 2001.
2. MPI (Message Passing Interface), http://www-unix.mcs.anl.gov/mpi/
3. J. Dean and S. Ghemawat, "Mapreduce: Simplified data processing on large clusters," ACM Commun., vol. 51, Jan. 2008, pp. 107-113.
4. R. Buyya, D. Abramson, and J. Giddy, "Nimrod/G: An Architecture for a Resource Management and Scheduling System in a Global Computational Grid," HPC Asia 2000, IEEE CS Press, USA, 2000.
5. J. B. MacQueen, "Some Methods for classification and Analysis of Multivariate Observations," Proceedings of 5-th Berkeley Symposium on Mathematical Statistics and Probability, Berkeley, University of California Press, vol. 1, pp. 281-297.
6. Apache Hadoop, http://hadoop.apache.org/core/
7. S. Ghemawat, H. Gobioff, and S. Leung, "The Google file system", Symposium on Op-erating Systems Principles, 2003, pp 29-43.
8. S. Pallickara and G. Fox, "NaradaBrokering: A Distributed Middleware Framework and Architecture for Enabling Durable Peer-to-Peer Grids," Middleware 2003, pp. 41-61.
9. S. Pallickara, H. Bulut, and G. Fox, "Fault-Tolerant Reliable Delivery of Messages in Distributed Publish/Subscribe Systems," 4th IEEE International Conference on Autonomic Computing, Jun. 2007, pp. 19.
10. ROOT - An Object Oriented Data Analysis Framework, http://root.cern.ch/
11. CINT - The CINT C/C++ Interpreter, http://root.cern.ch/twiki/bin/view/ ROOT/CINT/
12. C. Chu, S. Kim, Y. Lin, Y. Yu, G. Bradski, A. Ng, and K. Olukotun. Map-reduce for machine learning on multicore. In B. Scḧolkopf, J. Platt, and T. Hoffman, editors, Advances in Neural Information Processing Systems 19, pages 281-288. MIT Press, Cambridge, MA, 2007.
13. C. Ranger, R. Raghuraman, A. Penmetsa, G. R. Bradski, and C. Kozyrakis. "Evaluating MapReduce for Multi-core and Multiprocessor Systems," Proc. International Symposium on High-Performance Computer Architecture (HPCA), 2007, pp. 13-24.
14. K. Rose, E. Gurewwitz, and G. Fox, "A deterministic annealing approach to clustering," Pattern Recogn. Lett, vol. 11, no. 9, 1995, pp. 589-594.
15. PVM (Parallel Virtual Machine), http://www.csm.ornl.gov/pvm/
16. G. L. Steel, Jr. "Parallelism in Lisp," SIGPLAN Lisp Pointers, vol. VIII(2), 1995, pp. 1-14.
17. R. Pike, S. Dorward, R. Griesemer, and S. Quinlan, "Interpreting the data: Parallel analysis with sawzall," Scientific Programming Journal Special Issue on Grids and Worldwide Computing Programming Models and Infrastructure, vol. 13, no. 4, pp. 227-298, 2005.
18. M. Isard, M. Budiu, Y. Yu, A. Birrell, and D. Fetterly, "Dryad:Distributed data-parallel programs from sequential building blocks," European Conference on Computer Systems, March 2007.
19. H. Chih, A. Dasdan, R. L. Hsiao, and D. S. Parker, "Map-reducemerge: Simplified relational data processing on large clusters," Proc. ACM SIGMOD International Conference on Management of data (SIGMOD 2007), ACM Press, 2007, pp. 1029-1040.
20. Disco project, http://discoproject.org/
21. Erlang programming language, http://www.erlang.org/