Case study of scientific data processing on a cloud using Hadoop

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Volumn 5976 LNCS, Issue , 2010, Pages 400-415

  Zhang, Chen ^a   De Sterck, Hans ^b   Aboulnaga, Ashraf ^a   Djambazian, Haig ^c   Sladek, Rob ^c  

^a UNIVERSITY OF WATERLOO   (Canada)

^b UNIVERSITY OF WATERLOO   (Canada)

^c MCGILL UNIVERSITY   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

CLOUD COMPUTING; IN-FIELD; INPUT FORMAT; LARGE SCALE DATA; LIVE CELL; MICROSCOPE IMAGES; OPEN SOURCES; REAL-WORLD APPLICATION; SCIENTIFIC DATA; SCIENTIFIC DATA PROCESSING; WEB INDEXING;

INDEXING (MATERIALS WORKING);

DATA PROCESSING;

EID: 77952400308     PISSN: 03029743     EISSN: 16113349     Source Type: Book Series    
DOI: 10.1007/978-3-642-12659-8_29     Document Type: Conference Paper

References (30)

1. Aguilera, M.K., Merchant, A., Shah, M.A., Veitch, A.C., Karamanolis, C.T.: Sinfonia: A New Paradigm for Building Scalable Distributed Systems. In: SOSP 2007 (2007)
2. Aguilera, M., Golab, W., Shah, M.: A Practical Scalable Distributed B-Tree. In: VLDB 2008 (2008)
3. Amazon Elastic Compute Cloud, http://aws.amazon.com/ec2/ (retrieved date: September 27, 2009)
4. Apache Hadoop, http://hadoop.apache.org/ (retrieved date: September 27, 2009)
5. Apache HBase, http://hadoop.apache.org/hbase/ (retrieved date: September 27, 2009)
6. Apache Hama, http://incubator.apache.org/hama/ (retrieved date: September 27, 2000)
7. Barham, P., Dragovic, B., Fraser, K., Hand, S., Harris, T., Ho, A., Neugebauery, R., Pratt, I., Warfield, A.: Xen and the Art of Virtualization. In: SOSP 2003 (2003)
8. Brantner, M., Florescu, D., Graf, D.A., Kossmann, D., Kraska, T.: Building a Database on S3. In: SIGMOD 2008 (2008)
9. Catanzaro, B., Sundaram, N., Keutzer, K.: A MapReduce framework for programming graphics processors. In: Workshop on Software Tools for MultiCore Systems (2008)
10. Chaiken, R., Jenkins, B., Larson, P., Ramsey, B., Shakib, D., Weaver, S., Zhou, J.: SCOPE: Easy and Efficient Parallel Processing of Massive Data Sets. In: VLDB 2008 (2008)
11. Chang, F., Dean, J., Ghemawat, S., Hsieh, W.C., Wallach, D.A., Burrows, M., Chandra, T., Fikes, A., Gruber, R.: BigTable: A Distributed Storage System for Structured Data. In: OSDI 2006 (2006)
12. Cooper, B., Ramakrishnan, R., Srivastava, U., Silberstein, A., Bohannon, P., Jacobsen, H.-A., Puz, N., Weaver, D., Yerneni, R.: PNUTS: Yahoo!'s Hosted Data Serving Platform. In: VLDB 2008 (2008)
13. Dean, J., Ghemawat, S.: MapReduce: Simplified Data Processing on Large Clusters. In: OSDI 2004 (2004)
14. DeCandia, G., Hastorun, D., Jampani, M., Kakulapati, G., Lakshman, A., Pilchin, A., Sivasubramanian, S., Vosshall, P., Vogels, W.: Dynamo: Amazon's Highly Available Key-Value Store. In: SOSP 2007 (2007)
15. DeWitt, D.J., Robinson, E., Shankar, S., Paulson, E., Naughton, J., Krioukov, A., Royalty, J.: Clustera: An Integrated Computation and Data Management System. In: VLDB 2008 (2008)
16. ELASTRA, http://www.elastra.com/ (retrieved date: Sepember 27, 2009)
17. Elsayed, T., Lin, J., Oard, D.: Pairwise Document Similarity in Large Collections with MapReduce. In: Proc. Annual Meeting of the Association for Computational Linguistics (2008)
18. Ghemawat, S., Gobioff, H., Leung, S.-T.: The Google File System. In: SOSP 2003 (2003)
19. GigaSpaces, http://www.gigaspaces.com/ (retrieved date: September 27, 2009)
20. Google and IBM Announce University Initiative, http://www.ibm.com/ibm/ ideasfromibm/us/google/index.shtml (retrieved date: September 27, 2009)
21. Irwin, D.E., Chase, J.S., Grit, L.E., Yumerefendi, A.R., Becker, D., Yocum, K.: Sharing Networked Resources with Brokered Leases. In: USENIX Annual Conference 2006 (2006)
22. Isard, M., Budiu, M., Yu, Y., Birrell, A., Fetterly, D.: Dryad: Distributed Data-parallel Programs from Sequential Building Blocks. In: EuroSys 2007 (2007)
23. McNabb, A.W., Monson, C.K., Seppi, K.D.: MRPSO: MapReduce Particle Swarm Optimization. In: Genetic and Evolutionary Computation Conference (2007)
24. Olston, C., Reed, B., Srivastava, U., Kumar, R., Tomkins, A.: Pig Latin: A Not-So-Foreign Language for Data Processing. In: SIGMOD 2008 (2008)
25. Pike, R., Dorward, S., Griesemer, R., Quinlan, S.: Interpreting the Data: Parallel Analysis with Sawzall. Scientific Programming 13(4) (2005)
26. Ramakrishnan, L., Irwin, D.E., Grit, L.E., Yumerefendi, A.R., Iamnitchi, A., Chase, J.S.: Toward a Doctrine of Containment: Grid Hosting with Adaptive Resource Control. In: SC 2006 (2006)
27. Scalable Scientific Computing Group, University of Waterloo, http://www.math.uwaterloo.ca/groups/SSC/software/cloud (retrieved date: September 27, 2009)
28. Soror, A., Minhas, U.F., Aboulnaga, A., Salem, K., Kokosielis, P., Kamath, S.: Automatic Virtual Machine Configuration for Database Workloads. In: SIGMOD 2008 (2008)
29. Yang, H.C., Dasdan, A., Hsiao, R.-L., Parker, D.S.: Map-reduce-merge: simplified relational data processing on large clusters. In: SIGMOD 2007 (2007)
30. Zhang, C., De Sterck, H.: CloudWF: A Computational Work ow System for Clouds Based on Hadoop. In: The First International Conference on Cloud Computing, Beijing, China (2009)