Volunteer cloud computing: MapReduce over the internet

IEEE International Symposium on Parallel and Distributed Processing Workshops and Phd Forum

Volumn , Issue , 2011, Pages 1855-1862

  Costa, Fernando ^a   Silva, Luis ^a   Dahlin, Michael ^b  

^a UNIVERSITY OF COIMBRA   (Portugal)

^b UNIVERSITY OF TEXAS AT AUSTIN   (United States)

Author keywords

Cloud computing; MapReduce; Volunteer computing

Indexed keywords

COMPLEX APPLICATIONS; COMPUTATIONAL DEVICES; COMPUTATIONAL POWER; COMPUTING RESOURCE; DISTRIBUTED RESOURCES; EXISTING SYSTEMS; INDEPENDENT TASKS; MAP-REDUCE; MASTER/WORKER; PROGRAMMING PARADIGMS; PROOF OF CONCEPT; SMALL CLUSTERS; VOLUNTEER COMPUTING;

CLUSTER COMPUTING; COMPUTER SYSTEMS; DISTRIBUTED PARAMETER NETWORKS; INTERNET; MIDDLEWARE;

CLOUD COMPUTING;

EID: 83455181355     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/IPDPS.2011.345     Document Type: Conference Paper

References (30)

1. F. Costa, I. Kelley, L. Silva, and G. Fedak, "Optimizing Data Distribution in Desktop Grid Platforms", Parallel Processing Letters, vol. 18, # 3, September 2008.
2. J. Dean and S. Ghemawat, "MapReduce: Simplified Data Processing on Large Clusters," in Proceedings of OSDI '04: 6th Symposium on Operating System Design and Implementation, San Francisco, 2004.
3. Apache Hadoop. See website at: http://hadoop.apache.org/
4. M. Isard, M. Budiu, Y. Yu, A. Birrell, and D. Fetterly, "Dryad: Distributed data-parallel programs from sequential building blocks", in Proceedings of the 2nd ACM SIGOPS/EuroSys European Conference on Computer Systems 2007 (EuroSys '07), March 2007.
5. Folding@home Project. See website at: http://folding.stanford.edu/
6. Top 500. See website at: http://www.top500.org/
7. D. P. Anderson, J. Cobb, E. Korpela, M. Lebofsky, and D. Werthimer, "SETI@home: An experiment in public-resource computing", Communications of the ACM, Vol. 45 No. 11, pp. 56-61, Nov. 2002.
8. David Anderson, "BOINC: A System for Public-Resource Computing and Storage", in Proceedings of the 5th IEEE/ACM International Workshop on Grid Computing, Pittsburgh, USA, November 2004.
9. F. Cappello, S. Djilali, G. Fedak, T. Herault. F. Magniette, V. Neri, and O. Lodygensky, "Computing on large-scale distributed systems: XtremWeb architecture, programming models, security, tests and convergence with Grid", FGCS Future Generation Computer Science, 2004.
10. MilkyWay@home. See website at: http://milkyway.cs.rpi.edu/milkyway/
11. ClimatePrediction.net See website at: http://climateprediction.net/
12. David Anderson, "BOINC: The Year in Review", presented at the 6th Annual Pangalactic BOINC Workshop, London 2010.
13. Sanjay Ghemawat, Howard Gobioff, and Shun-Tak Leung, "The Google file system", in 19th Symposium on Operating Systems Principles, pp. 29-43, Lake George, New York, 2003.
14. J. Brenton, C. Caldas, J. Davies, S. Harris, and P. Maccallum, "CancerGrid: developing open standards for clinical cancer informatics", in Proceedings of the UK e-science All Hands Meeting, pp. 678-681, 2005.
15. D. Borthakur, "The Hadoop Distributed File System: Architecture and Design". http://hadoop.apache.org/hdfs/docs/r0.21.0/hdfs-design.html
16. List of institutions using Hadoop: http://wiki.apache.org/hadoop/PoweredB
17. A. Venkataramani, R. Kokku, and M. Dahlin, "TCP Nice: A mechanism for background transfers", in Proc. of OSDI, Boston, MA, Dec. 2002.
18. B. Ford, P. Srisuresh, and D. Kegel, "Peer-to-peer communication across network address translators", in Proceedings of the 2005 USENIX Annual Technical Conference, Anaheim, CA, Apr. 2005.
19. J. Rosenberg, J. Weinberger, C. Huitema, and R. Mahy, "STUN: Simple Traversal of User Datagram Protocol (UDP) Through Network Address Translators (NATs)", RFC 3489, IETF, Mar. 2003.
20. J. Rosenberg, R. Mahy, and C. Huitema, "Traversal Using Relay NAT (TURN)", IETF Internet Draft, February 2005.
21. S. Guha, N. Daswani, and R. Jain, "An experimental study of the Skype peer-to-peer VoIP system," in IPTPS, 2006.
22. M. Castro, M. Costa, and A. Rowstron, "A. Debunking some myths about structured and unstructured overlays", in Proceedings of the NSDI '05, Boston, MA, May 2005.
23. J. Liang, R. Kumar, K. W. Ross, "The KaZaA Overlay: A Measurement Study", Computer Networks Journal, Oct. 2005.
24. M. Ripeanu, I. Foster, and A. Iamnitchi, "Mapping the gnutella network: Properties of large-scale peer-to-peer systems and implications for system design," IEEE Internet Computing Journal, vol. 6, no. 1, 2002.
25. μtorrent Micro Transport Protocol (μTP). See website at: http://www.utorrent.com/documentation/utp
26. University of Texas' Emulab. See web site at: https://boss.cias.utexas. edu
27. Vincenzo D. Cunsolo, Salvatore Distefano, Antonio Puliafito and Marco Scarpa, "Cloud@Home: Bridging the Gap between Volunteer and Cloud Computing", in ICIC'09 Proceedings of the 5th international conference on Emerging intelligent computing technology and applications, 2009.
28. D. Kondo, B. Javadi, P. Malecot, F. Cappello, and D. Anderson, "Costbenefit analysis of Cloud Computing versus desktop grids", in Proceedings of the 2009 IEEE international Symposium on Parallel & Distributed Processing, pp. 1-12, May 2009.
29. Heshan Lin, Xiaosong Ma, Jeremy Archuleta, Wu-chun Feng, Mark Gardner, Zhe Zhang, "MOON: MapReduce On Opportunistic eNvironments", in ACM International Symposium on High Performance Distributed Computing (HPDC '10), June 2010.
30. P. Balaji, W. Feng, and H. Lin, "Semantics-Based Distributed I/O with the ParaMEDIC Framework", in Proceedings of the IEEE/ACM International Conference on High Performance Distributed Computing (HPDC), Boston, MA, June 2008.