Passive network performance estimation for large-scale, data-intensive computing

IEEE Transactions on Parallel and Distributed Systems

Volumn 22, Issue 8, 2011, Pages 1365-1373

  Kim, Jinoh ^a   Chandra, Abhishek ^a   Weissman, Jon B ^a  

^a University of Minnesota   (United States)

Author keywords

data intensive computing; Network performance estimation; replica selection; resource selection; secondhand estimation

Indexed keywords

DATA-INTENSIVE COMPUTING; NETWORK PERFORMANCE ESTIMATION; REPLICA SELECTION; RESOURCE SELECTION; SECONDHAND ESTIMATION;

ESTIMATION; NETWORK PERFORMANCE; PASSIVE NETWORKS; TOPOLOGY;

DISTRIBUTED COMPUTER SYSTEMS;

EID: 79959696997     PISSN: 10459219     EISSN: None     Source Type: Journal    
DOI: 10.1109/TPDS.2010.201     Document Type: Article

References (34)

1. "LHC: Large Hadron Collider," http://lhc.web.cern.ch/lhc/, 2011.
2. "Sloan Digital Sky Survey/Skyserver," http://cas.sdss.org/dr7/ en/, 2011.
3. S. Bharathi, A. Chervenak, E. Deelman, G. Mehta, and M.-H. Su, "Characterization of Scientific Workflows," Proc. Third Workshop Workflows in Support of Large-Scale Science (WORKS '08), 2008.
4. E. Deelman and A. Chervenak, "Data Management Challenges of Data-Intensive Scientific Workflows," Proc. Eighth IEEE Int'l Symp. Cluster Computing and the Grid (CCGRID '08), pp. 687- 692, 2008.
5. D. Thain, T. Tannenbaum, and M. Livny, "Distributed Computing in Practice: The Condor Experience," Concurrency - Practice and Experience, vol. 17, nos. 2-4, pp. 323-356, 2005. (Pubitemid 40285781)
6. "The Globus Alliance," http://www.globus.org/, 2011.
7. D. Kondo, A.A. Chien, and H. Casanova, "Resource Management for Rapid Application Turnaround on Enterprise Desktop Grids," Proc. ACM/IEEE Conf. Supercomputing (SC '04), 2004.
8. V. Lo, D. Zappala, D. Zhou, Y. Liu, and S. Zhao, "Cluster Computing on the Fly: P2p Scheduling of Idle Cycles in the Internet," Proc. IEEE Fourth Int'l Conf. Peer-to-Peer Systems, pp. 227- 236, 2004. (Pubitemid 41190554)
9. D.P. Anderson and G. Fedak, "The Computational and Storage Potential of Volunteer Computing," Proc. Sixth IEEE Int'l Symp. Cluster Computing and the Grid (CCGRID '06), pp. 73-80, 2006. (Pubitemid 44761684)
10. T.S.E. Ng, Y. hua Chu, S.G. Rao, K. Sripanidkulchai, and H. Zhang, "Measurement-Based Optimization Techniques for Bandwidth- Demanding Peer-to-Peer Systems," Proc. IEEE INFOCOM, pp. 2199-2209, 2003.
11. S. Seshan, M. Stemm, and R.H. Katz, "SPAND: Shared Passive Network Performance Discovery," Proc. USENIX Symp. Internet Technologies and Systems, pp. 135-146, Dec. 1997.
12. M. Andrews, B. Shepherd, A. Srinivasan, P. Winkler, and F. Zane, "Clustering and Server Selection Using Passive Monitoring," Proc. IEEE INFOCOM, pp. 1717-1725, 2002.
13. J. Kim, A. Chandra, and J.B. Weissman, "Using Data Accessibility for Resource Selection in Large-Scale Distributed Systems," IEEE Trans. Parallel and Distributed Systems, vol. 20, no. 6, pp. 788-801, June 2009.
14. "PlanetLab," http://www.planet-lab.org, 2011.
15. N. Kourtellis, L. Prieto, A. Iamnitchi, G. Zarrate, and D. Fraser, "Data Transfers in the Grid: Workload Analysis of Globus Gridftp," Proc. Int'l Workshop Data-Aware Distributed Computing (DADC '08), pp. 29-38, 2008.
16. P. Yalagandula, P. Sharma, S. Banerjee, S. Basu, and S.-J. Lee, "S3: A Scalable Sensing Service for Monitoring Large Networked Systems," Proc. SIGCOMM Workshop Internet Network Management (INM '06), pp. 71-76, 2006. (Pubitemid 46719058)
17. G.B. Berriman, A.C. Laity, J.C. Good, J.C. Jacob, D.S. Katz, E. Deelman, G. Singh, M.-H. Su, and T.A. Prince, "Montage: The Architecture and Scientific Applications of a National Virtual Observatory Service for Computing Astronomical Image Mosaics," Proc. Earth Sciences Technology Conf., 2006.
18. A.B. Downey, "Using Pathchar to Estimate Internet Link Characteristics," Proc. ACM SIGCOMM '99, pp. 241-250, 1999. (Pubitemid 32081869)
19. R. Wolski, N. Spring, and J. Hayes, "The Network Weather Service: A Distributed Resource Performance Forecasting Service for Metacomputing," J. Future Generation Computing Systems, vol. 15, pp. 757-768, 1999.
20. S. Keshav, "Packet-Pair Flow Control," IEEE/ACM Trans. Networking, 1995.
21. Q. He, C. Dovrolis, and M. Ammar, "On the Predictability of Large Transfer Tcp Throughput," Proc. ACM SIGCOMM '05, pp. 145-156, 2005. (Pubitemid 46323501)
22. R.L. Carter and M. Crovella, "Server Selection Using Dynamic Path Characterization in Wide-Area Networks," Proc. IEEE INFOCOM, pp. 1014-1021, 1997.
23. A.-M. Kermarrec, L. Massoulié, and A.J. Ganesh, "Probabilistic Reliable Dissemination in Large-Scale Systems," IEEE Trans. Parallel and Distributed Systems, vol. 14, no. 3, pp. 248-258, Mar. 2003.
24. S. Voulgaris and M. van Steen, "Hybrid Dissemination: Adding Determinism to Probabilistic Multicasting in Large-Scale p2p Systems," Proc. ACM/IFIP/USENIX Int'l Conf. Middleware, pp. 389- 409, 2007.
25. M. Deshpande, B. Xing, I. Lazardis, B. Hore, N. Venkatasubramanian, and S. Mehrotra, "Crew: A Gossip-Based Flash-Dissemination System," Proc. 26th IEEE Int'l Conf. Distributed Computing Systems (ICDCS '06), p. 45, 2006.
26. Z.J. Haas, J.Y. Halpern, and L. Li, "Gossip-Based Ad Hoc Routing," IEEE/ACM Trans. Networking, vol. 14, no. 3, pp. 479- 491, June 2006. (Pubitemid 43985040)
27. P. Kyasanur, R. Choudhury, and I. Gupta, "Smart Gossip: An Adaptive Gossip-Based Broadcasting Service for Sensor Networks," Proc. IEEE Int'l Conf. Mobile Adhoc and Sensor Systems, pp. 91-100, 2006.
28. J. Strauss, D. Katabi, and F. Kaashoek, "A Measurement Study of Available Bandwidth Estimation Tools," Proc. Third ACM SIGCOMM Conf. Internet Measurement (IMC '03), pp. 39-44, 2003. (Pubitemid 40730637)
29. R. Wolski, "Experiences with Predicting Resource Performance Online in Computational Grid Settings," SIGMETRICS Performance Evaluation Rev., vol. 30, no. 4, pp. 41-49, 2003.
30. H.V. Madhyastha, T. Isdal, M. Piatek, C. Dixon, T. Anderson, A. Krishnamurthy, and A. Venkataramani, "iPlane: An Information Plane for Distributed Services," Proc. Seventh USENIX Symp. Operating Systems Design and Implementation (OSDI '06), 2006.
31. H.V. Madhyastha, E. Katz-Bassett, T. Anderson, A. Krishnamurthy, and A. Venkataramani, "Iplane Nano: Path Prediction for Peer-to-Peer Applications," Proc. Sixth USENIX Symp. Networked Systems Design and Implementation (NSDI '09), pp. 137-152, 2009.
32. I. Gupta, A.-M. Kermarrec, and A.J. Ganesh, "Efficient and Adaptive Epidemic-Style Protocols for Reliable and Scalable Multicast," IEEE Trans. Parallel and Distributed Systems, vol. 17, no. 7, pp. 593-605, July 2006. (Pubitemid 43997183)
33. M. jang Lin and K. Marzullo, "Directional Gossip: Gossip in a Wide Area Network," Proc. European Dependable Computing Conf., pp. 364-379, 1999.
34. "perfsonar," http://www.perfSONAR.net/, 2011.