Extrema propagation: Fast distributed estimation of sums and network sizes

IEEE Transactions on Parallel and Distributed Systems

Volumn 23, Issue 4, 2012, Pages 668-675

  Baquero, Carlos ^a   Almeida, Paulo Sérgio ^a   Menezes, Raquel ^a   Jesus, Paulo ^a  

^a UNIVERSITY OF MINHO   (Portugal)

Author keywords

Aggregation; distributed sums; network size estimation; probabilistic estimation; self configuration

Indexed keywords

AVERAGE SYSTEM; CENTRALIZED ALGORITHMS; DATA VALUES; DISTRIBUTED COMPUTATIONS; DISTRIBUTED ESTIMATION; DISTRIBUTED HASH TABLE; DISTRIBUTED SUMS; GLOBAL IDENTIFIERS; GLOBAL SYSTEMS; MESSAGE EXCHANGE; MESSAGE LOSS; MULTI PATH ROUTING; NETWORK SIZE; NETWORK SIZE ESTIMATION; PEER TO PEER; POSITIVE REAL; PROBABILISTIC ESTIMATION; PROBABILISTIC TECHNIQUE; SELF CONFIGURATION; SELF-ADAPTIVE; SINGLE POINT; THEORETICAL MINIMUM;

AGGLOMERATION; ALGORITHMS; ESTIMATION; PEER TO PEER NETWORKS; SENSOR NETWORKS;

DISTRIBUTED COMPUTER SYSTEMS;

EID: 84858071468     PISSN: 10459219     EISSN: None     Source Type: Journal    
DOI: 10.1109/TPDS.2011.209     Document Type: Article

References (38)

1. R. van Renesse, "The Importance of Aggregation," Proc. Future Directions in Distributed Computing, pp. 87-92, 2003. (Pubitemid 137629085)
2. M. Jelasity, A. Montresor, and Ö. Babaoglu, "Gossip-Based Aggregation in Large Dynamic Networks," ACM Trans. Computer System, vol. 23, no. 3, pp. 219-252, 2005.
3. D. Kempe, A. Dobra, and J. Gehrke, "Gossip-Based Computation of Aggregate Information," Proc. IEEE 44th Ann. Symp. Foundations of Computer Science (FOCS), pp. 482-491, 2003.
4. I. Stoica, R. Morris, D.R. Karger, M.F. Kaashoek, and H. Balakrishnan, "Chord: A Scalable Peer-to-Peer Lookup Service for Internet Applications," Proc. SIGCOMM, pp. 149-160, 2001. (Pubitemid 32981961)
5. I. Abraham and D. Malkhi, "Probabilistic Quorums for Dynamic Systems," Proc. 17th Int'l Symp. Distributed Computing, pp. 60-74, 2003. (Pubitemid 37273741)
6. S. Madden, M.J. Franklin, J.M. Hellerstein, and W. Hong, "Tag: A Tiny Aggregation Service for Ad-Hoc Sensor Networks," Proc. Fifth Symp. Operating Systems Design and Implementation (OSDI), 2002.
7. J. Li, K.R. Sollins, and D.-Y. Lim, "Implementing Aggregation and Broadcast over Distributed Hash Tables," Computer Comm. Rev., vol. 35, no. 1, pp. 81-92, 2004.
8. S. Nath, P.B. Gibbons, S. Seshan, and Z.R. Anderson, "Synopsis Diffusion for Robust Aggregation in Sensor Networks," Proc. Second Int'l Conf. Embedded Networked Sensor Systems (SenSys), pp. 250-262, 2004. (Pubitemid 41563364)
9. C. Baquero, P.S. Almeida, and R. Menezes, "Fast Estimation of Aggregates in Unstructured Networks," Proc. Fifth Int'l Conf. Autonomic and Autonomous Systems (ICAS), pp. 88-93, http://doi. ieeecomputersociety.org/10. 1109/ICAS.2009.31, 2009.
10. D. Psaltoulis, D. Kostoulas, I. Gupta, K. Birman, and A. Demers, "Practical Algorithms for Size Estimation in Large and Dynamic Groups," technical report, Univ. of Illinois, http://www.cs. cornell.edu/Info/Projects/Spinglass/Pubs.html, 2004.
11. E.J. Gumbel, Statistics of Extremes. Columbia Univ. Press, 1958.
12. R.V. Hogg and A.F. Craig, Introduction to Mathematical Statistics, fifth ed. Prentice-Hall, 1995.
13. S. Madden, M. Franklin, J. Hellerstein, and W. Hong, "TAG: A Tiny AGgregation Service for Ad-Hoc Sensor Networks," ACM SIGOPS Operating Systems Rev., vol. 36, no. SI, pp. 131-146, Dec. 2002.
14. S. Madden, R. Szewczyk, M. Franklin, and D. Culler, "Supporting Aggregate Queries over Ad-Hoc Wireless Sensor Networks," Proc. IEEE Fourth Workshop Mobile Computing Systems and Applications, pp. 49-58, Mar. 2002.
15. Y. Birk, I. Keidar, L. Liss, A. Schuster, and R. Wolff, "Veracity Radius: Capturing the Locality of Distributed Computations," Proc. 25th Ann. ACM Symp. Principles of Distributed Computing (PODC), July 2006.
16. Y. Birk, I. Keidar, L. Liss, and A. Schuster, "Efficient Dynamic Aggregation," Proc. 20th Int'l Symp. DIStributed Computing (DISC), pp. 90-104, Sept. 2006. (Pubitemid 44849704)
17. K. Horowitz and D. Malkhi, "Estimating Network Size from Local Information," Information Processing Letters, vol. 88, no. 5, pp. 237-243, 2003.
18. T. Shafaat, A. Ghodsi, and S. Haridi, "A Practical Approach to Network Size Estimation for Structured Overlays," Proc. Third Int'l Self-Organizing Systems, pp. 71-83, Dec. 2008.
19. P. Jesus, C. Baquero, and P.S. Almeida, "Dependability in Aggregation by Averaging," Simpósio de Informatica (INForum), Sept. 2009.
20. E.L. Merrer, A.-M. Kermarrec, and L. Massoulie, "Peer to Peer Size Estimation in Large and Dynamic Networks: A Comparative Study," Proc. IEEE 15th Int'l Symp. High Performance Distributed Computing, Jan. 2006.
21. A. Ganesh, A. Kermarrec, E.L. Merrer, and L. Massoulié, "Peer Counting and Sampling in Overlay Networks Based on Random Walks," Distributed Computing, vol. 20, no. 4, pp. 267-278, 2007. (Pubitemid 350005917)
22. L. Massoulié, E. Merrer, A.-M. Kermarrec, and A. Ganesh, "Peer Counting and Sampling in Overlay Networks: Random Walk Methods," Proc. 25th Ann. ACM Symp. Principles of Distributed Computing (PODC), 2006.
23. S. Mane, S. Mopuru, K. Mehra, and J. Srivastava, "Network Size Estimation in a Peer-to-Peer Network," technical report, Dept. of Computer Science, Univ. of Minnesota, p. 12, Sept. 2005.
24. D. Kostoulas, D. Psaltoulis, I. Gupta, K. Birman, and A. Demers, "Decentralized Schemes for Size Estimation in Large and Dynamic Groups," Proc. IEEE Fourth Int'l Symp. Network Computing and Applications, pp. 41-48, 2005. (Pubitemid 46116738)
25. D. Kostoulas, D. Psaltoulis, I. Gupta, K.P. Birman, and A.J. Demers, "Active and Passive Techniques for Group Size Estimation in Large-Scale and Dynamic Distributed Systems," J. Systems and Software, vol. 80, no. 10, pp. 1639-1658, Jan. 2007. (Pubitemid 47238423)
26. J. Considine, F. Li, G. Kollios, and J.W. Byers, "Approximate Aggregation Techniques for Sensor Databases," Proc. 20th Int'l Conf. Data Eng. (ICDE), pp. 449-460, 2004.
27. M. Bawa, H. Garcia-Molina, A. Gionis, and R. Motwani, "Estimating Aggregates on a Peer-To-Peer Network," Technical Report TR-2003-24, Stanford Univ., http://dbpubs.stanford.edu/pub/2003-24, 2003.
28. A. Manjhi, S. Nath, and P. Gibbons, "Tributaries and Deltas: Efficient and Robust Aggregation in Sensor Network Streams," Proc. ACM SIGMOD Int'l Conf. Management of Data, pp. 287-298, 2005. (Pubitemid 43038935)
29. P. Flajolet and G.N. Martin, "Probabilistic Counting Algorithms for Data Base Applications," J. Computer and System Sciences, vol. 31, no. 2, pp. 182-209, 1985.
30. M. Durand and P. Flajolet, "Loglog Counting of Large Cardinalities (Extended Abstract)," Proc. 11th Ann. European Symp. Algorithms, pp. 605-617, 2003.
31. P. Flajolet, E. Fusy, O. Gandouet, and F. Meunier, "Hyperloglog: The Analysis of a Near-Optimal Cardinality Estimation Algorithm," Int'l Conf. Analysis of Algorithms (AofA), pp. 127-146, June 2007.
32. D. Mosk-Aoyama and D. Shah, "Computing Separable Functions via Gossip," Proc. 25th Ann. ACM Symp. Principles of Distributed Computing, pp. 113-122, July 2006. (Pubitemid 44393699)
33. E. Cohen, "Size-Estimation Framework with Applications to Transitive Closure and Reachability," J. Computer and System Sciences, vol. 55, no. 3, pp. 441-453, 1997. (Pubitemid 127432363)
34. D. Kempe, A. Dobra, and J. Gehrke, "Gossip-Based Computation of Aggregate Information," Proc. IEEE 44th Ann. Symp. Foundations of Computer Science, pp. 482-491, 2003.
35. M. Jelasity and A. Montresor, "Epidemic-Style Proactive Aggregation in Large Overlay Networks," Proc. 24th Int'l Conf. Distributed Computing Systems, pp. 102-109, Jan. 2004.
36. F. Wuhib, M. Dam, R. Stadler, and A. Clemm, "Robust Monitoring of Network-Wide Aggregates through Gossiping," Proc. IFIP/IEEE 10th Int'l Symp. Integrated Network Management, pp. 226-235, May 2007. (Pubitemid 47485176)
37. P. Jesus, C. Baquero, and P.S. Almeida, "Fault-Tolerant Aggregation by Flow Updating," Proc. Ninth IFIP Int'l Conf. Distributed Applications and Interoperable Systems (DAIS), pp. 73-86, 2009.
38. P. Jesus, C. Baquero, and P.S. Almeida, "Fault-Tolerant Aggregation for Dynamic Networks," Proc. IEEE 29th Symp. Reliable Distributed Systems, pp. 37-43, 2010.