Uncovering global icebergs in distributed streams: Results and implications

Journal of Network and Systems Management

Volumn 19, Issue 1, 2011, Pages 84-110

  Huang, Guanyao ^a   Lall, Ashwin ^b   Chuah, Chen Nee ^a   Xu, Jun ^b  

^a University of California   (United States)

^b Georgia Institute of Technology   (United States)

Author keywords

Iceberg; Sampling; Sketch

Indexed keywords

DISTRIBUTED PROCESSING; DISTRIBUTED STREAMING; FUTURE DESIGNS; ICEBERG; INTERNET TRACES; MULTIPLE STREAMS; QOS METRICS; SKETCH; STREAMING ALGORITHM;

NETWORK MANAGEMENT; TRACE ANALYSIS;

SEA ICE;

EID: 79751532615     PISSN: 10647570     EISSN: None     Source Type: Journal    
DOI: 10.1007/s10922-010-9186-5     Document Type: Article

References (26)

1. P.E. Ayres H. Sun H.J. Chao W.C. Lau 2006 ALPi: A DDoS defense system for high-speed networks IEEE J. Sel. Areas Commun. 24 10 1864 1876 10.1109/JSAC.2006.877136 (Pubitemid 44559600)
2. Akamai Technologies Inc. (2004). http://www.akamai.com/
3. Cheetancheri, S.G.; Agosta, J.M.; Dash, D.H.; Levitt, K.N.; Rowe, J.; Schooler, E.M.: A distributed host-based worm detection system. In: Proceedings of the 2006 SIGCOMM Workshop on Large-Scale Attack Defense (LSAD) (2006)
4. Sommers, J.; Barford, P.; Duffield, N.; Ron, A.: Accurate and efficient sla compliance monitoring. In: Proceedings of the 2007 Conference on Applications, Technologies, Architectures, and Protocols for Computer Communications (SIGCOMM) (2007)
5. Cisco Netflow (2007). http://www.cisco.com/
6. G. Cormode S. Muthukrishnan 2005 An improved data stream summary: the count-min sketch and its applications J. Algorithms 55 1 58 75 1068.68048 10.1016/j.jalgor.2003.12.001 2132028 (Pubitemid 40357145)
7. Estan, C.; Varghese, G.: New directions in traffic measurement and accounting. In: Proceedings of the ACM SIGCOMM (2002)
8. M. Charikar K. Chen M. Farach-Colton 2004 Finding frequent items in data streams Theor. Comput. Sci. 312 1 3 15 1071.68020 10.1016/S0304-3975(03)00400-6 2045483
9. Arasu, A.; Manku, G.S.: Approximate counts and quantiles over sliding windows. In: Proceedings of the ACM Symposium on Principles of Database Systems (PODS) (2004)
10. Raspall, F.; Sallent, S.; Yufera, J.: Shared-state sampling. In: IMC '06: Proceedings of the 6th ACM SIGCOMM Conference on Internet Measurement, pp. 1-14. ACM, New York, NY (2006) (Pubitemid 47165584)
11. Karp, R.M.; Shenker, S.; Papadimitriou, C.H.: A simple algorithm for finding frequent elements in streams and bags. ACM Trans. Database Syst. 28(1), 51-55 (2003)
12. Huang, G.; Lall, A.; Chuah, C.-N.; Xu, J.: Uncovering global icebergs in distributed monitors. In: Proceedings of the IEEE IWQoS (2009)
13. B. Babcock S. Babu M. Datar R. Motwani D. Thomas 2004 Operator scheduling in data stream systems VLDB J. 13 4 333 353 10.1007/s00778-004-0132-6 (Pubitemid 39653741)
14. Feigenbaum, J.; Kannan, S.; Strauss, M.; Viswanathan, M.: Streaming algorithms for distributed, massive data sets. In: Proceedings of the IEEE FOCS (1999)
15. Cherniack, M.; Balakrishnan, H.; Balazinska, M.; Carney, D.; Cetintemel, U.; Xing, Y.; Zdonik, S.: Scalable distributed stream processing. In: Proceedings of the 2003 CIDR Conference (2003)
16. Cormode, G.; Muthukrishnan, S.; Yi, K.: Algorithms for distributed functional monitoring. In: Proceedings of the 19th Annual ACM-SIAM Symposium on Discrete Algorithms (SODA) (2008)
17. Manjhi, A.; Shkapenyuk, V.; Dhamdhere, K.; Olston, C.: Finding (recently) frequent items in distributed data streams. In: Proceedings of the IEEE ICDE (2005)
18. Babcock, B.; Olston, C.: Distributed top-k monitoring. In: Proceedings of the ACM International Conference on Management of Data (SIGMOD) (2003)
19. Olston, C.; Jiang, J.; Widom, Jennifer: Adaptive filters for continuous queries over distributed data streams. In: Proceedings of the ACM International Conference on Management of Data (SIGMOD) (2003)
20. Zhao, Q.; Ogihara, M.; Wang, H.; Xu, J.: Finding global icebergs over distributed data sets. In: Proceedings of the ACM Symposium on Principles of Database Systems (PODS) (2006)
21. Cohen, E.; Duffield, N.; Kaplan, H.; Lund, C.; Thorup, M.: Sketching unaggregated data streams for subpopulation-size queries. In: PODS '07: Proceedings of the Twenty-Sixth ACM SIGMOD-SIGACT-SIGART Symposium on Principles of Database Systems, pp. 253-262. ACM, New York, NY (2007) (Pubitemid 47620900)
22. G. Cormode M. Hadjieleftheriou 2008 Finding frequent items in data streams PVLDB 1 2 1530 1541
23. Kumar, A.; Xu, J.: Sketch guided sampling-using on-line estimates of flow size for adaptive data collection. In: Proceedings of the IEEE INFOCOM (2006)
24. Huang, Guanyao, Lall, Ashwin, Chuah, ChenNee, Xu, Jun: Uncovering global icebergs in distributed streams. Technical Report ECE-CE-2009-1, UCDavis (2009)
25. Internet2 Abilene Network (2005). http://abilene.internet2.edu/
26. Loiseau, P.; Gonçalves, P.; Girard, S.; Forbes, F.; Primet, P.V.-B.: Maximum likelihood estimation of the flow size distribution tail index from sampled packet data. In: SIGMETRICS '09: Proceedings of the Eleventh International Joint Conference on Measurement and Modeling of Computer Systems, pp. 263-274 (2009)