Duplicate-insensitive order statistics computation over data streams

IEEE Transactions on Knowledge and Data Engineering

Volumn 22, Issue 4, 2010, Pages 493-507

  Zhang, Ying ^a   Lin, Xuemin ^a   Yuan, Yidong ^a   Kitsuregawa, Masaru ^b   Zhou, Xiaofang ^c   Yu, Jeffrey Wu ^d  

^a UNIVERSITY OF NEW SOUTH WALES   (Australia)

^b UNIVERSITY OF TOKYO   (Japan)

^c UNIVERSITY OF QUEENSLAND   (Australia)

^d CHINESE UNIVERSITY OF HONG KONG   (Hong Kong)

Author keywords

Data stream; Duplicate insensitive; Order statistic; Relative error

Indexed keywords

COMPREHENSIVE PERFORMANCE; COMPUTATION PROBLEMS; DATA STREAM; DISTINCT ELEMENTS; DISTRIBUTED COMMUNICATIONS; FAULT-TOLERANT; HEAVY-HITTER; HIGH-SPEED DATA; OBSERVED DATA; ORDER STATISTICS; REAL-TIME COMPUTATIONS; RELATIVE ERRORS; SPACE EFFICIENCIES;

ALGORITHMS; DATA COMMUNICATION SYSTEMS; HYDRAULICS;

COMPUTATIONAL EFFICIENCY;

EID: 77649244160     PISSN: 10414347     EISSN: None     Source Type: Journal    
DOI: 10.1109/TKDE.2009.68     Document Type: Article

References (38)

1. M. Ajtai, I.S. Jayram, R. Kumar, and D. Sivakumar, "Approximate Counting of Inversions in a Data Stream," Proc. Symp. Theory of Computing (STOC), 2002.
2. A. Arasu and G.S. Manku, "Approximate Counts and Quantiles over Sliding Windows," Proc. ACM Symp. Principles of Database Systems (PODS), 2004.
3. B. Babcock, S. Babu, M. Datar, R. Motwani, and J. Widom, "Models and Issues in Data Stream Systems," Proc. ACM Symp. Principles of Database Systems (PODS '02), 2002.
4. Z. Bar-Yossef, T.S. Jayram, R. Kumar, D. Sivakumar, and L. Trevisan, "Counting Distinct Elements in a Data Stream," Proc. Int'l Workshop Randomization and Approximation Techniques (RANDOM), 2002.
5. M. Bawa, H.G. Molina, A. Gionis, and R. Motwani, "Estimating Aggregates on a Peer-to-Peer Network," technical report, Stanford Univ., 2003.
6. J.L. Bentley, H.T. Kung, M. Schkolnick, and C.D. Thompson, "On the Average Number of Maxima in a Set of Vectors and Applications," J. ACM, vol. 25, no. 4, pp. 536-543, 1978.
7. S. Börzsönyi, D. Kossmann, and K. Stocker, "The Skyline Operator," Proc. Int'l Conf. Data Eng. (ICDE), pp. 421-430, 2001.
8. J.-Y. Chen, G. Pandurangan, and D. Xu, "Robust Computation of Aggregates in Wireless Sensor Networks: Distributed Randomized Algorithms and Analysis," Proc. Int'l Conf. Information Processing in Sensor Networks (IPSN), 2005.
9. J. Considine, F. Li, G. Kollios, and J. Byers, "Approximate Aggregation Techniques for Sensor Databases," Proc. Int'l Conf. Data Eng. (ICDE), pp. 449-460, 2004.
10. C. Cormode and S. Muthukrishnan, "An Improved Data Stream: The Count-Min Sketch and Its Applications," Proc. Latin Am. Theoretical Informatics (LATIN), 2004.
11. G. Cormode, M. Garofalakis, S. Muthukrishnan, and R. Rastogi, "Holistic Aggregates in a Networked World: Distributed Tracking of Approximate Quantiles," Proc. ACM SIGMOD, pp. 25-36, 2005.
12. G. Cormode, F. Korn, S. Muthukrishnan, and D. Srivastava, "Effective Computation of Biased Quantiles over Data Streams," Proc. Int'l Conf. Data Eng. (ICDE), 2005.
13. G. Cormode, F. Korn, S. Muthukrishnan, and D. Srivastava, "Space- and Time-Efficient Deterministic Algorithms for Biased Quantiles over Data Streams," Proc. ACM Symp. Principles of Database Systems (PODS), 2006.
14. G. Cormode and S. Muthukrishnan, "Space Efficient Mining of Multigraph Streams," Proc. Symp. Principles of Database Systems (PODS), 2005.
15. G. Cormode, S. Muthukrishnan, and W. Zhuang, "What's Different: Distributed, Continuous Monitoring of Duplicate-Resilient Aggregates on Data Streams," Proc. Int'l Conf. Data Eng. (ICDE), 2006.
16. C. Dwork, R. Kumar, M. Naor, and D. Sivakumar, "Rank Aggregation Methods for the Web," Proc. Int'l Conf. World Wide Web (WWW), 2001.
17. W. Feller, An Introduction to Probability Theory and Its Applications. John Wiley & Sons, Inc., 1966.
18. P. Flajolet and G.N. Martin, "Probabilistic Counting Algorithms for Data Base Applications," J. Computer and System Sciences, vol. 31, no. 2, pp. 182-209, 1985.
19. A. Gilbert, Y. Kotidis, S. Muthukrishnan, and M. Strauss, "How to Summarize the Universe: Dynamic Maintenance of Quantiles," Proc. Int'l Conf. Very Large Data Bases (VLDB), 2002.
20. M. Greenwald and S. Khanna, "Space-Efficient Online Computation of Quantile Summaries," Proc. ACM SIGMOD, 2001.
21. M. Greenwald and S. Khanna, "Power-Conserving Computation of Order Statistics over Sensor Networks," Proc. Symp. Principles of Database Systems (PODS), 2004.
22. S. Guha, N. Koudas, and K. Shim, "Data-Streams and Histograms," Proc. Symp. Theory of Computing (STOC), 2001.
23. S. Guha and A. McGregor, "Approximate Quantile and the Order of the Stream," Proc. Symp. Principles of Database Systems (PODS), 2006.
24. A. Gupta and F. Zane, "Counting Inversions in Lists," Proc. Symp. Discrete Algorithms (SODA), 2003.
25. M. Hadjieleftheriou, J.W. Byers, and G. Kollios, "Robust Sketching and Aggregation of Distributed Data Streams," technical report, Boston Univ., 2005.
26. J. Hershberger, N. Shrivastava, S. Suri, and C. Toth, "Adaptive Spatial Partitioning for Multidimensional Data Streams," Proc. Int'l Symp. Algorithms and Computation (ISAAC), 2004.
27. Internet Traffic Archive, http://ita.ee.lbl.gov, 2009.
28. J.I. Munro and M.S. Paterson, "Selection and Sorting with Limited Storage," Theoretical Computer Science (TCS), vol. 12, pp. 315-323, 1980.
29. D. Kempe, A. Dobra, and J. Gehrke, "Gossip-Based Computation of Aggregate Information," Proc. Ann. Symp. Foundations of Computer Science (FOCS), 2003.
30. X. Lin, H. Lu, J. Xu, and J.X. Yu, "Continuously Maintaining Quantile Summaries of the Most Recent n Elements over a Data Stream," Proc. Int'l Conf. Data Eng. (ICDE), 2004.
31. A. Manjhi, S. Nath, and P.B. Gibbons, "Tributaries and Deltas: Efficient and Robust Aggregation in Sensor Network Streams," Proc. ACM SIGMOD, 2005.
32. G.S. Manku, S. Rajagopalan, and B.G. Lindsay, "Random Sampling Techniques for Space Efficient Online Computation of Order Statistics of Large Datasets," Proc. ACM SIGMOD, 1999.
33. Massive Data Analysis Lab., http://www.cs.rutgers.edu/~muthu/massdal. html, 2009.
34. K. Mouratidis, S. Bakiras, and D. Papadias, "Continuous Monitoring of Top-k Queries over Sliding Windows," Proc. ACM SIGMOD, pp. 635-646, 2006.
35. S. Nath, P.B. Gibbons, S. Seshan, and Z.R. Anderson, "Synopsis Diffusion for Robust Aggregation in Sensor Networks," Proc. Int'l Conf. Embedded Networked Sensor Systems (SenSys), pp. 250-262, 2004.
36. N. Shrivastava, C. Buragohain, D. Agrawal, and S. Suri, "Medians and Beyond: New Aggregation Techniques for Sensor Networks," Proc. Int'l Conf. Embedded Networked Sensor Systems (SenSys '04), pp. 239-249, 2004.
37. M. Yiu, N. Marmoulis, and Y. Tao, "Efficient Quantile Retrieval on Multi-Dimensional Data," Proc. Int'l Conf. Extending Database Technology (EDBT), 2006.
38. Y. Zhang, X. Lin, J. Xu, F. Korn, and W. Wang, "Space-Efficient Relative Error Order Sketch over Data Streams," Proc. Int'l Conf. Data Eng. (ICDE), 2006.