Continuously monitoring top-k uncertain data streams: a probabilistic threshold method

Distributed and Parallel Databases

Volumn 26, Issue 1, 2009, Pages 29-65

  Hua, Ming ^a   Pei, Jian ^a  

^a SIMON FRASER UNIVERSITY   (Canada)

Author keywords

Probabilistic threshold top k queries; Query processing; Uncertain streams

Indexed keywords

CONTINUOUS QUERIES; DATA STREAM; EMPIRICAL STUDIES; EXACT ALGORITHMS; PROBABILISTIC THRESHOLD TOP-K QUERIES; RANDOMIZED METHOD; REAL DATA SETS; SYNTHETIC DATASETS; THRESHOLD METHODS; UNCERTAIN DATA; UNCERTAIN STREAMS;

DATA COMMUNICATION SYSTEMS; INFORMATION RETRIEVAL; QUERY PROCESSING;

DATA PROCESSING;

EID: 67651207502     PISSN: 09268782     EISSN: 15737578     Source Type: Journal    
DOI: 10.1007/s10619-009-7043-x     Document Type: Article

References (57)

1. Abiteboul, S., Kanellakis, P., Grahne, G.: On the representation and querying of sets of possible worlds. In: Proceedings of the 1987 ACM SIGMOD International Conference on Management of Data (SIGMOD'87), pp. 34-48. ACM Press, New York (1987)
2. Angluin, D., Valiant, L.G.: Fast probabilistic algorithms for Hamiltonian circuits and matchings. In: Proceedings of the Ninth Annual ACM Symposium on Theory of Computing (STOC'77), pp. 30-41. ACM Press, New York (1977)
3. 106 worlds and beyond: efficient representation and processing of incomplete information. In: Proceedings of the 2007 IEEE 23rd International Conference on Data Engineering (ICDE'07), 2007
4. Arasu, A., Manku, G.S.: Approximate counts and quantiles over sliding windows. In: Proceedings of the Twenty-third ACM SIGACT-SIGMOD-SIGART Symposium on Principles of Database Systems (PODS'04), Paris, France, 2004
5. Babcock, B., Babu, S., Datar, M., Motwani, R., Widom, J.: Models and issues in data stream systems. In: Proc. 21st ACM SIGACT-SIGMOD-SIGART Symposium on Principles of Database Systems (PODS'02), Madison, WI, 2002
6. Babcock, B., Olston, C.: Distributed top-k monitoring. In: SIGMOD Conference, pp. 28-39, 2003
7. S. Babu J. Widom 2001 Continuous queries over data streams SIGMOD Rec. 30 109 120
8. Chen, J., DeWitt, D., Tian, F., Wang, Y.: NiagraCQ: A scalable continuous query system for internet databases. In: Proc. 2000 ACM-SIGMOD Int. Conf. Management of Data (SIGMOD'00), Dallas, TX, 2000
9. Cheng, R., Kalashnikov, D.V., Prabhakar, S.: Evaluating probabilistic queries over imprecise data. In: Proceedings of the 2003 ACM SIGMOD International Conference on Management of Data (SIGMOD'03), pp. 551-562. ACM Press, New York (2003)
10. Cheng, R., Xia, Y., Prabhakar, S., Shah, R., Vitter, J.S.: Efficient indexing methods for probabilistic threshold queries over uncertain data. In: VLDB, pp. 876-887, 2004
11. Cormode, G., Garofalakis, M.N.: Sketching probabilistic data streams. In: SIGMOD Conference, pp. 281-292, 2007
12. Cormode, G., Korn, F., Muthukrishnan, S., Srivastava, D.: Effective computation of biased quantiles over data streams. In: ICDE'05, 2005
13. Cormode, G., Korn, F., Muthukrishnan, S., Srivastava, D.: Space- and time-efficient deterministic algorithms for biased quantiles over data streams. In: PODS'06, 2006
14. Dalvi, N., Suciu, D.: The dichotomy of conjunctive queries on probabilistic structures. In: Proceedings of the Twenty-sixth ACM SIGMOD-SIGACT-SIGART Symposium on Principles of Database Systems (PODS'07), pp. 293-302. ACM Press, New York (2007)
15. Dalvi, N., Suciu, D.: Management of probabilistic data: foundations and challenges. In: Proceedings of the Twenty-sixth ACM SIGMOD-SIGACT-SIGART Symposium on Principles of Database Systems (PODS'07), pp. 1-12. ACM Press, New York (2007)
16. Dalvi, N.N., Suciu, D.: Efficient query evaluation on probabilistic databases. In: Proceedings of the Thirtieth International Conference on Very Large Data Bases, Toronto, Canada, pp. 864-875, 2004
17. Das, G., Gunopulos, D., Koudas, N., Sarkas, N.: Ad-hoc top-k query answering for data streams. In: VLDB, pp. 183-194, 2007
18. Das, G., Gunopulos, D., Koudas, N., Tsirogiannis, D.: Answering top-k queries using views. In: VLDB, pp. 451-462, 2006
19. Deshpande, A., Guestrin, C., Madden, S., Hellerstein, J.M., Hong, W.: Model-driven data acquisition in sensor networks. In: VLDB, pp. 588-599, 2004
20. D. Dey S. Sarkar 1996 A probabilistic relational model and algebra ACM Trans. Database Syst. 21 3 339 369
21. Doucet, A., Vo, B.N., Andrieu, C., Davy, M.: Particle filtering for multi-target tracking and sensor management. In: Proceedings of the Fifth International Conference on Information Fusion, pp. 474-481, 2002
22. Fagin, R., Lotem, A., Naor, M.: Optimal aggregation algorithms for middleware. In: PODS '01: Proceedings of the Twentieth ACM SIGMOD-SIGACT-SIGART Symposium on Principles of Database Systems, pp. 102-113. ACM Press, New York (2001)
23. Gardner, D., Bostrom, R.: Ohio and Kentucky approach to data archiving in Cincinnati. http://trb.mtc.ca.gov/urban/adus/Ohio.pdf
24. Gehrke, J., Korn, F., Srivastava, D.: On computing correlated aggregates over continuous data streams. In: Proc. 2001 ACM-SIGMOD Int. Conf. Management of Data (SIGMOD'01), Santa Barbara, CA, pp. 13-24, 2001
25. Gilbert, A.C., Kotidis, Y., Muthukrishnan, S., Strauss, M.: How to summarize the universe: Dynamic maintenance of quantiles. In: VLDB, pp. 454-465, 2002
26. Greenwald, M., Khanna, S.: Space-efficient online computation of quantile summaries. In: SIGMOD Conference, pp. 58-66, 2001
27. Gupta, A., Zane, F.X.: Counting inversions in lists. In: SODA '03: Proceedings of the Fourteenth Annual ACM-SIAM Symposium on Discrete Algorithms, pp. 253-254. Society for Industrial and Applied Mathematics, Philadelphia (2003)
28. S. Han E. Chan R. Cheng K. yiu Lam 2007 A statistics-based sensor selection scheme for continuous probabilistic queries in sensor networks Real-Time Syst. 35 1 33 58
29. W. Hoeffding 1956 On the distribution of the number of successes in independent trials Ann. Math. Stat. 27 713 721
30. Hua, M., Pei, J., Zhang, W., Lin, X.: Efficiently answering probabilistic threshold top-k queries on uncertain data (extended abstract). In: Proc. International Conference on Data Engineering (ICDE'08), Cancun, Mexico, 2008
31. Hua, M., Pei, J., Zhang, W., Lin, X.: Ranking queries on uncertain data: a probabilistic threshold approach. In: Proc. ACM International Conference on Management of Data (SIGMOD'08), Vancouver, Canada, 2008
32. I.F. Ilyas G. Beskales M.A. Soliman 2008 A survey of top-k query processing techniques in relational database systems ACM Comput. Surv. 40 4 1 58
33. T. Imielinski J. Witold Lipski 1984 Incomplete information in relational databases J. ACM 31 4 761 791
34. Jayram, T.S., Kale, S., Vee, E.: Efficient aggregation algorithms for probabilistic data. In: SODA '07: Proceedings of the Eighteenth Annual ACM-SIAM Symposium on Discrete Algorithms, pp. 346-355. Society for Industrial and Applied Mathematics, Philadelphia (2007)
35. Jayram, T.S., McGregor, A., Muthukrishnan, S., Vee, E.: Estimating statistical aggregates on probabilistic data streams. In: PODS, pp. 243-252, 2007
36. Kalton, G.: Introduction to Survey Sampling, 1st edn. Sage Publications, Thousand Oaks (1983)
37. Kanagal, B., Deshpande, A.: Efficient query evaluation over temporally correlated probabilistic streams. In: ICDE, 2009
38. Lange, K.: Numerical Analysis for Statisticians. Springer, New York (1999)
39. Lee, S.K.: Imprecise and uncertain information in databases: an evidential approach. In: Proceedings of the Eighth International Conference on Data Engineering (ICDE'92), pp. 614-621. IEEE Computer Society, Washington (1992)
40. Lian, X., Chen, L.: Probabilistic ranked queries in uncertain databases. In: Proc. 2008 International Conference on Extended Data Base Technology (EDBT'08), 2008
41. Lin, X., Lu, H., Xu, J., Yu, J.X.: Continuously maintaining quantile summaries of the most recent n elements over a data stream. In: ICDE, pp. 362-374, 2004
42. Liu, Z., Sia, K.C., Cho, J.: Cost-efficient processing of min/max queries over distributed sensors with uncertainty. In: SAC, pp. 634-641, 2005
43. Manku, G.S., Rajagopalan, S., Lindsay, B.G.: Approximate medians and other quantiles in one pass and with limited memory. In: SIGMOD Conference, pp. 426-435, 1998
44. Manku, G.S., Rajagopalan, S., Lindsay, B.G.: Random sampling techniques for space efficient online computation of order statistics of large datasets. In: SIGMOD '99: Proceedings of the 1999 ACM SIGMOD International Conference on Management of Data, pp. 251-262. ACM Press, New York (1999)
45. Michel, S., Triantafillou, P., Weikum, G.: Klee: A framework for distributed top-k query algorithms. In: VLDB, pp. 637-648, 2005
46. Mouratidis, K., Bakiras, S., Papadias, D.: Continuous monitoring of top-k queries over sliding windows. In: SIGMOD Conference, pp. 635-646, 2006
47. J.I. Munro M. Paterson 1980 Selection and sorting with limited storage Theor. Comput. Sci. 12 315 323
48. Nepal, S., Ramakrishna, M.: Query processing issues in image(multimedia) databases. In: Proceedings of the 15th International Conference on Data Engineering, pp. 22-29. IEEE Computer Society, Washington (1999)
49. Ré, C., Dalvi, N., Suciu, D.: Efficient top-k query evaluation on probabilistic data. In: Proceedings of the 23nd International Conference on Data Engineering (ICDE'07), Istanbul, Turkey. IEEE, New York (2007)
50. Sarma, A.D., Benjelloun, O., Halevy, A., Widom, J.: Working models for uncertain data. In: ICDE '06: Proceedings of the 22nd International Conference on Data Engineering (ICDE'06), p. 7. IEEE Computer Society, Washington (2006)
51. Singh, S., Mayfield, C., Mittal, S., Prabhakar, S., Hambrusch, S., Shah, R.: Orion 2.0: native support for uncertain data. In: SIGMOD '08: Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data, pp. 1239-1242. ACM Press, New York (2008)
52. Singh, S., Mayfield, C., Prabhakar, S., Shah, R., Hambrusch, S.: Indexing uncertain categorical data. In: Proceedings of the 2007 IEEE 23rd International Conference on Data Engineering (ICDE'07), 2007
53. Singh, S., Mayfield, C., Shah, R., Prabhakar, S., Hambrusch, S.E., Neville, J., Cheng, R.: Database support for probabilistic attributes and tuples. In: ICDE, pp. 1053-1061, 2008
54. Soliman, M.A., Ilyas, I.F., Chang, K.C.C.: Top-k query processing in uncertain databases. In: Proceedings of the 23nd International Conference on Data Engineering (ICDE'07), Istanbul, Turkey. IEEE, New York (2007)
55. Tao, Y., Cheng, R., Xiao, X., Ngai, W.K., Kao, B., Prabhakar, S.: Indexing multi-dimensional uncertain data with arbitrary probability density functions. In: Proceedings of the 31st International Conference on Very Large Data Bases (VLDB'05), VLDB Endowment, pp. 922-933, 2005
56. Zhang, X., Chomicki, J.: On the semantics and evaluation of top-k queries in probabilistic databases. In: Proc. the Second International Workshop on Ranking in Databases (DBRank'08), 2008
57. Zhang, Y., Lin, X., Xu, J., Korn, F., Wang, W.: Space-efficient relative error order sketch over data streams. In: ICDE '06: Proceedings of the 22nd International Conference on Data Engineering (ICDE'06), p. 51. IEEE Computer Society, Washington (2006)