Sketching information divergences

Machine Learning

Volumn 72, Issue 1-2, 2008, Pages 5-19

  Guha, Sudipto ^a   Indyk, Piotr ^b   McGregor, Andrew ^c  

^a UNIVERSITY OF PENNSYLVANIA   (United States)

^b MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

^c University of California at San Diego   (United States)

Author keywords

Approximation algorithms; Communication complexity; Data stream model; Information divergences; Sketches

Indexed keywords

COMPUTER SCIENCE; PROBABILITY DISTRIBUTIONS;

COMMUNICATION COMPLEXITY; DATA STREAM MODEL; DIMENSIONALITY REDUCTION; DISCRETE PROBABILITY DISTRIBUTION; IMPOSSIBILITY RESULTS; INFORMATION DIVERGENCE; MULTIPLICATIVE APPROXIMATIONS; SKETCHES;

APPROXIMATION ALGORITHMS;

EID: 84897581392     PISSN: 08856125     EISSN: 15730565     Source Type: Journal    
DOI: 10.1007/s10994-008-5054-x     Document Type: Article

References (34)

1. Alon, N., Matias, Y., & Szegedy, M. (1999). The space complexity of approximating the frequency moments. Journal of Computer and System Sciences, 58(1), 137-147.
2. Amari, S. (1985). Differential-geometrical methods in statistics. New York: Springer.
3. Bar-Yossef, Z., Jayram, T. S., Kumar, R., & Sivakumar, D. (2004). An information statistics approach to data stream and communication complexity. Journal of Computer and System Sciences, 68(4), 702-732.
4. Bhuvanagiri, L., Ganguly, S., Kesh, D., & Saha, C. (2006). Simpler algorithm for estimating frequency moments of data streams. In ACM-SIAM symposium on discrete algorithms (pp. 708-713).
5. Bregman, L. M. (1967). The relaxation method of finding the common point of convex sets and its application to the solution of problems in convex programming. USSR Computational Mathematics and Mathematical Physics, 7(1), 200-217.
6. Breiman, L. (1999). Prediction games and arcing algorithms. Neural Computation, 11(7), 1493-1517.
7. Brinkman, B., & Charikar, M. (2003). On the impossibility of dimension reduction in ℓ1. In IEEE symposium on foundations of computer science (pp. 514-523).
8. Broder, A. Z., Charikar, M., Frieze, A. M., & Mitzenmacher, M. (2000). Min-wise independent permutations. Journal of Computer and System Sciences, 60(3), 630-659.
9. Chakrabarti, A., Khot, S., & Sun, X. (2003). Near-optimal lower bounds on the multi-party communication complexity of set disjointness. In IEEE conference on computational complexity (pp. 107-117).
10. Chakrabarti, A., Cormode, G., & McGregor, A. (2007). A near-optimal algorithm for computing the entropy of a stream. In ACM-SIAM symposium on discrete algorithms (pp. 328-335).
11. Charikar, M., Chen, K., & Farach-Colton, M. (2002). Finding frequent items in data streams. In International colloquium on automata, languages and programming (pp. 693-703).
12. Collins, M., Schapire, R. E., & Singer, Y. (2002). Logistic regression, AdaBoost and Bregman distances. Machine Learning, 48(1-3), 253-285.
13. Cormode, G., & Ganguly, S. (2007). On estimating frequency moments of data streams. In International workshop on randomization and approximation techniques in computer science.
14. Cormode, G., & Muthukrishnan, S. (2005). An improved data stream summary: the count-min sketch and its applications. Journal of Algorithms, 55(1), 58-75.
15. Cormode, G., Datar, M., Indyk, P., & Muthukrishnan, S. (2003). Comparing data streams using hamming norms (how to zero in). IEEE Transactions on Knowledge and Data Engineering, 15(3), 529-540.
16. Cover, T. M., & Thomas, J. A. (1991). Elements of information theory. Wiley series in telecommunications Wiley, New York.
17. Csiszár, I. (1991). Why least squares and maximum entropy? An axiomatic approach to inference for linear inverse problems. In Annals of statistics (pp. 2032-2056).
18. Feigenbaum, J., Kannan, S., Strauss, M., & Viswanathan, M. (2002). An approximate L1 difference algorithm for massive data streams. SIAM Journal on Computing, 32(1), 131-151.
19. Friedman, J., Hastie, T., & Tibshirani, R. (2000). Additive logistic regression: a statistical view of boosting. Annals of Statistics, 28, 337-407.
20. Guha, S., McGregor, A., & Venkatasubramanian, S. (2006). Streaming and sublinear approximation of entropy and information distances. In ACM-SIAM symposium on discrete algorithms (pp. 733-742).
21. Henzinger, M. R., Raghavan, P., & Rajagopalan, S. (1999). Computing on data streams. In External memory algorithms (pp. 107-118).
22. Indyk, P. (2000). Stable distributions, pseudorandom generators, embeddings and data stream computation. In IEEE symposium on foundations of computer science (pp. 189-197).
23. Indyk, P., & Woodruff, D. P. (2005). Optimal approximations of the frequency moments of data streams. In ACM symposium on theory of computing (pp. 202-208).
24. Johnson, W. B., & Lindenstrauss, J. (1984). Extensions of Lipschitz mapping into Hilbert space. Contemporary Mathematics, 26, 189-206.
25. Kivinen, J., & Warmuth, M. K. (1999). Boosting as entropy projection. In Conference on learning theory (pp. 134-144).
26. Lafferty, J. D. (1999). Additive models, boosting, and inference for generalized divergences. In Conference on learning theory (pp. 125-133).
27. Lafferty, J. D., Pietra, S. D., & Pietra, V. J. D. (1997). Statistical learning algorithms based on Bregman distances. In Proceedings of Canadian workshop on information theory.
28. Liese, F., & Vajda, F. (1987). Convex statistical distances. Teubner-Texte zur Mathematik (Band 95), Leipzig.
29. Mason, L., Baxter, J., Bartlett, P., & Frean, M. (1999). Functional gradient techniques for combining hypotheses. In Advances in large margin classifiers. Cambridge: MIT Press.
30. McGregor, A. (Ed.). (2007). Open problems in data streams and related topics. Available at: www.cse.iitk.ac.in/users/sganguly/data-stream-probs.pdf.
31. Nguyen, X., Wainwright, M. J., & Jordan, M. I. (2005). Divergences, surrogate loss functions and experimental design. In Proceedings of NIPS.
32. Razborov, A. A. (1992). On the distributional complexity of disjointness. Theoretical Computer Science, 106(2), 385-390.
33. Saks, M. E., & Sun, X. (2002). Space lower bounds for distance approximation in the data stream model. In ACM symposium on theory of computing (pp. 360-369).
34. Woodruff, D. P. (2004). Optimal space lower bounds for all frequency moments. In ACM-SIAM symposium on discrete algorithms (pp. 167-175).