On the performance of clustering in Hilbert spaces

IEEE Transactions on Information Theory

Volumn 54, Issue 2, 2008, Pages 781-790

  Biau, Gérard ^a   Devroye, Luc ^b   Lugosi, Gábor ^c  

^a SORBONNE UNIVERSITÉ   (France)

^b MCGILL UNIVERSITY   (Canada)

^c UNIVERSITAT POMPEU FABRA   (Spain)

Author keywords

Clustering; Empirical risk minimization; Hilbert space; K means; Random projections; Vector quantization

Indexed keywords

COMPUTATIONAL COMPLEXITY; OPTIMIZATION; VECTOR QUANTIZATION;

EMPIRICAL RISK MINIMIZATION; K-MEANS CLUSTERING; RANDOM PROJECTIONS;

HILBERT SPACES;

EID: 39849091360     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIT.2007.913516     Document Type: Article

References (45)

1. E. A. Abaya and G. L. Wise, "Convergence of vector quantizers with applications to optimal quantization," SIAM J. Appl. Math., vol. 44, pp. 183-189, 1984.
2. A. Ambroladze, E. Parrado-Hernandez, and J. Shawe-Taylor, "Complexity of pattern classes and the Lipschitz property," Theoret. Comput. Sci., vol. 382, pp. 232-246, 2007.
3. A. Antos, "Improved minimax bounds on the test and training distortion of empirically designed vector quantizers," IEEE Trans. Inf. Theory vol. 51, no. 11, pp. 4022-4032, 2005.
4. A. Antos, L. Györfi, and A. György, "Improved convergence rates in empirical vector quantizer design," IEEE Trans. Inf. Theory, vol. 51, no. 1l, pp. 4013-4022, 2005.
5. P. L. Bartlett, "Prediction algorithms: Complexity, concentration and convexity," in Proc. 13th IFAC Symp. Syst. Ident., 2003, pp. 1507-1517.
6. P. L. Bartlett, S. Boucheron, and G. Lugosi, "Model selection and error estimation," Machine Learn., vol. 48, pp. 85-113, 2001.
7. P. L. Bartlett, T. Linder, and G. Lugosi, "The minimax distortion redundancy in empirical quantizer design," IEEE Trans. Inf. Theory vol. 44, no. 5, pp. 1802-1813, 1998.
8. P. L. Bartlett and S. Mendelson, "Rademacher and Gaussian complexities: Risk bounds and structural results," J. Machine Learn. Res., vol. 3, pp. 463-482, 2002.
9. E. Bingham and H. Mannila, "Random projection in dimensionality reduction: Applications to image and text data," in Proc. Seventh ACM SIGKDD Int. Conf. Knowledge Discovery and Data Mining (KDD 2001), 2001, pp. 245-250.
10. H. Chernoff, "A measure of asymptotic efficiency for tests of a hypothesis based on the sum of observations," Ann. Math. Stat., vol. 23, pp. 493-507, 1952.
11. P. A. Chou, "The distortion of vector quantizers trained on n vectors decreases to the optimum at O p (1/n)," in Proc. IEEE Int. Symp. Inf. Theory, Trondheim, Norway, 1994, p. 457.
12. S. Dasgupta and A. Gupta, "An elementary proof of a theorem of Johnson and Lindenstrauss," Random Struct. Algor., vol. 22, pp. 60-65, 2003.
13. L. Devroye, L. Györfi, and G. Lugosi, A Probabilistic Theory of Pattern Recognition. New York: Springer-Verlag, 1996.
14. R. O. Duda, P. E. Hart, and D. G. Stork, Pattern Classification, 2nd ed. New York: Wiley-Interscience, 2000.
15. R. M. Dudley, Real Analysis and Probability, 2nd ed. Cambridge, U.K.: Cambridge University Press, 2002.
16. P. Frankl and H. Maehara, "The Johnson-Lindenstrauss lemma and the sphericity of some graphs," J. Combin. Theory, Series B, vol. 44, pp. 355-362, 1988.
17. P. Frankl and H. Maehara, "Some geometric applications of the beta distribution," Ann. Inst. Stat. Math., vol. 42, pp. 463-474, 1990.
18. A. Gersho and R. M. Gray, Vector Quantization and Signal Compression Boston, MA: Kluwer Academic, 1992.
19. S. Graf and H. Luschgy, Foundations of Quantization for Probability Distributions, ser. Lecture Notes in Mathematics, 1730. Berlin, Germany: Springer, 2000.
20. R. M. Gray and D. L. Neuhoff, "Quantization," IEEE Trans. Inf. Theory, vol. 44, no. 6, pp. 2325-2384, 1998.
21. J. A. Hartigan, Clustering Algorithms. New York: Wiley, 1975.
22. J. A. Hartigan, "Asymptotic distributions for clustering criteria," Ann. Stat., vol. 6, pp. 117-131, 1978.
23. P. Indyk and R. Motwarti, "Approximate nearest neighbors: Toward removing the curse of dimensionality," in Proc. 30th Symp. Theory Comput., 1998, pp. 604-613.
24. W. Johnson and J. Lindenstrauss, "Extensions of Lipschitz maps into a Hilbert space," Contemp. Mathematics., vol. 26, pp. 189-206, 1984.
25. J. M. Kleinberg, "Two algorithms for nearest-neighbor search in high dimensions," in Proc. 29th Ann. ACM Symp. Theory Comput., 1997, pp. 599-608.
26. V. Koltchinskii, "Rademacher penalties and structural risk minimization," IEEE Trans. Inf. Theory, vol. 47, no. 5, pp. 1902-1914, 2001.
27. V. Koltchinskii, "Local Rademacher complexities and oracle inequalities in risk minimization," Ann Stat., vol. 34, pp. 2593-2656, 2006.
28. M. Ledoux and M. Talagrand, Probability in Banach Spaces. Berlin, Germany: Springer-Verlag, 1991.
29. T. Linder, "On the training distortion of vector quantizers," IEEE Trans. Inf. Theory, vol. 46, no. 4, pp. 1617-1623, 2000.
30. T. Linder, Learning-Theoretic Methods in Vector Quantization, ser. Lecture Notes for the Advanced School on the Principles of Nonparametric Learning (Udine, Italy). Berlin, Germany: Springer-Verlag, 2001, pp. 163-210.
31. T. Linder, G. Lugosi, and K. Zeger, "Rates of convergence in the source coding theorem, in empirical quantizer design, and in universal lossy source coding," IEEE Trans. Inf. Theory, vol. 40, no. 6, pp. 1728-1740, 1994.
32. N. Linial, E. London, and Y. Rabinovich, "The geometry of graphs and some of its algorithmic applications," Combinatorica, vol. 15, pp. 215-245, 1995.
33. K. V. Mardia, J. T. Kent, and J. M. Bibby, Multivariate Analysis. London, U.K.: Academic, 1979.
34. C. McDiarmid, "On the method of bounded differences," in Surveys in Combinatorics 1989. Cambridge, U.K.: Cambridge University Press, 1989, pp. 148-188.
35. D. Pollard, "Strong consistency of k-means clustering," Ann. Stat., vol. 9, pp. 135-140, 1981.
36. D. Pollard, "A central limit theorem for k-means clustering," Ann. Prob., vol. 10, pp. 919-926, 1982.
37. D. Pollard, "Quantization and the method of k-means," IEEE Trans. Inf. Theory, vol. IT-28, no. 2, pp. 199-205, 1982.
38. S. T. Rachev and L. Rüschendorf, Mass Transportation Problems, Volume I: Theory. New York: Springer-Verlag, 1998.
39. S. T. Rachev a nd L. Rüschendorf, Mass Transportation Problems, Volume II: Applications. New York: Springer-Verlag, 1998.
40. J. O. Ramsay and B. W. Silverman, Functional Data Analysis. New York: Springer, 1997.
41. B. Schölkopf and A. J. Smola, Learning With Kernels. Cambridge, MA: The MIT Press, 2002.
42. J. Shawe-Taylor and N. Cristianini, Kernel Methods for Pattern Analysis. Cambridge, UX.: Cambridge University Press, 2004.
43. V. N. Vapnik, Statistical Learning Theory. New York: Wiley, 1998.
44. S. Vempala and G. Wang, "A spectral algorithm for learning mixtures of distributions," in Proc. 43rd IEEE Conf. Foundations of Computer Science (FOCS'02), 2002, p. 113.
45. D. Williams, Probability With Martingales. Cambridge, U.K.: Cambridge University Press, 1991.