Nonparametric divergence estimation with applications to machine learning on distributions

Proceedings of the 27th Conference on Uncertainty in Artificial Intelligence, UAI 2011

Volumn , Issue , 2011, Pages 599-608

  Póczos, Barnabás ^a   Xiong, Liang ^a   Schneider, Jeff ^a  

^a Carnegie Mellon University   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ANOMALY DETECTION; EMBEDDINGS; PROBABILITY DISTRIBUTIONS;

ASTRONOMICAL DATA; CONTINUOUS PROBABILITY DISTRIBUTION; DIVERGENCE ESTIMATION; ESTIMATION ALGORITHM; FEATURE REPRESENTATION; FINITE DIMENSIONAL; LOW DIMENSIONAL EMBEDDING; MANIFOLD LEARNING;

MACHINE LEARNING;

EID: 80053153929     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (37)

1. M. Belkin and P. Niyogi. Laplacian eigenmaps for dimensionality reduction and data representation. Neural Computation, 15:1373-1396, 2003. (Pubitemid 37049796)
2. I. Borg and P. Groenen. Modern Multidimensional Scaling: theory and applications. Springer-Verlag, New York, 2005.
3. A. Bosch, A. Zisserman, and X. Munoz. Scene classification via pLSA. In ECCV, 2006.
4. V. Chandola, A. Banerjee, and V. Kumar. Anomaly detection: A survey. ACM Computing Surveys, 41-3, 2009.
5. A. Cichocki, R. Zdunek, A. Phan, and S. Amari. Nonnegative Matrix and Tensor Factorizations. John Wiley and Sons, 2009.
6. D. Donoho and C. Grimes. Hessian eigenmaps: new locally linear embedding techniques for highdimensional data. PNAS, 100:5591-5596, 2003. (Pubitemid 36576852)
7. L. Fei-Fei and P. Perona. A bayesian hierarchical model for learning natural scene categories. IEEE Conf. CVPR, pages 524-531, 2005.
8. A. Garcia. General study of group membership. IIdetermination of nearby groups. Astronomy & Astrophysics Suppliment Series, 100:47-90, 1993.
9. M. Goria, N. Leonenko, V. Mergel, and N. Inverardi. A new class of random vector entropy estimators and its applications in testing statistical hypotheses. Journal of Nonparametric Statistics, 17:277-297, 2005. (Pubitemid 40465829)
10. M. Gupta and S. Srivastava. Parametric bayesian estimation of differential entropy and relative entropy. Entropy, 12:818-843, 2010.
11. A. Hero, B. Ma, O. Michel, and J. Gorman. Alphadivergence for classification, indexing and retrieval, 2002a. Communications and Signal Processing Laboratory Technical Report CSPL-328.
12. A. Hero, B. Ma, O. Michel, and J. Gorman. Applications of entropic spanning graphs. IEEE Signal Processing Magazine, 19(5):85-95, 2002b.
13. T. Hofmann. Probabilistic latent semantic analysis. In UAI, 1999.
14. B. Jian and B. Vemuri. A robust algorithm for point set registration using mixture of gaussians. In ICCV, 2005.
15. N. Leonenko and L. Pronzato. Correction of 'a class of Rényi information estimators for mulitidimensional densities' Annals of Statistics, 36(2008) 2153-2182, 2010.
16. N. Leonenko, L. Pronzato, and V. Savani. A class of Rényi information estimators for multidimensional densities. Annals of Statistics, 36(5):2153-2182, 2008.
17. D. Loftsgaarden and C. Quesenberry. A nonparametric estimate of a multivariate density function. Annals of Mathematical Statistics, 36:1049-1051, 1965.
18. D. Lowe. Distinctive image features from scale-invariant keypoints. IJCV, 60(2):91-110, 2004.
19. X. Nguyen, M. Wainwright, and M. Jordan. On surrogate loss functions and f-divergences. Annals of Statistics, 37:876-904, 2009.
20. X. Nguyen, M. Wainwright, and M. Jordan. Estimating divergence functionals and the likelihood ratio by convex risk minimization. IEEE Transactions on Information Theory, To appear., 56:5847-5861, 2010.
21. D. Pál, B. Póczos, and Cs. Szepesvári. Estimation of Rényi entropy and mutual information based on generalized nearest-neighbor graphs. In NIPS 2010, 2010.
22. F. Pérez-Cruz. Estimation of information theoretic measures for continuous random variables. In NIPS- 2008, volume 21, 2008.
23. B. Póczos and J. Schneider. On the estimation of alpha-divergences. In AISTATS 2011, 2011.
24. B. Póczos, S. Kirshner, and Cs. Szepesvári. REGO: Rank-based estimation of Rényi information using Euclidean graph optimization. In AISTATS 2010, 2010.
25. B. Póczos, L. Xiong, and J. Schneider. Nonparametric divergence estimation with applications to machine learning on distributions, Supplementary material. http://www.autonlab.org/autonweb/library/papers.html, 2011.
26. A. Rényi. On measures of entropy and information. In Fourth Berkeley Symposium on Mathematical Statistics and Probability. University of California Press, 1961.
27. S. Roweis and L. Saul. Nonlinear dimensionality reduction by locally linear embedding. Science, 290(5500): 2323-2326, 2000. (Pubitemid 32041578)
28. K. Sricharan, R. Raich, and A. Hero. Empirical estimation of entropy functionals with confidence. Technical Report, http://arxiv.org/abs/1012.4188, 2010.
29. J. Sun, C. Fyfe, and M. Crowe. Curvilinear component analysis and bregman divergences. In ESANN 2010, 2010.
30. J. B. Tenenbaum, V. de Silva, and J. C. Langford. A global geometric framework for nonlinear dimensionality reduction. Science, 290(5500):2319-2323, 2000. (Pubitemid 32041577)
31. A. van der Wart. Asymptotic Statistics. Cambridge University Press, 2007.
32. T. Villmann and S. Haase. Mathematical aspects of divergence based vector quantization using Frechetderivatives, 2010. University of Applied Sciences Mittweida.
33. Q. Wang, S. Kulkarni, and S. Verdú. Universal estimation of information measures for analog sources. Foundations and Trends in Communications and Information Theory, 5(3):265-352, 2009a.
34. Q. Wang, S. Kulkarni, and S. Verdú. Divergence estimation for multidimensional densities via k-nearestneighbor distances. IEEE Transactions on Information Theory, 55(5), 2009b.
35. L. Zelnik-Manor and P. Perona. Self-tuning spectral clustering. In NIPS 17, 2004.
36. Z. Zhang and H. Zha. Principal manifolds and nonlinear dimensionality reduction via tangent space alignment. SIAM J. Scientific Computing, 26:313-338, 2004.
37. M. Zhao and V. Saligrama. Anomaly detection with score functions based on nearest neighbor graphs. In NIPS, 2009.