Sufficient dimensionality reduction

Journal of Machine Learning Research

Volumn 3, Issue , 2003, Pages 1307-1331

  Globerson, Amir ^a   Tishby, Naftali ^a  

^a HEBREW UNIVERSITY OF JERUSALEM   (Israel)

Author keywords

Association Analysis; Feature Extraction; Information Geometry; Maximum Entropy; Mutual Information

Indexed keywords

ASSOCIATION ANALYSIS; DIMENSIONALITY REDUCTION; DIMENSIONALITY REDUCTION METHOD; GEOMETRIC INTERPRETATION; INFORMATION GEOMETRY; INFORMATION PROJECTIONS; MUTUAL INFORMATIONS; SUPERVISED AND UNSUPERVISED LEARNING;

ALGORITHMS; FEATURE EXTRACTION; GEOMETRY; INFORMATION THEORY; ITERATIVE METHODS; MAXIMUM ENTROPY METHODS; TEXT PROCESSING; UNSUPERVISED LEARNING;

DATA REDUCTION;

EID: 80455131402     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (35)

1. S. Amari and N. Nagaoka. Methods of Information Geometry. Oxford University Press, 2000.
2. M.P. Becker and C.C. Clogg. Analysis of sets of two-way contingency tables using association models. Journal of the American Statistical Association, 84(405):142-151, 1989.
3. A.L. Berger, S.A. Della-Pietra, and V.J. Della-Pietra. A maximum entropy approach to natural language processing. Computational Linguistics, 22(1):39-71, 1996.
4. T.M. Cover and J.A Thomas. Elements of information theory. Wiley, 1991.
5. T. Cox and M. Cox. Multidimensional Scaling. Chapman and Hall, 1994.
6. I. Csiszar. I-divergence geometry of probability distributions and minimization problems. Annals of Probability, 3(1):146-158, 1975.
7. J.N. Darroch and D. Ratcliff. Generalized iterative scaling for log-linear models. Ann.Math. Statist., 43:1470-1480, 1972.
8. S.C. Deerwester, S.T. Dumais, T.K. Landauer, G.W. Furnas, and R.A. Harshman. Indexing by latent semantic analysis. Journal of the American Society of Information Science, 41(6):391-407, 1990.
9. S.A. Della-Pietra, V.J. Della-Pietra, and J.D. Lafferty. Inducing features of random fields. IEEE Transactions on PAMI, 19(4):380-393, 1997.
10. L. Devroye, L. Gyorfi, and G. Lugosi. A Probabilistic Thoery of Pattern Recognition. Springer, 1996.
11. R.A. Fisher. The precision of discriminant functions. Annals of Eugenics, 10:422-429, 1940.
12. N. Friedman, O. Mosenzon, N. Slonim, and N. Tishby. Multivariate information bottleneck. In Proceedings of the Seventeenth Conference on Uncertainty in Artificial Intelligence (UAI), 2001.
13. D. Geiger, A. Rudra, and L.T. Maloney. Features as sufficient statistics. In Advances in Neural Information Processing Systems, volume 10, 1998.
14. L.A. Goodman. Simplemodels for the analysis of association in cross-classifications having ordered categories. Journal of the American Statistical Association, 74(367):537-552, 1979.
15. L.A. Goodman. The analysis of cross-classified data having ordered and/or unordered categories: Association models, correlation models, and asymmetry models for contingency tables with or without missing entries. Annals of Statistics, 13(1):10-69, 1985.
16. S.J. Haberman. Analysis of qulitative data. Academic Press, 1978.
17. T. Hofmann. Unsupervised learning by probabilistic latent semantic analysis. Machine Learning, 42(1):177-196, 2001.
18. E.T. Jaynes. Information theory and statistical mechanics. Physical Review, 106:620-630, 1957.
19. T. Joachims. Text categorization with support vector machines: learning with many relevant features. In 10th European Conference on Machine Learning, number 1398, 1998.
20. M. Kearns. Efficient noise-tolerant learning from statistical queries. In ACM Symposium on Theory of Computing, pages 392-401, 1993.
21. K. Lang. Newsweeder: Learning to filter news. In ICML 95, pages 331-339, 1995.
22. A. Lapidoth and P. Narayan. Reliable communication under channel uncertainty. IEEE Transactions on Information Theory, 44(6):2148-2177, 1998.
23. D. Lee and H. Seung. Learning the parts of objects by non-negative matrix factorization. Nature, 401(6755):788-791, 1999.
24. R. Malouf. A comparison of algorithms for maximum entropy parameter estimation. In Sixth Conf. on Natural Language Learning, pages 49-55, 2002.
25. K. Nigam, J. Lafferty, and A. McCallum. Using maximum entropy for text classification. In IJCAI- 99 Workshop on Machine Learning for Information Filtering, pages 61-67, 1999.
26. E. Pitman. Sufficient statistics and intrinsic accuracy. Proc. of the Cambridge Phil. Soc., 32:567-579, 1936.
27. S.T. Roweis and L.K. Saul. Nonlinear dimensionality reduction by locally linear embedding. Science, 290(5500):2323-6, 2000.
28. S.T. Roweis, L.K. Saul, and G. Hinton. Global coordination of local linear models. In Advances in Neural Information Processing Systems 14, pages 889-896, 2001.
29. N. Slonim and N. Tishby. Agglomerative information bottleneck. In Advances in neural information processing systems (Vol. 12). MIT Press, 1999.
30. N. Slonim and N. Tishby. Document clustering using word clusters via the information bottleneck method. In ACM SIGIR 2000, pages 208-215, 2000.
31. J.B. Tenenbaum, V. de Silva, and J.C. Langford. A global geometric framework for nonlinear dimensionality reduction. Science, 290(5500):2319-2323, 2000.
32. N. Tishby, F.C. Pereira, and W. Bialek. The information bottleneck method. In Proc. of the 37-th Annual Allerton Conference on Communication, Control and Computing, pages 368-377, 1999.
33. D.R. Wolf and E.I. George. Maximally informative statistics. 2000. URL http://lanl.arXiv.org/abs/physics/0010039.
34. Y. Yang and X. Liu. A re-examination of text categorization methods. In ACM SIGIR 1999, pages 42-49, 1999.
35. S.C. Zhu, Z.N. Wu, and D. Mumford. Minimax entropy principle and its application to texture modeling. Neural Computation, 9(8):1627-1660, 1997.