Maximization of mutual information for supervised linear feature extraction

IEEE Transactions on Neural Networks

Volumn 18, Issue 5, 2007, Pages 1433-1441

  Leiva Murillo, Jose M ^a   Artés Rodríguez, Antonio ^a  

^a UNIVERSIDAD CARLOS III DE MADRID   (Spain)

Author keywords

Feature extraction; Information theory; Pattern recognition

Indexed keywords

CLASSIFICATION (OF INFORMATION); COMPUTER SIMULATION; GRADIENT METHODS; INDEPENDENT COMPONENT ANALYSIS; OPTIMIZATION; PATTERN RECOGNITION;

COMPONENT-BY-COMPONENT GRADIENT-ASCENT METHOD; MUTUAL INFORMATION (MI); SUPERVISED LINEAR FEATURE EXTRACTION;

FEATURE EXTRACTION;

ALGORITHM; ARTICLE; ARTIFICIAL INTELLIGENCE; AUTOMATED PATTERN RECOGNITION; COMPUTER SIMULATION; INFORMATION RETRIEVAL; METHODOLOGY; REPRODUCIBILITY; SENSITIVITY AND SPECIFICITY; STATISTICAL MODEL;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; COMPUTER SIMULATION; INFORMATION STORAGE AND RETRIEVAL; LINEAR MODELS; PATTERN RECOGNITION, AUTOMATED; REPRODUCIBILITY OF RESULTS; SENSITIVITY AND SPECIFICITY;

EID: 34548656483     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/TNN.2007.891630     Document Type: Article

References (28)

1. B. Schölkopf, A. Smola, and K. R. Müller, "Nonlinear component analysis as a kernel eigenvalue problem," Neural Comput., vol. 10, no. 15, pp. 1299-1319, 1998.
2. J. Goldberger, S. Roweis, G. Hinton, and R. Salakhutdinov, "Neighbourhood components analysis," in Advances in Neural Information Processing Systems 17, L. K. Saul, Y. Weiss, and L. Bottou, Eds. Cambridge, MA: MIT Press, 2005, pp. 513-520.
3. K. Fukunaga, Introduction to Statistical Pattern Recognition. New York: Academic, 1990.
4. K. Li, "Sliced inverse regression for dimension reduction," J. Amer. Statist. Soc., vol. 86, no. 414, pp. 316-327, 1991.
5. J. Wang, L. Ji, and Z. Kou, "Facial feature point extraction by partial least square regression," The Pennsylvania State University Cite Seer Archives, Mar. 19, 2003.
6. H. Knutsson, M. Borga, and T. Landelius, "Learning canonical correlations," Comp. Vis. Lab., Linköping, Sweden, Rep. LiTH-ISY-R-1761, Jun. 1995.
7. T. M. Cover and J. A.Thomas, Elements of Information Theory. New York: Wiley, 1991.
8. K. Torkkola, "Feature extraction by non-parametric mutual information maximization," J. Mach. Learn. Res., vol. 3, pp. 1415-1438, 2003.
9. J. M. Leiva and A. Artés, "A Gaussian mixture based maximization of mutual information for supervised feature extraction," in Proc. 5th Int. Conf. ICA, Granada, Spain, 2004, pp. 271-278.
10. K. Torkkola and W. Campbell, "Mutual information in learning feature transformations," in Proc. 17th Int. Conf. Mach. Learn., 2000, pp. 1015-1022.
11. J. Principe, D. Xu, and J. W. Fischer, III, Information-Theoretic Learning. New York: Wiley, 2000, vol. 1.
12. J. Peltonen and S. Kaski, "Discriminative components of data," IEEE Trans. Neural Netw., vol. 16, no. 1, pp. 68-83, Jan. 2005.
13. A. Kraskov, H. Stoegbauer, and P. Grassberger, "Estimating mutual information," Phys. Rev. E, Stat. Phys. Plasmas Fluids Relat., vol. 69, pp. 066138-1-066138-16, 2004.
14. L. F. Kozachenko and N. N. Leonenko, "Sample estimate of entropy of a random vector," Problems Inf. Transmission, vol. 23, no. 9, pp. 95-101, 1987.
15. R. Battiti, "Using mutual information for selecting features in supervised neural net learning," IEEE Trans. Neural Netw., vol. 5, no. 4, pp. 537-550, Jul. 1994.
16. F. C. Pereira, N. Tishby, and W. Bialek, "The information bottleneck method," in 37th Annu. Allerton Conf. Commun., Control, Computing. Urbana, IL, 1999, pp. 368-377.
17. N. Rust, T. Sharpee, and W. Bialek, "Analyzing neural responses to natural signals: Maximally informative dimensions," Neural Comput. vol. 16, pp. 223-250, 2004.
18. J.-F. Cardoso, "Blind signal separation: Statistical principles," Proc. IEEE, vol. 86, no. 10, pp. 2009-2025, Oct. 1998.
19. J. Karhunen, A. Hyvarinen, and E. Oja, Independent Component Analysis New York: Wiley, 2001.
20. S. Haykin, Neural Networks: A comprehensive Foundation, 2nd ed. Englewood Cliffs, NJ: Prentice-Hall, 1999.
21. M. Welling, "Robust higher order statistics," in Proc. Int. Workshop Artif. Intell. Statist. (AISTATS), 2005, pp. 405-412.
22. A. Hyvarinen, "New approximations of differential entropy for independent component analysis," in Advances in Neural Information Processing System. Cambridge, MA: MIT Press, 1998, vol. 10, pp. 273-279.
23. J. Weston, S. Mukherjee, O. Chapelle, M. Pontil, T. Poggio, and V. Vapnik, "Feature selection for SVMs," in Advances in Neural Information Processing Systems. Cambridge, MA: MIT Press, 2001.
24. V. N. Vapnik Statistical Learning Theory. New York: Wiley, 1998.
25. L. Devroye, L. Györfi, and G. Lugosi, A Probabilistic Theory of Pattern Recognition. New York: Springer-Verlag, 1996.
26. J. M. Leiva and A. Artés, "A fixed-point algorithm for finding the optimal covariance matrix in kernel density modeling," in Int. Conf. Acoust., Speech, Signal Process., 2006, pp. V-705-V-708.
27. J. L. Marple, Digital Spectral Analysis With Applications. Englewood Cliffs, NJ: Prentice-Hall, 1987.
28. J. Rissanen, "Information theory and neural nets," in Mathematical Perspectives on Neural Networks, P. Smolensky, Ed. Upper Saddle River, NJ: Lawrence Erlbaum, 1993.