Dimensionality reduction based on non-parametric mutual information

Neurocomputing

Volumn 80, Issue , 2012, Pages 31-37

  Faivishevsky, Lev ^a   Goldberger, Jacob ^a  

^a BAR ILAN UNIVERSITY   (Israel)

Author keywords

Classification; Dimensionality reduction; Regression; Semisupervised learning

Indexed keywords

DATA SETS; DIFFERENTIAL ENTROPY; DIMENSIONALITY REDUCTION; DIMENSIONALITY REDUCTION ALGORITHMS; INPUT AND OUTPUTS; INPUT SPACE; MUTUAL INFORMATIONS; NON-LINEAR REGRESSION; NON-PARAMETRIC; REDUCED-COMPLEXITY; REGRESSION; REGRESSION ALGORITHMS; SEMI-SUPERVISED CLASSIFICATION; SEMI-SUPERVISED LEARNING; STATE-OF-THE-ART PERFORMANCE; SUPERVISED LINEAR DIMENSIONALITY REDUCTION; THREE ORDERS OF MAGNITUDE;

CLASSIFICATION (OF INFORMATION); REGRESSION ANALYSIS;

ALGORITHMS;

ARTICLE; CLASSIFICATION; ENTROPY; INFORMATION PROCESSING; K NEAREST NEIGHBOR; MATHEMATICAL ANALYSIS; MULTIFACTOR DIMENSIONALITY REDUCTION; NONLINEAR REGRESSION ANALYSIS; NONPARAMETRIC TEST; PRIORITY JOURNAL;

EID: 84855229970     PISSN: 09252312     EISSN: 18728286     Source Type: Journal    
DOI: 10.1016/j.neucom.2011.07.028     Document Type: Article

References (21)

1. Hotelling H. Relations between two sets of variates. Biometrika 1936, 321-377.
2. Geladi P., Kowalski B.R. Partial least-squares regression: a tutorial. Anal. Chim. Acta 1986, 185:1-17.
3. Fukumizu K., Bach F., Jordan M.I. Dimensionality reduction for supervised learning with reproducing kernel Hilbert spaces. J. Mach. Learn. Res. 2004, 5:73-99.
4. A. Globerson, S. Roweis, Metric learning by collapsing classes, in: Advances in Neural Information Processing Systems, vol. 18, 2006.
5. K. Weinberger, J. Blitzer, L. Saul, Distance metric learning for large margin nearest neighbor classification, in: Advances in Neural Information Processing Systems, vol. 18, 2006.
6. Rossi F., Lendasse A., François D., Wertz V., Verleysen M. Mutual information for the selection of relevant variables in spectrometric nonlinear modelling. Chemometrics Intell. Lab. Syst. 2007, 215-226.
7. Torkkola K. Feature extraction by non-parametric mutual information maximization. J. Mach. Learn. Res. 2003, 1415-1438.
8. K.D. Bollacker, J. Ghosh, Mutual information feature extractors for neural classifiers, in: Proceedings of the International Conference on Neural Networks (ICNN '96), 1996, pp. 1528-1533.
9. Leiva-Murillo J.M., Artes-Rodriguez A. Maximization of mutual information for supervised linear feature extraction. IEEE Trans. Neural Networks 2007, 1433-1441.
10. L. Faivishevsky, J. Goldberger, ICA based on a smooth estimation of the differential entropy, in: Advances in Neural Information Processing Systems, vol. 21, 2009.
11. Kozachenko L., Leonenko N. On statistical estimation of entropy of random vector. Probl. Inf. Transm. 1987, 23(2):95-101.
12. Victor J.D. Binless strategies for estimation of information from neural data. Phys. Rev. E 2002, 051903.
13. Wang Q., Kulkarni S.R., Verdu S. Divergence estimation for multidimensional densities via k-nearest-neighbor distances. IEEE Trans. Inf. Theory 2009, 2392-2405.
14. Singh H., Misra N., Hnizdo V., Fedorowicz A., Demchuk E. Nearest neighbor estimates of entropy. Am. J. Math. Manage. Sci. 2003, 301-321.
15. Friedman J.H., Tukey J.W. A projection pursuit algorithm for exploratory data analysis. IEEE Trans. Comput. 1974, 881-890.
16. Fisher R.A. The use of multiple measurements in taxonomic problems. Annu. Eugen. 1936, 179-188.
17. J. Peltonen, J. Goldberger, S. Kaski, Fast semi-supervised discriminative component analysis, in: IEEE Workshop on Machine Learning for Signal Processing, 2007.
18. L. Faivishevsky, J. Goldberger, A nonparametric information theoretic clustering algorithm, in: Proceedings of the 27th International Conference on Machine Learning, 2010.
19. Rasmussen C.E., Williams C.K.I. Gaussian Processes for Machine Learning 2006, The MIT Press, Cambridge, MA, USA.
20. A. Asuncion, D.J. Newman, UCI Machine Learning Repository, 2007.
21. J. Goldberger, S. Roweis, G. Hinton, R. Salakhutdinov, Neighbourhood components analysis, in: Advances in Neural Information Processing Systems, vol. 16, 2004.