An estimate of mutual information that permits closed-form optimisation

Entropy

Volumn 15, Issue 5, 2013, Pages 1690-1704

  Liu, Raymondd ^a   Gillies Duncan, Abdul ^a  

^a IMPERIAL COLLEGE LONDON   (United Kingdom)

Author keywords

Dimensionality reduction; Feature extraction; Machine learning; Mutual information; Pattern recognition

Indexed keywords

EID: 84878366601     PISSN: None     EISSN: 10994300     Source Type: Journal    
DOI: 10.3390/e15051690     Document Type: Article

References (16)

1. Guyon, I.; Gunn, S.; Nikravesh, M.; Zadeh, L. Feature Extraction: Foundations and Applications; Springer: Berlin/Heidelberg, Germany, 2006; Volume 207.
2. Van der Maaten, L.; Postma, E.; van den Herik, H. Dimensionality reduction: A comparative review. J. Mach. Learn. Res. 2009, 10, 1-41.
3. Torkkola, K. Feature extraction by non parametric mutual information maximization. J. Mach. Learn. Res. 2003, 3, 1415-1438.
4. Kapur, J. Measures of Information and Their Applications; Wiley: New Delhi, India, 1994.
5. Bishop, C.M. Pattern Recognition and Machine Learning; Springer-Verlag New York, Inc.: Secaucus, NJ, USA, 2006.
6. Qiu, X.; Wu, L. Info-margin maximization for feature extraction. Pattern Recognit. Lett. 2009, 30, 1516-1522.
7. Hild, K.; Erdogmus, D.; Torkkola, K.; Principe, J. Feature extraction using information-theoretic learning. Pattern Anal. Mach. Intell. IEEE Trans. 2006, 28, 1385-1392.
8. Liu, R.; Gillies, D.F. An Eigenvalue-problem Formulation for Non-parametric Mutual Information Maximisation for Linear Dimensionality Reduction. In Proceedings of the International Conference on Image Processing, Computer Vision, and Pattern Recognition, Las Vegas, USA, 16-19 July 2012; Volume 2, pp. 905-910.
9. Shannon, C. A mathematical theory of communication. Bell Syst. Tech. J. 1948, Volume 27, pp. 379-423, 623-656.
10. Cover, T.; Thomas, J. Elements of Information Theory; Wiley-interscience: Hoboken, NJ, USA, 2006.
11. Renyi A. On Measures of Entropy and Information; University of California Press: Berkeley, CA, USA, 1961; Volume 1, pp. 547-561.
12. Parzen, E. On estimation of a probability density function and mode. Ann. Math. Stat. 1962, 33, 1065-1076.
13. Belkin, M.; Niyogi, P. Laplacian eigenmaps for dimensionality reduction and data representation. Neural Comput. 2003, 15, 1373-1396.
14. Sharpee, T.; Rust, N.C.; Bialek, W. Analyzing neural responses to natural signals: Maximally informative dimensions. Neural Comput. 2004, 16, 223-250.
15. Lawrence, N. Spectral Dimensionality Reduction via Maximum Entropy. In Proceedings of the International Conference on Artificial Intelligence and Statistics, Fort Lauderdale, FL, USA, 11-13 April 2011; pp. 51-59.
16. Jain, A.; Duin, R.; Mao, J. Statistical pattern recognition: A review. Pattern Anal. Mach. Intell. IEEE Trans. 2000, 22, 4-37.