Learning from examples with Renyi's information criterion

Conference Record of the 33rd Asilomar Conference on Signals, Systems, and Computers

Volumn 2, Issue , 1999, Pages 966-970

  Principe, J C ^a   Xu, Dongxin ^a  

^a UNIVERSITY OF FLORIDA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ENTROPY; COMPUTATIONAL GEOMETRY; LEARNING SYSTEMS; MATHEMATICAL MODELS; PROBABILITY DENSITY FUNCTION;

DISTANCE MEASURE; INFORMATION CRITERION; INFORMATION THEORETIC CRITERION; LEARNING FROM EXAMPLES; MUTUAL INFORMATIONS; NOVEL ALGORITHM; SCHWARTZ INEQUALITY; TRAINING SETS;

INFORMATION THEORY;

QUADRATIC ENTROPY; QUADRATIC MUTUAL INFORMATION; RENYI'S INFORMATION CRITERION;

EID: 0033324712     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ACSSC.1999.831853     Document Type: Conference Paper

References (14)

1. Linsker R. "An application of the principle of maximum information preservation to linear systems", in Advances in Neural Information Processing Systems 1, Morgan-Kaufman, pp 485-494, 1988.
2. Deco, G. An Information-Theoretic Approach to Neural Computing, New York, Springer, 1996
3. Plumbley M., Fallside F., "An information theoretic approach to unsupervised networks", Int. J. Conf. on Neural Nets, vol 2, p 598, Washington, DC, 1989.
4. Fisher J. W. 111 "Nonlinear Extensions to the Minimum Average Correlation Energy Filter" Ph.D dissertation, Dept. of ECE, University of Florida, 1997.
5. Xu Dongxin. "Feature Extraction: From Decorrelation to Statistical Independence" Ph.D Dissertation, Dept. of ECE, University of Florida, 1999.
6. Xu D., Principe J., Fisher J. and Wu H-C. "A Novel Measure for Independent Component Analysis (ICA)" in Proc. ICASSP98, VOI 11, 1161-1164, 1998.
7. Renyi, A. "Some Fundamental Questions of Information Theory", Selected Papers of Alfred Renyi, V01.2, 1976.
8. Kapur, J.N. "Measures of Information and Their Applications". John Wiley & Sons. I994
9. Rumelhart, D.E., Hinton, G.E. and Williams, J.R. "Learning representations by back-propagating errors", Nature, 323. pp533-536, 1986.
10. Fisher J., Principe J. "Entropy manipulation of arbitrary nonlinear mappings", Proc. IEEE Wokshop Neural Nets for Signal Proc., 14-23. Amelia Island, 1997.
11. Principe J., Xu D., Fisher J., "Pose estimationin SAR using an information theoretic Criterion", Proc. SPIE Conf. vol 3370, 218-229, Orlando, 1998.
12. Parzen. E. "On the estimation of a probability density function and the mode", Ann. Math. Stat. 33, 1962, p1065
13. Viola P. Schraudolph N., Sejnowski T., "Empirical entropy manipulation for real-world problems", Proc. Neural Information Processing Systems NIPS 8, 851-857. 1995.
14. Principe J., Xu D. Fisher J., "Information THeoretic Learning", in Unsupervised Adaptive Filtering, Wiley, in Press, 1999.