A recursive Renyi's entropy estimator

Neural Networks for Signal Processing - Proceedings of the IEEE Workshop

Volumn 2002-January, Issue , 2002, Pages 209-217

  Erdogmus, D ^a   Principe, J C ^a   Kim, Sung Phil ^a   Sanchez, J C ^b  

^a University of Florida   (United States)

^b UNIVERSITY OF FLORIDA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ENTROPY; ESTIMATION; INFORMATION THEORY;

ENTROPY ESTIMATE; ENTROPY ESTIMATOR; LEARNING SCENARIOS; NONSTATIONARY; OPTIMIZATION CRITERIA; QUADRATIC ENTROPY; RECURSIVE FORMULA; RENYI'S ENTROPY;

SIGNAL PROCESSING;

EID: 84953735880     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/NNSP.2002.1030032     Document Type: Conference Paper

References (15)

1. C.E. Shannon, "A mathematical theory of communication," Bell Sys. Tech. J., vol. 27, pp. 379-423,623-653, 1948.
2. X. Feng, K.A. Loparo, Y. Fang, "Optimal State Estimation for Stochastic Systems: An information Theoretic Approach," IEEE Transactions on Automatic Control, vol. 42, no. 6, pp. 771-785, 1997.
3. D. Erdogmus, J.C. Principe, "An Entropy Minimization Algorithm for Supervised Training of Nonlinear Adaptive Systems," to appear in IEEE Transactions on Signal Processing, 2002.
4. D. Erdogmus, J.C. Principe, "Generalized Information Potential Criterion for Adaptive System Training," to appear in IEEE Trans. on Neural Networks, 2002.
5. J.W. Fisher, Nonlinear Extensions to the Minimum Average Correlation Energy Filter, PhD Dissertation, University of Florida, 1997.
6. J. Atick, "Could Information Theory Provide an Ecological Theory of Sensory Processing?" Network, vol. 3, pp. 213-251, 1992.
7. J. Atick, A. Redlich, "Convergent Algorithms for Sensory Receptive Field Development," Neural Computation, vol. 5, pp. 45-60, 1993.
8. R. Linsker, "An Application of the Principle of Maximum Information Preservation to Linear Systems," in D.S. Tourezky (ed.), Morgan-Kauffman, San Francisco, 1988.
9. D. Erdogmus, J.C. Principe, "Convergence Analysis of the Information Potential Criterion in Adaline Training," Proceedings of Neural Networks for Signal Processing 2001 (NNSP XI), pp. 123-132, Falmouth, MA, Sep 2001.
10. D. Erdogmus, D. Rende, J.C. Principe, T.F. Wong, "Nonlinear Channel Equalization Using Multilayer Perceptrons with Information-Theoretic Criterion," Proceedings of Neural Networks for Signal Processing 2001 (NNSP XI), pp. 443-451, Falmouth, MA, Sep 2001.
11. D. Erdogmus, J.C. Principe, "An On-Line Adaptation Algorithm for Adaptive System Training with Minimum Error Entropy: Stochastic Information Gradient," in Proc. of Independent Component Analysis 2001 (ICA'01), 2001.
12. A. Renyi, Probability Theory, American Elsevier Pub. Co., New York, 1970.
13. E. Parzen, "On Estimation of a Probability Density Function and Mode,". in Time Series Analysis Papers, Holden-Day, Inc., California, 1967.
14. J.C. Principe, D. Xu, J. Fisher, "Information Theoretic Learning," in Unsupervised Adaptive Filtering, vol I, S. Haykin (ed.), Wiley, New York, 2000.
15. D. Erdogmus, Information Theoretic Learning: Renyi's Entropy and Its Applications to Adaptive System Training, PhD Dissertation, University of Florida, 2002.