Continuously differentiable sample-spacing entropy estimation

IEEE Transactions on Neural Networks

Volumn 19, Issue 11, 2008, Pages 1978-1984

  Ozertem, Umut ^a   Uysal, Ismail ^b   Erdogmus, Deniz ^c  

^a YAHOO INC   (United States)

^b J Crayton Pruitt Family Department of Biomedical Engineering   (United States)

^c Northeastern University   (United States)

Author keywords

Entropy estimation; Minimum error entropy (MEE) criterion; Supervised neural network training

Indexed keywords

ADAPTIVE SYSTEMS; CLUSTER ANALYSIS; COMPUTATIONAL COMPLEXITY; DISCRIMINANT ANALYSIS; ERROR STATISTICS; IMAGE CLASSIFICATION; IMAGE SEGMENTATION; INFORMATION THEORY; LEARNING SYSTEMS; NEURAL NETWORKS; REGRESSION ANALYSIS; SIGNAL ANALYSIS; SIGNAL PROCESSING; STATISTICS; SYSTEMS ANALYSIS;

COMPUTATION TIMES; COMPUTATIONAL LOADS; COMPUTATIONALLY INEXPENSIVE; DESIRABLE PROPERTIES; ENTROPY ESTIMATION; ENTROPY ESTIMATIONS; ENTROPY ESTIMATORS; ERROR ENTROPIES; HIGHER ORDER STATISTICS; KERNEL DENSITY ESTIMATIONS; KERNEL ESTIMATORS; MACHINE LEARNINGS; MINIMUM ERROR ENTROPY (MEE) CRITERION; NONPARAMETRIC ENTROPY ESTIMATORS; OUTPUT ERRORS; PERFORMANCE COMPARISONS; SECOND-ORDER STATISTICS; SUPERVISED NEURAL NETWORK TRAINING; SUPERVISED NEURAL NETWORKS;

ENTROPY;

ALGORITHM; ARTICLE; ARTIFICIAL NEURAL NETWORK; AUTOMATED PATTERN RECOGNITION; COMPUTER SIMULATION; ENTROPY; METHODOLOGY; SAMPLE SIZE; STATISTICAL ANALYSIS; THEORETICAL MODEL;

ALGORITHMS; COMPUTER SIMULATION; DATA INTERPRETATION, STATISTICAL; ENTROPY; MODELS, THEORETICAL; NEURAL NETWORKS (COMPUTER); PATTERN RECOGNITION, AUTOMATED; SAMPLE SIZE;

EID: 56549107038     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/TNN.2008.2006167     Document Type: Article

References (21)

1. B. Widrow and S. D. Stearns, Adaptive Signal Processing. Englewood Cliffs, NJ: Prentice-Hall, 1985.
2. S. Haykin, Neural Networks: A Comprehensive Foundation, 2nd ed. Englewood Cliffs, NJ: Prentice-Hall, 1999.
3. A. H. Sayed, Fundamentals of Adaptive Filtering. New York: Wiley, 2003.
4. A. Hyvarinen, J. Karhunen, and E. Oja, Independent Component Analysis New York: Wiley, 2001.
5. A. Cichocki and S. I. Amari, Adaptive Blind Signal and Image Processing: Learning Algorithms and Applications. New York: Wiley, 2002.
6. D. Erdogmus and J. C. Principe, "An error-entropy minimization algorithm for supervised training of nonlinear adaptive systems," IEEE Trans. Signal Process., vol. 50, no. 7, pp. 1780-1786, Jul. 2002.
7. D. Erdogmus and J. C. Principe, "Generalized information potential criterion for adaptive system training," IEEE Trans. Neural Netw. vol. 13, no. 5, pp. 1035-1044, Sep. 2002.
8. C. E. Shannon and W. Weaver, The Mathematical Theory of Communication Urbana, IL: Univ. Illionis Press, 1964.
9. T. Cover and J. Thomas, Elements of Information Theory. New York: Wiley, 1991.
10. J. Berilant, E. J. Dudewicz, L. Gyorfi, and E. C. van der Meulen, "Nonparametric entropy estimation: An overview," Int. J. Math. Statist. Sci., vol. 6, no. 1, pp. 17-39, 1997.
11. E. Parzen, "On estimation of a probability density function and mode," Ann. Math. Statist., vol. 33, no. 3, pp. 1065-1076, 1962.
12. L. Devroye and G. Lugosi, Combinatorial Methods in Density Estimation New York: Springer-Verlag, 2001.
13. B. W. Silverman, Density Estimation for Statistics and Data Analysis London, U.K.: Champman & Hall, 1986.
14. D. Comaniciu, "An algorithm for data-driven bandwidth selection," IEEE Trans. Pattern Anal. Mach. Intell., vol. 25, no. 2, pp. 281-288, Feb. 2003.
15. E. G. Miller and J. W. Fisher, "ICA using spacings estimates of entropy," in Proc. 4th Int. Symp. ICA Blind Signal Separat., 2003, pp. 1047-1052.
16. O. Vasicek, "A test for normality based on sample entropy," J. Roy. Statist. Soc. B, vol. 38, no. 1, pp. 54-59, 1976.
17. A. Kraskov, H. Stogbauer, and P. Grassberger, "Estimating mutual information," Phys. Rev. E, Stat. Phys. Plasmas Fluids Relat. Interdiscip. Top., vol. 69, 2004, 066138.
18. D. Kaplan and L. Glass, Understanding Nonlinear Dynamics. New York: Springer-Verlag, 1995.
19. A. S. Weigend and N. A. Gershenfeld, Time Series Prediction: Forecasting the Future and Understanding the Past. Reading, MA: Addison-Wesley, 1994.
20. D. G. Luenberger, Linear and Nonlinear Programming. Reading, MA: Addison-Wesley, 1973.
21. S. Linkwitz and H. Grabert, "Energy diffusion of a weakly damped and periodically driven particle in an anharmonic potential well," Phys. Rev. B, Condens. Matter, vol. 44, no. 21, pp. 11888-11900, 1991.