Stochastic gradient algorithm under (h, φ)-entropy criterion

Circuits, Systems, and Signal Processing

Volumn 26, Issue 6, 2007, Pages 941-960

  Chen, B ^a   Hu, J ^a   Pu, L ^a   Sun, Z ^a  

^a TSINGHUA UNIVERSITY   (China)

Author keywords

(H ) Entropy; Adaptive filtering; Kernel methods; Stochastic gradient algorithm

Indexed keywords

ADAPTIVE LINEAR NEURONS; ENTROPY CRITERIA; FINITE-IMPULSE RESPONSE; GAUSSIANS; GRADIENT ALGORITHM; KERNEL METHODS; NONPARAMETRIC ESTIMATORS; PERFORMANCE IMPROVEMENTS; SIMULATION EXPERIMENTS; STEP SIZE; STOCHASTIC GRADIENT ALGORITHMS; STOCHASTIC INFORMATION; UPPER BOUND;

ADAPTIVE FILTERING; ADAPTIVE FILTERS; ALGORITHMS; COMMUNICATION CHANNELS (INFORMATION THEORY); IMPULSE RESPONSE; STOCHASTIC SYSTEMS;

ENTROPY;

EID: 75149113421     PISSN: 0278081X     EISSN: 15315878     Source Type: Journal    
DOI: 10.1007/s00034-007-9004-9     Document Type: Article

References (17)

1. Cover, T.M., Thomas, J.A.: Elements of Information Theory. Wiley, Chichester (1991)
2. Douglas, S.C., Meng, H.Y.: Stochastic gradient adaptation under general error criteria. IEEE Trans. Signal Process. 42, 1335-1351 (1994)
3. Erdogmus, D., Principe, J.C.: Comparison of entropy and mean square error criteria in adaptive system training using higher order statistics. In: Second International Workshop on Independent Component Analysis and Blind Signal Separation, pp. 75-80 (2000)
4. Erdogmus, D., Principe, J.C.: Generalized information potential criterion for adaptive system training. IEEE Trans. Neural Netw. 13, 1035-1044 (2002)
5. Erdogmus, D., Principe, J.C.: Convergence properties and data efficiency of the minimum error entropy criterion in Adaline training. IEEE Trans. Signal Process. 51, 1966-1978 (2003)
6. Erdogmus, D., Kenneth, E.H., Principe, J.C.: Online entropy manipulation: stochastic information gradient. IEEE Signal Process. Lett. 10, 242-245 (2003)
7. Feng, X., Loparo, K.A., Fang, Y.: Optimal state estimation for stochastic systems: an information theoretic approach. IEEE Trans. Autom. Control 42(6), 771-785 (1997)
8. Gibson, J.D., Gray, S.D.: MVSE adaptive filtering subject to a constraint on MSE. IEEE Trans. Circuits Syst. 35(5), 603-608 (1988). May
9. Haykin, S.: Adaptive Filtering Theory, 3rd edn. Prentice-Hall, Upper Saddle River (1996)
10. Kaplan, D., Glass, L.: Understanding Nonlinear Dynamics. Springer, New York (1995)
11. Lo, J.T., Wanner, T.: Existence and uniqueness of risk-sensitivity estimates. IEEE Trans. Autom. Control 47(11), 1945-1948 (2002)
12. Menendez, M.L., Pardo, J.A., Pardo, M.C.: Estimators based on sample quantiles using (h, φ)-entropy measures. Appl. Math. Lett. 11, 99-104 (1998)
13. Pardo, L.: Statistical Inference Based on Divergence Measures. Chapman & Hall/CRC, Boca Raton (2006)
14. Salicru, M., Menendez, M.L., Morales, D., Pardo, L.: Asymptotic distribution of (h, φ)-entropies. Commun. Stat. Theory Methods 22, 2015-2031 (1993)
15. Sherman, S.: Non-mean-square error criteria. IRE Trans. Inf. Theory IT-4, 125-126 (1958). September
16. Silverman, B.W.: Density Estimation for Statistic and Data Analysis. Chapman & Hall, New York (1986)
17. Walach, E., Widrow, B.: The least mean fourth (LMF) adaptive algorithm and its family. IEEE Trans. Inf. Theory IT-30(2), 275-283 (1984). March