Quantized kernel least mean square algorithm

IEEE Transactions on Neural Networks and Learning Systems

Volumn 23, Issue 1, 2012, Pages 22-32

  Chen, Badong ^a   Zhao, Songlin ^a   Zhu, Pingping ^a   Principe, José C ^a  

^a University of Florida   (United States)

Author keywords

Kernel methods; mean square convergence; quantized kernel least mean square; vector quantization

Indexed keywords

CHAOTIC TIME SERIES PREDICTION; ENERGY CONSERVATION RELATION; EXCESS MEAN SQUARE ERROR; KERNEL ADAPTIVE FILTERING; KERNEL LEAST MEAN SQUARES; KERNEL METHODS; LOWER AND UPPER BOUNDS; MEAN SQUARE CONVERGENCES;

ADAPTIVE FILTERING; ALGORITHMS; RADIAL BASIS FUNCTION NETWORKS;

VECTOR QUANTIZATION;

EID: 84865065651     PISSN: 2162237X     EISSN: 21622388     Source Type: Journal    
DOI: 10.1109/TNNLS.2011.2178446     Document Type: Article

References (39)

1. B. Scholkopf and A. J. Smola, Learning with Kernels: Support Vector Machines, Regularization, Optimization and Beyond. Cambridge, MA: MIT Press, 2002.
2. V. Vapnik, The Nature of Statistical Learning Theory. New York: Springer-Verlag, 1995.
3. J. A. K. Suykens, T. Van Gestel, J. De Brabanter, B. De Moor, and J. Vandewalle, Least Squares Support Vector Machine. Singapore: World Scientific, 2002.
4. F. Girosi, M. Jones, and T. Poggio, "Regularization theory and neural networks architectures," Neural Comput., vol. 7, no. 2, pp. 219-269, Mar. 1995.
5. B. Scholkopf, A. J. Smola, and K. Muller, "Nonlinear component analysis as a kernel eigenvalue problem," Neural Comput., vol. 10, no. 5, pp. 1299-1319, Jul. 1998.
6. M. Espinoza, J. A. K. Suykens, and B. De Moor, "Fixed-size least squares support vector machines: A large scale application in electrical load forecasting," Comput. Manage. Sci., Special Issue Support Vector Mach., vol. 3, no. 2, pp. 113-129, Apr. 2006.
7. L. Hoegaerts, J. A. K. Suykens, J. Vandewalle, and B. De Moor, "Subset based least squares subspace regression in RKHS," Neurocomputing, vol. 63, pp. 293-323, Jan. 2005.
8. K. De Brabanter, J. De Brabanter, J. A. K. Suykens, and B. De Moor, "Optimized fixed-size kernel models for large data sets," Comput. Stat. Data Anal., vol. 54, no. 6, pp. 1484-1504, 2010.
9. J. Kivinen, A. J. Smola, and R. C. Williamson, "Online learning with kernels," IEEE Trans. Signal Process., vol. 52, no. 8, pp. 2165-2176, Aug. 2004.
10. J. Platt, "A resource-allocating network for function interpolation," Neural Comput., vol. 3, no. 2, pp. 213-225, Jun. 1991.
11. G. Huang, P. Saratchandran, and N. Sundararajan, "A generalized growing and pruning RBF (GGAP-RBF) neural network for function approximation," IEEE Trans. Neural Netw., vol. 16, no. 1, pp. 57-67, Jan. 2005.
12. K. Slavakis, S. Theodoridis, and I. Yamada, "Online kernel-based classification using adaptive projection algorithms," IEEE Trans. Signal Process., vol. 56, no. 7, pp. 2781-2796, Jul. 2008.
13. O. Dekel, S. Shalev-Shwartz, and Y. Singer, "The Forgetron: A kernelbased perceptron on a budget," SIAM J. Comput., vol. 37, no. 5, pp. 1342-1372, Jan. 2008.
14. W. Liu, J. Príncipe, and S. Haykin, Kernel Adaptive Filtering: A Comprehensive Introduction. New York: Wiley, 2010.
15. N. Aronszajn, "Theory of reproducing kernels," Trans. Amer. Math. Soc., vol. 68, no. 3, pp. 337-404, May 1950.
16. C. J. C. Burges, "A tutorial on support vector machines for pattern recognition," Data Min. Knowl. Disc., vol. 2, no. 2, pp. 121-167, 1998.
17. W. Liu, P. Pokharel, and J. Príncipe, "The kernel least-mean-square algorithm," IEEE Trans. Signal Process., vol. 56, no. 2, pp. 543-554, Feb. 2008.
18. P. Bouboulis and S. Theodoridis, "Extension of Wirtinger's calculus to reproducing kernel Hilbert spaces and the complex kernel LMS," IEEE Trans. Signal Process., vol. 59, no. 3, pp. 964-978, Mar. 2011.
19. W. Liu and J. Príncipe, "Kernel affine projection algorithms," EURASIP J. Adv. Signal Process., vol. 2008, no. 1, pp. 1-13, 2008.
20. Y. Engel, S. Mannor, and R. Meir, "The kernel recursive least-squares algorithm," IEEE Trans. Signal Process., vol. 52, no. 8, pp. 2275-2285, Aug. 2004.
21. W. Liu, I. Park, Y. Wang, and J. C. Príncipe, "Extended kernel recursive least squares algorithm," IEEE Trans. Signal Process., vol. 57, no. 10, pp. 3801-3814, Oct. 2009.
22. L. Csato and M. Opper, "Sparse on-line Gaussian processes," Neural Comput., vol. 14, no. 3, pp. 641-668, Mar. 2002.
23. W. Liu, I. Park, and J. C. Príncipe, "An information theoretic approach of designing sparse kernel adaptive filters," IEEE Trans. Neural Netw., vol. 20, no. 12, pp. 1950-1961, Dec. 2009.
24. D. Duttweiler, "Adaptive filter performance with nonlinearities in the correlation multiplier," IEEE Trans. Acoust., Speech, Signal Process., vol. 30, no. 4, pp. 578-586, Aug. 1982.
25. P. Xue and B. Liu, "Adaptive equalizer using finite-bit power-of-two quantizer," IEEE Trans. Acoust., Speech, Signal Process., vol. 34, no. 6, pp. 1603-1611, Dec. 1986.
26. W. Sethares and C. R. Johnson, "A comparison of two quantized state adaptive algorithms," IEEE Trans. Acoust., Speech, Signal Process., vol. 37, no. 1, pp. 138-143, Jan. 1989.
27. E. Eweda, "Convergence analysis and design of an adaptive filter with finite-bit power-of-two quantized error," IEEE Trans. Circuits Syst. II: Analog Digital Signal Process., vol. 39, no. 2, pp. 113-115, Feb. 1992.
28. J. C. M. Bermudez and N. J. Bershad, "Transient and tracking performance analysis of the quantized LMS algorithm for time-varying system identification," IEEE Trans. Signal Process., vol. 44, no. 8, pp. 1990-1997, Aug. 1996.
29. M. A. Aldajani, "Logarithmic quantization in the least mean squares algorithm," Digital Signal Process., vol. 18, no. 3, pp. 321-333, May 2008.
30. M. U. Otaru, A. Zerguine, and L. Cheded, "Channel equalization using simplified least mean-fourth algorithm," Digital Signal Process., vol. 21, no. 3, pp. 447-465, May 2011.
31. Y. Y. Linde, A. Buzo, and R. M. Gray, "An algorithm for vector quantizer design," IEEE Trans. Commun., vol. 28, no. 1, pp. 84-95, Jan. 1980.
32. P. A. Chou, T. Lookabaugh, and R. M. Gray, "Entropy-constrained vector quantization," IEEE Trans. Acoust., Speech, Signal Process., vol. 37, no. 1, pp. 31-42, Jan. 1989.
33. T. Lehn-Schiøler, A. Hegde, D. Erdogmus, and J. C. Príncipe, "Vector quantization using information theoretic concepts," Natural Comput., vol. 4, no. 1, pp. 39-51, 2005.
34. S. Craciun, D. Cheney, K. Gugel, J. C. Sanchez, and J. C. Príncipe, "Wireless transmission of neural signals using entropy and mutual information compression," IEEE Trans. Neural Syst. Rehabil. Eng., vol. 19, no. 1, pp. 35-44, Feb. 2011.
35. A. H. Sayed, Fundamentals of Adaptive Filtering. New York: Wiley, 2003.
36. N. R. Yousef and A. H. Sayed, "A unified approach to the steady-state and tracking analyses of adaptive filters," IEEE Trans. Signal Process., vol. 49, no. 2, pp. 314-324, Feb. 2001.
37. T. Y. Al-Naffouri and A. H. Sayed, "Adaptive filters with error nonlinearities: Mean-square analysis and optimum design," EURASIP J. Appl. Signal Process., vol. 2001, no. 4, pp. 192-205, 2001.
38. T. Y. Al-Naffouri and A. H. Sayed, "Transient analysis of datanormalized adaptive filters," IEEE Trans. Signal Process., vol. 51, no. 3, pp. 639-652, May 2003.
39. T. Y. Al-Naffouri and A. H. Sayed, "Transient analysis of adaptive filters with error nonlinearities," IEEE Trans. Signal Process., vol. 51, no. 3, pp. 653-663, Mar. 2003.