Adaptive learning in complex reproducing kernel hilbert spaces employing wirtinger's subgradients

IEEE Transactions on Neural Networks and Learning Systems

Volumn 23, Issue 3, 2012, Pages 425-438

  Bouboulis, Pantelis ^a   Slavakis, Konstantinos ^b   Theodoridis, Sergios ^c  

^a UNIVERSITY OF ATHENS   (Greece)

^b UNIVERSITY OF PELOPONNESE   (Greece)

^c University of Patras and Computer Technology Institute   (Greece)

Author keywords

Adaptive kernel learning; complex kernels; projection; subgradient; widely linear estimation; Wirtinger's calculus

Indexed keywords

ADAPTIVE KERNELS; COMPLEX KERNELS; PROJECTION; SUBGRADIENT; WIDELY LINEAR ESTIMATIONS; WIRTINGER'S CALCULUS;

CALCULATIONS; LEARNING ALGORITHMS; SIGNAL PROCESSING;

COMPUTATION THEORY;

EID: 84865223593     PISSN: 2162237X     EISSN: 21622388     Source Type: Journal    
DOI: 10.1109/TNNLS.2011.2179810     Document Type: Article

References (66)

1. B. Scholkopf and A. Smola, Learning with Kernels: Support Vector Machines, Regularization, Optimization and Beyond. Cambridge, MA: MIT Press, 2002.
2. S. Theodoridis and K. Koutroumbas, Pattern Recognition, 4th ed. New York: Academic, 2008.
3. V. Vapnik, The Nature of Statistical Learning Theory. New York: Springer-Verlag, 1999.
4. J. Shawe-Taylor and N. Cristianini, Kernel Methods for Pattern Analysis. Cambridge, U.K.: Cambridge Univ. Press, 2004.
5. W. Liu, J. C. Principe, and S. Haykin, Kernel Adaptive Filtering. New York: Wiley, 2010.
6. J. Kivinen, A. Smola, and R. C. Williamson, "Online learning with kernels," IEEE Trans. Sig. Process., vol. 52, no. 8, pp. 2165-2176, Aug. 2004.
7. Y. Engel, S. Mannor, and R. Meir, "The kernel recursive least-squares algorithm," IEEE Trans. Sig. Process., vol. 52, no. 8, pp. 2275-2285, Aug. 2004.
8. F. Pérez-Cruz and O. Bousquet, "Kernel methods and their potential use in signal processing," IEEE Sig. Process. Mag., vol. 21, no. 3, pp. 57-65, May 2004.
9. K. Slavakis, S. Theodoridis, and I. Yamada, "On line kernel-based classification using adaptive projection algorithms," IEEE Trans. Sig. Process., vol. 56, no. 7, pp. 2781-2796, Jul. 2008.
10. J. Xu, A. Paiva, I. Park, and J. Principe, "A reproducing kernel Hilbert space framework for information theoretic learning," IEEE Trans. Sig. Process., vol. 56, no. 12, pp. 5891-5902, Dec. 2008.
11. W. Liu, P. Pokharel, and J. C. Principe, "The kernel least-mean-square algorithm," IEEE Trans. Sig. Process., vol. 56, no. 2, pp. 543-554, Feb. 2008.
12. K. Slavakis, S. Theodoridis, and I. Yamada, "Adaptive constrained learning in reproducing kernel Hilbert spaces: The robust beamforming case," IEEE Trans. Sig. Process., vol. 57, no. 12, pp. 4744-4764, Dec. 2009.
13. F. Dufrenois, J. Colliez, and D. Hamad, "Bounded influence support vector regression for robust single-model estimation," IEEE Trans. Neural Netw., vol. 20, no. 11, pp. 1689-1706, Nov. 2009.
14. D. R. Musicant and A. Feinberg, "Active set support vector regression," IEEE Trans. Neural Netw., vol. 15, no. 2, pp. 268-275, Mar. 2004.
15. M. E. Mavroforakis, M. Sdralis, and S. Theodoridis, "A geometric nearest point algorithm for the efficient solution of the SVM classification task," IEEE Trans. Neural Netw., vol. 18, no. 5, pp. 1545-1549, Sep. 2007.
16. S. Theodoridis and M. Mavroforakis, "Reduced convex hulls: A geometric approach to support vector machines," IEEE Sig. Process. Mag., vol. 24, no. 3, pp. 119-122, May 2007.
17. D. Tzikas, A. C. Likas, and N. P. Galatsanos, "Sparse bayesian modeling with adaptive kernel learning," IEEE Trans. Neural Netw., vol. 20, no. 6, pp. 926-937, Jun. 2009.
18. P. Bouboulis, K. Slavakis, and S. Theodoridis, "Adaptive kernelbased image denoising employing semi-parametric regularization," IEEE Trans. Image Process., vol. 19, no. 6, pp. 1465-1479, Jun. 2010.
19. P. Bouboulis and S. Theodoridis, "Extension of Wirtinger's calculus to reproducing kernel Hilbert spaces and the complex kernel LMS," IEEE Trans. Sig. Process., vol. 59, no. 3, pp. 964-978, Mar. 2011.
20. B. Scholkopf, K. Tsuda, and J.-P. Vert, Kernel Methods in Computational Biology. Cambridge, MA: MIT Press, 2004.
21. G. Wachman, R. Khardon, P. Protopapas, and C. R. Alcock, "Kernels for periodic time series arising in astronomy," Machine Learn. Knowledge Discovery Databases Lecture Notes Comput. Sci., vol. 5782, pp. 489-505, Oct. 2009.
22. D. B. Ion, "Reproducing kernel Hilbert spaces and extremal problems for scattering of particles with arbitrary spins," Int. J. Theoretical Phys., vol. 24, no. 12, pp. 1217-1231, 1985.
23. H. Yoshino, C. Dong, Y. Washizawa, and Y. Yamashita, "Kernel wiener filter and its application to pattern recognition," IEEE Trans. Neural Netw., vol. 21, no. 11, pp. 1719-1730, Nov. 2010.
24. A. Tanaka, H. Imai, and M. Miyakoshi, "Kernel-induced sampling theorem," IEEE Trans. Sig. Process., vol. 58, no. 7, pp. 3569-3577, Jul. 2010.
25. N. Aronszajn, "Theory of reproducing kernels," Trans. Amer. Math. Soc., vol. 68, no. 3, pp. 337-404, May 1950.
26. J. Mercer, "Functions of positive and negative type and their connection with the theory of integral equation," Philoso. Trans. Royal Soc. London, vol. 209, pp. 415-446, May 1909.
27. D. Mandic and V. Goh, Complex Valued Nonlinear Adaptive Filters. New York: Wiley, 2009.
28. T. Adali and H. Li, Adaptive Signal Processing: Next Generation Solutions. New York: Wiley, 2010.
29. B. Picinbono and P. Chevalier, "Widely linear estimation with complex data," IEEE Trans. Sig. Process., vol. 43, no. 8, pp. 2030-2033, Aug. 1995.
30. B. Picinbono, "On circularity," IEEE Trans. Sig. Process., vol. 42, no. 12, pp. 3473-3482, Dec. 1994.
31. M. Novey and T. Adali, "On extending the complex ICA algorithm to noncircular sources," IEEE Trans. Sig. Process., vol. 56, no. 5, pp. 2148-2154, May 2008.
32. P. Chevalier and F. Pipon, "New insights into optimal widely linear receivers for the demodulation of BPSK, MSK and GMSK signals corrupted by noncircular interferences-application to SAIC," IEEE Trans. Sig. Process., vol. 54, no. 3, pp. 870-883, Mar. 2006.
33. J. J. Jeon, J. G. Andrews, and K. M. Sung, "The blind widely linear output energy algorithm for DS-CDMA systems," IEEE Trans. Sig. Process., vol. 54, no. 5, pp. 1926-1931, May 2006.
34. A. Aghaei, K. Plataniotis, and S. Pasupathy, "Widely linear MMSE receivers for linear dispersion space-time block-codes," IEEE Trans. Wireless Commun., vol. 9, no. 1, pp. 8-13, Jan. 2010.
35. J. Via, D. Ramirez, and I. Santamaria, "Properness and widely linear processing of quaternion random vectors," IEEE Trans. Informat. Theory, vol. 56, no. 7, pp. 3502-3515, Jul. 2010.
36. A. Cacciapuoti, G. Gelli, L. Paura, and F. Verde, "Finite-sample performance analysis of widely linear multiuser receivers for DS-CDMA systems," IEEE Trans. Sig. Process., vol. 56, no. 4, pp. 1572-1588, Apr. 2008.
37. A. S. Cacciapuoti, G. Gelli, L. Paura, and F. Verde, "Widely linear versus linear blind multiuser detection with subspace-based channel estimation: Finite sample-size effects," IEEE Trans. Sig. Process., vol. 57, no. 4, pp. 1426-1443, Apr. 2008.
38. K. Kuchi and V. Prabhu, "Performance evaluation for widely linear demodulation of PAM/QAM signals in the presence of Rayleigh fading and co-channel interference," IEEE Trans. Commun., vol. 57, no. 1, pp. 183-193, Jan. 2009.
39. J. Navarro-Moreno, J. Moreno-Kayser, R. Fernandez-Alcala, and J. Ruiz-Molina, "Widely linear estimation algorithms for second-order stationary signals," IEEE Trans. Sig. Process., vol. 57, no. 12, pp. 4930-4935, Dec. 2009.
40. R. Schober, W. Gerstacker, and L. H.-J. Lampe, "Data-aided and blind stochastic gradient algorithms for widely linear MMSE MAI suppression for DS-CDMA," IEEE Trans. Sig. Process., vol. 52, no. 3, pp. 746-756, Mar. 2004.
41. D. Mattera, L. Paura, and F. Sterle, "Widely linear MMSE equaliser for MIMO linear time-dispersive channel," Electron. Lett., vol. 39, no. 20, pp. 1481-1482, 2003.
42. F. Sterle, "Widely linear MMSE transceivers for MIMO channels," IEEE Trans. Sig. Process., vol. 52, no. 8, pp. 4258-4270, Aug. 2007.
43. K. C. Pun and T. Nguyen, "Widely linear filter bank equalizer for real STBC," IEEE Trans. Sig. Process., vol. 56, no. 9, pp. 4544-4548, Sep. 2008.
44. I. Yamada and N. Ogura, "Adaptive projected subgradient method for asymptotic minimization of sequence of nonnegative convex functions," Numer. Funct. Anal. Opt., vol. 25, nos. 7-8, pp. 593-617, 2004.
45. K. Slavakis, I. Yamada, and N. Ogura, "The adaptive projected subgradient method over the fixed point set of strongly attracting nonexpansive mappings," Numer. Funct. Anal. Opt., vol. 27, nos. 7-8, pp. 905-930, 2006.
46. K. Slavakis and I. Yamada, "The adaptive projected subgradient method constrained by families of quasi-nonexpansive mappings and its application to online learning," Jul. 2011, to be published.
47. H. H. Bauschke and P. L. Combettes, Convex Analysis and Monotone Operator Theory in Hilbert Spaces. New York: Springer-Verlag, 2011.
48. S. Haykin, Adaptive Filter Theory, 3rd ed. Englewood Cliffs, NJ: Prentice-Hall, 1996.
49. A. H. Sayed, Fundamentals of Adaptive Filtering. NJ: Wiley, 2003.
50. S. Theodoridis, K. Slavakis, and I. Yamada, "Adaptive learning in a world of projections: A unifying framework for linear and nonlinear classification and regression tasks," IEEE Sig. Process. Mag., vol. 28, no. 1, pp. 97-123, Jan. 2011.
51. P. J. Huber, Robust Statistics. New York: Wiley, 1981.
52. V. I. Paulsen. (Sep. 2009). An Introduction to the Theory of Reproducing Kernel Hilbert Spaces [Online]. Available: http://www.math.uh.edu/vern/rkhs.pdf
53. P. Bouboulis and M. Mavroforakis, "Reproducing kernel Hilbert spaces and fractal interpolation," J. Computat. Appl. Math., vol. 235, no. 12, pp. 3425-3434, Apr. 2011.
54. I. Steinwart, D. Hush, and C. Scovel, "An explicit description of the reproducing Kernel Hilbert spaces of Gaussian RBF kernels," IEEE Trans. Info. Theory, vol. 52, no. 10, pp. 4635-4643, Oct. 2006.
55. W. Wirtinger, "Zur formalen theorie der functionen von mehr complexen veranderlichen," Math. Annal., vol. 97, no. 1, pp. 357-375, 1927.
56. H. Li, "Complex-valued adaptive signal processing using Wirtinger calculus and its application to independent component analysis," Ph.D. dissertation, Dept. Comput. Sci. Electr. Eng., Univ. Maryland, Baltimore, MD, 2008.
57. T. Adali, H. Li, M. Novey, and J. F. Cardoso, "Complex ICA using nonlinear functions," IEEE Trans. Sig. Process., vol. 56, no. 9, pp. 4536-4544, Sep. 2008.
58. P. Bouboulis. Wirtinger's Calculus in General Hilbert Spaces [Online]. Available: http://arxiv.org/abs/1005.5170
59. R. T. Rockafellar and R. J.-B. Wets, Variational Analysis. Berlin: Springer, 2004.
60. K. Slavakis, P. Bouboulis, and S. Theodoridis, "Adaptive multiregression in reproducing kernel Hilbert spaces: The multiaccess MIMO channel case," IEEE Trans. Neural Netw., 2011, to be published.
61. R. T. Rockafellar, Convex Analysis Princeton Mathematics Series. Princeton, NJ: Princeton Univ. Press, 1970.
62. P. Bouboulis, S. Theodoridis, and M. Mavroforakis, "Widely linear kernel-based adaptive filters," in Proc. 19th Eur. Sig. Process. Conf., Barcelona, Spain, Aug.-Sep. 2011, pp. 1-5.
63. T. Hastie, R. Tibshirani, and J. Friedman, The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd ed. New York: Springer-Verlag, 2009.
64. A. H. Sayed, Fundamentals of Adaptive Filtering. Hoboken, NJ: Wiley, 2003.
65. G. S. Kimeldorf and G. Wahba, "Some results on Tchebycheffian spline functions," J. Math. Anal. Applict., vol. 33, no. 1, pp. 82-95, 1971.
66. K. Zarifi and A. B. Gershman, "Blind subspace-based signature waveform estimation in BPSK-modulated DS-CDMA systems with circular noise," IEEE Trans. Sig. Process., vol. 54, no. 9, pp. 3592-3602, Sep. 2006.