Kernel methods and their potential use in signal processing

IEEE Signal Processing Magazine

Volumn 21, Issue 3, 2004, Pages 57-65

  Pérez Cruz, Fernando ^a   Bousquet, Olivier ^b  

^a UNIVERSITY COLLEGE LONDON   (United Kingdom)

^b MAX PLANCK INSTITUTE FOR BIOLOGICAL CYBERNETICS   (Germany)

Author keywords

[No Author keywords available]

Indexed keywords

DATA REDUCTION; LEARNING ALGORITHMS; LEARNING SYSTEMS; MARKOV PROCESSES; MATHEMATICAL TRANSFORMATIONS; MATRIX ALGEBRA; MAXIMUM LIKELIHOOD ESTIMATION; POLYNOMIALS; PRINCIPAL COMPONENT ANALYSIS; SPEECH RECOGNITION; TIME SERIES ANALYSIS;

KERNEL METHODS; SUPPORT VECTOR MACHINE; UNSUPERVISED LEARNING;

SIGNAL PROCESSING;

EID: 85032751903     PISSN: 10535888     EISSN: None     Source Type: Journal    
DOI: 10.1109/MSP.2004.1296543     Document Type: Article

References (50)

1. B. Boser, I. Guyon, and V.N. Vapnik, "A training algorithm for optimal margin classifiers," in Proc. 5th COLT, D. Haussler, Ed. New York: ACM Press, 1992, pp. 144-152.
2. V.N. Vapnik, Statistical Learning Theory. New York: Wiley, 1998.
3. N. Cristianini and J. Shawe-Taylor, An Introduction to Support Vector Machines. Cambridge: Cambridge University Press, 2000.
4. B. Schölkopf and A. Smola, Learning with Kernels. Cambridge, MA: MIT Press, 2002.
5. I. Steinwart, "On the influence of the kernel on the consistency of support vector machines," J. Mach. Learn. Res., vol. 2, pp. 67-93, 2001.
6. G.S. Kimeldorf and G. Wahba, "Some results in tchebycheffian spline functions," J. Math. Anal. Applicat., vol. 33, no. 1, pp. 82-95, 1971.
7. C.K.I. Williams and C.E. Rasmussen, "Advances in neural processing systems," in Advances in Neural Information Processing Systems, D.S. Touretzky, M.C. Mozer, and M.E. Hasselmo, Eds. Cambridge, MA: MIT Press, 1996, pp. 598-604.
8. A.N. Tikhonov and V.Y. Arsenin, Solution of Ill-Posed Problems. New York: Wiley, 1977.
9. V.N. Vapnik and A. Lerner, "Pattern recognition using generalized portrait method," Automat. Remote Contr., vol. 24, pp. 774-780, 1963.
10. V.N. Vapnik, Estimation of Dependences Based on Empirical Data. New York: Springer-Verlag, 1982.
11. B. Schölkopf, K.-K. Sung, C. Burges, F. Girosi, P. Niyogi, T. Poggio, and V. Vapnik, "Comparing support vector machines with Gaussian kernels to radial basis function classifiers," IEEE Trans. Signal Processing, vol. 45, no. 11, pp. 2758-2765, Nov. 1997.
12. E. Osuna, R. Freund, and F. Girosi, "Training support vector machines: An application to face detection," in Proc. IEEE Conf. CVPR, 1997, pp. 130-136
13. R. Fletcher, Practical Methods of Optimization, 2nd ed. Chichester, U.K.: Wiley, 1987.
14. F. Pérez-Cruz, P.L. Alarcón-Diana, A. Navia-Vázquez, and A. Artés-Rodríguez, "Fast training of support vector classifiers," in NIPS 13, T. Leen, T. Dietterich, and V. Tresp, Eds. Cambridge, MA: MIT Press, Nov. 2000, pp. 734-740.
15. V.N. Vapnik, S. Golowich, and A. Smola, "Support vector method for function approximation, regression estimation, and signal processing," in NIPS, M. Mozer, M. Jordan, and T. Petsche, Eds. Cambridge, MA: MIT Press, 1997, pp. 169-184.
16. H. Drunker, C. Burges, L. Kaufman, A. Smola, and V.N. Vapnik, "Support vector regression machines," in NIPS, M. Mozer, M. Jordan, and T. Petsche, Eds. Cambridge, MA: MIT Press, 1997, pp. 155-161.
17. J. Weston and C. Watkins, "Multi-class support vector machines," in Proc. ESANN, 1999, pp. 219-224.
18. E.L. Allwein, R.E. Schapire, and Y. Singer, "Reducing multiclass to binary: A unifying approach for margin classifiers," J. Mach. Learn. Res., pp. 113-141, 2000.
19. A. Elisseeff and J. Weston, "Kernel method for multi-labelled classification," in Advances in Neural Information Processing Systems 14, T.G. Dietterich, S. Becker, and Z. Ghahramani, Eds. Cambridge, MA: MIT Press, 2001, pp. 681-687.
20. F. Pérez-Gruz, G. Camps, E. Soria, J. Pérez, A.R. Figueiras-Vidal, and A. Artés-Rodríguez, "Multi-dimensional function approximation and regression estimation," in Proc. ICANN02, Madrid, Spain: Springer, 2002, pp. 757-762.
21. J.A.K. Suykens and J. Vandewalle, "Least squares support vector machine classifiers," Neural Process. Lett., vol. 9, no. 3, pp. 293-300, 1999.
22. B. Schölkopf, A. Smola, R. Williamson, and P.L. Bartlett, "New support vector algorithms," Neural Comput., vol. 12, no. 5, pp. 1207-1245, May 2000.
23. B. Schölkopf, A. Smola, and K.R. Müller, "Kernel principal component analysis," in Proc. ICANN97, Berlin, Germany: Springer, 1997, pp. 583-588.
24. J. Weston, O. Chapelle, A. Elisseeff, B. Schölkopf, and V. Vapnik, "Kernel dependency estimation," in Advances in Neural Information Processing Systems 15, S. Becker, S. Thrun, and K. Obermayer, Eds. Cambridge, MA: MIT Press, 2002, pp. 873-880.
25. B. Schölkopf, A.J. Smola, and K.-R. Müller, "Kernel principal component analysis," in Advances in Kernel Methods - Support Vector Learning, B. Schölkopf, C.J.C. Burges, and A.J. Smola, Eds. Cambridge, MA: MIT Press, 1999, pp. 327-352.
26. B. Schölkopf, R.C. Williamson, A.J. Smola, J. Shawe-Taylor, and J.C. Platt, "Support vector method for novelty detection," in NIPS 12, S.A. Solla, T.K. Leen, and K.-R. Müller, Eds. Cambridge, MA: MIT Press, 1999, pp. 582-588.
27. M. Tipping and B. Schölkopf, "A kernel approach for vector quantization with guaranteed distortion bounds," in Artificial Intelligence and Statistics, T. Jaakkola and T. Richardson, Eds. San Mateo, CA: Morgan Kaufmann, 2001, pp. 129-134.
28. F. Bach and M. Jordan, "Kernel independent component analysis," J. Mach. Learn. Res., vol. 3, pp. 1-48, 2002.
29. A. Gretton, R. Herbrich, and A. Smola. The kernel mutual information. [Online]. Available: http://www.kyb.mpg.de/publication.html?user=arthur
30. M. Kuss and T. Graepel. (2003, May). The geometry of kernel canonical correlation analysis. Max Planck Institute. [Online]. Available: http://www.kyb.mpg.de/publication.html?user=kuss
31. G. Wahba, "Spline models for observational data," CBMS-NFS Regional Conf. Series in Applied Mathematics, 1990, vol. 59, pp. xii-169.
32. O. Chapelle, V.N. Vapnik, O. Bousquet, and S. Mukherjee, "Choosing multiple parameters for support vector machines," Mach. Learn., vol. 46, no. 1, pp. 131-159, 2002.
33. T. Joachims, "Text categorization with support vector machines," Univ. of Dortmund, Informatik, AI-Unit Collaborative Research Center on Complexity Reduction in Multivariate Data, Tech. Rep. LS VIII, 23, 1997.
34. C. Leslie, E. Eskin, J. Weston, and W.S. Noble, "Mismatch string kernels for SVM protein classification," in NIPS 15, S. Becker, S. Thrun, and K. Obermayer, Eds. Cambridge, MA: MIT Press, 2002, pp. 1417-1424.
35. D. Haussler, "Convolutional kernels on discrete structures," Computer Science Dept., Univ. of Califonia at Santa Cruz, Tech. Rep. UCSC-CRL-99-10, 1999.
36. H. Lodhi, C. Saunders, J. Shawe-Taylor, N. Cristianini, and C. Watkins, "Text classification using string kernels," J. Mach. Learn. Res., pp. 419-444, 2002.
37. N. Cristianini, J. Shawe-Taylor, A. Elisseeff, and J. Kandola, "On kernel target alignment," in NIPS 14, T. G. Dietterich, S. Becker, and Z. Ghahramani, Eds. Cambridge, MA: MIT Press, 2001, pp. 367-373.
38. G.R.G. Lanckriet, N. Cristianini, P. Bartlett, L. El Ghaoui, and M.I. Jordan, "Learning the kernel matrix with semi-definite programming," in Proc. 19th ICML, Sidney, 2002, pp. 57-64.
39. L. Vandenberghe and S. Boyd, "Semidefinite programming," SIAM Rev., vol. 1, pp. 40-95, 1996.
40. C.S. Ong, A.J. Smola, and R.C. Williamson, "Supperkernels," in Advances in Neural Information Processing Systems 15, S. Becker, S. Thrun, and K. Obermayer, Eds. Cambridge, MA: MIT Press, 2002.
41. O. Bousquet and D.J.L. Herrmann, "On the complexity of learning the kernel matrix," in NIPS 15, S. Becker, S. Thrun, and K. Obermayer, Eds. Cambridge, MA: MIT Press, 2002, pp. 399-406.
42. N. Smith and M. Gales, "Speech recognition using SVMs," in NIPS 14, T.G. Dietterich, S. Becker, and Z. Ghahramani, Eds. Cambridge, MA: MIT Press, 2001, pp. 1197-1204.
43. V. Wan and S. Renals, "Evaluation of kernel methods for speaker verification and identification," in Proc. ICASSP 2002, vol. 1. Orlando, FL, 2002, pp. 669-672.
44. C.J.C. Burges, J.C. Platt, and S. Jana, "Extracting noise-robust features from audio data," in Proc. ICASSP 2002, vol. 1, Orlando, FL, 2002, pp. 1021-1024.
45. F. Pérez-Cruz, P. Alarcón-Diana, A. Navia-Vázquez, and A. Artés-Rodríguez, "SVC-based equalizer for burst TDMA transmissions," Signal Process., vol. 81, pp. 1681-1693, 2001.
46. S. Chakrabartty and G. Cauwenberghs, "Sequence estimation and channel equalization using forward decoding kernel machines," in Proc. ICASSP 2002, vol. III. Orlando, FL, 2002, pp. 2669-2672.
47. F. Pérez-Cruz and A. Artés-Rodríguez, "Adaptive SVC for nonlinear channel equalization," in Proc. EUSIPCO'02, Toulouse, France, 2002.
48. S. Chen, A.K. Samingan, and L. Hanzo, "Support vector machine multi-user receiver for DS-CDMA signal in multipath channels," IEEE Trans. Neural Networks, vol. 12, no. 3, pp. 604-611, 2001.
49. D. Mattera and S. Haykin, "Support vector machines for dynamic reconstruction of a chaotic system," in Advances in Kernel Methods - Support Vector Learning, B. Schölkopf, C.J.C. Burges, and A.J. Smola, Eds. Cambridge, MA: MIT Press, 1998, pp. 211-242.
50. F. Pérez-Cruz, J. Afonso-Rodríguez, and J. Giner, "Estimating GARCH models using support vector machines," Quantitative Finance, vol. 3, no. 3, pp. 163-172, 2003.