An Improved Training Algorithm for Nonlinear Kernel Discriminants

IEEE Transactions on Signal Processing

Volumn 52, Issue 10, 2004, Pages 2798-2806

  Abdallah, Fahed ^a   Richard, Cedric ^a   Lengellé, Regis ^a  

^a UNIVERSITÉ DE TECHNOLOGIE DE TROYES   (France)

Author keywords

Kernel Fisher discriminant; learning machine; second order criteria; support vector machines

Indexed keywords

EID: 32544443001     PISSN: 1053587X     EISSN: 19410476     Source Type: Journal    
DOI: 10.1109/TSP.2004.834346     Document Type: Article

References (29)

1. K. R. Muller, S. Mika, G. Ratsch, K. Tsuda, and B. Scholkopf, “An introduction to kernel-based learning algorithms”, IEEE Trans. Neural Networks, vol. 12, pp. 181-201, May 2001.
2. B. Boser, I. Guyon, and V. Vapnik, “A training algorithm for optimal margin classifiers”, in Proc. Fifth Annual Workshop Computational Learning Theory, Pittsburgh, PA, 1992, pp. 144-152.
3. C. Cortes and V. Vapnik, “Support vector networks”, Machine Learning, vol. 20, pp. 273-297, 1995.
4. V. Vapnik, The Nature of Statistical Learning Theory. New York: Springer Verlag, 1995.
5. S. Mika, G. Ratsch, J. Weston, B. Scholkopf, and K. R. Muller, “Fisher discriminant analysis with kernels”, Adv. Neural Networks Signal Process., pp. 41-48, 1999.
6. J. Rissanen, “Stochastic complexity in statistical inquiry”, Ann. Statist., vol. 14, no. 3, pp. 1080-1100, 1986.
7. Y. LeCun, J. S. Denker, and S. A. Solla, “Optimal brain damage”, in Proc. Advances Neural Inform. Process. Syst., vol. 2, 1990, pp. 598-605.
8. W. A. Gardner, “A unifying view of second-order measures of quality for signal classification”, IEEE Trans. Commun., vol. COM-28, pp. 807-816, June 1980.
9. C. Richard, R. Lengelle, and F. Abdallah, “Bayes-optimal detectors design using relevant second-order criteria”, IEEE Signal Processing Lett., vol. 9, Jan. 2002.
10. K. Fukunaga, Statistical Pattern Recognition. San Diego, CA: Academic, 1990.
11. L. Devroye, L. Gyorfi, and G. Lugosi, A Probabilistic Theory of Pattern Recognition. New York: Springer-Verlag, 1996.
12. D.W. Patterson and R. L. Mattson, “A method of finding linear discriminant functions for a class of performance criteria”, IEEE Trans. Inform. Theory, vol. IT-12, pp. 380-387, June 1966.
13. C. Richard, “Une methodologie pour la detection a structure imposee”, Ph.D. dissertation, Compiegne Univ. Technol., Compiegne, France, 1998.
14. F. Abdallah, C. Richard, and R. Lengelle, “On equivalence between detectors obtained from second-order measures of performance”, in Proc. XI European Signal Process. Conf., Toulouse, France, Sept. 3-6, 2002.
15. S. Mika, G. Ratsch, and K. R. Muller, “A mathematical programming approach to the kernel Fisher algorithm”, in Proc. Advances Neural Inform. Process. Syst., T. K. Leen, T. G. Dietterich, and V. Tresp, Eds. Cambridge, MA: MIT Press, 2001, pp. 591-597.
16. S. Mika, A. J. Smola, and B. Scholkopf, “An improved training algorithm for kernel Fisher discriminants”, in Proc. AISTATS, T. Jaakkola and T. Richardson, Eds. San Francisco, CA: Morgan Kaufmann, 2001, pp. 98-104.
17. S. A. Billings and K. L. Lee, “Nonlinear Fisher discriminant analysis using a minimum squared error cost function and the orthogonal least squares algorithm”, Neural Networks, vol. 15, pp. 263-270, 2002.
18. R. Courant and D. Hilbert, Methods of Mathematical Physics. New York: Wiley, 1953.
19. B. Efron, “Bootstrap methods: another look at jacknife”, Ann. Statist., vol. 7, pp. 1-26, 1979.
20. J. Friedman, “Regularized discriminant analysis”, J. Amer. Statist. Assoc., vol. 84, no. 405, pp. 165-175, 1989.
21. V. Vapnik and A.Y. Chervonenkis, “On the uniform convergence of relative frequencies of events to their probabilities”, Theory Probability Applicat., vol. 16, pp. 264-280, 1971.
22. G. H. Golub and C. F. Van Loan, Matrix Computations. Baltimore, MD: Johns Hopkins Univ. Press, 1993.
23. V. Vapnik and A. Y. Chervonenkis, Theory of Pattern Recognition (in Russian). Moscow, Russia: Nauka, 1974.
24. R. O. Duda, P. E. Hart, and D. C. Stork, Pattern Classification. New York: Wiley, 2001.
25. P. Lachenbruch and M. Mickey, “Estimation of error rates in discriminant analysis”, Technometr., vol. 10, pp. 1-11, 1968.
26. K. J. Lang and M. J. Witbrock, “Learning to tell two spirals apart”, in Proc. Connectionnist Summer Schools. San Francisco, CA: Morgan Kaufmann, 1988.
27. A. N. Tikhonov and V. Y. Arsenin, Solutions of Ill-Posed Problems. New York: Wiley, 1977.
28. M. Tipping, “The relevance vector machine”, Adv. Neural Inform. Process. Syst., vol. 12, pp. 652-658, 2000.
29. C. Hansen, Rank-Deficient and Ill-Posed Problems: Numerical Aspects of Linear Inversion, PA: SIAM, 1998.