Analyzing local structure in kernel-based learning: Explanation, complexity, and reliability assessment

IEEE Signal Processing Magazine

Volumn 30, Issue 4, 2013, Pages 62-74

  Montavon, Gregoire ^a   Braun, Mikio L ^a   Krueger, Tammo ^a   Muller, Klaus Robert ^a  

^a TECHNISCHE UNIVERSITÄT BERLIN   (Germany)

Author keywords

[No Author keywords available]

Indexed keywords

LEARNING SYSTEMS; NONLINEAR ANALYSIS;

DIMENSION ESTIMATION; KERNEL-BASED LEARNING; LEARNING MACHINES; NON-LINEAR PREDICTIONS; NON-LINEAR SIGNAL PROCESSING; NONLINEAR KERNELS; RELIABILITY ASSESSMENTS; SIGNAL PROCESSING APPLICATIONS;

SIGNAL PROCESSING;

EID: 85032751251     PISSN: 10535888     EISSN: None     Source Type: Journal    
DOI: 10.1109/MSP.2013.2249294     Document Type: Article

References (47)

1. V. N. Vapnik, The Nature of Statistical Learning Theory. New York: Springer-Verlag, 1995.
2. B. Schölkopf, S. Mika, C. J. C. Burges, P. Knirsch, K.-R. Müller, G. Rätsch, and A. J. Smola, "Input space versus feature space in kernel-based methods, " IEEE Trans. Neural Networks, vol. 10, no. 5, pp. 1000-1017, 1999.
3. K.-R. Müller, S. Mika, G. Rätsch, K. Tsuda, and B. Schölkopf, "An introduction to kernel-based learning algorithms, " IEEE Trans. Neural Networks, vol. 12, no. 2, pp. 181-202, 2001.
4. B. Schölkopf and A. J. Smola, Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond. Cambridge, MA: MIT Press, 2002.
5. J. C. Principe, Information Theoretic Learning: Renyi's Entropy and Kernel Perspectives, 1st ed. New York: Springer, 2010.
6. S. Theodoridis and K. Koutroumbas, Pattern Recognition. New York: Academic, 2009.
7. V. D. Calhoun, T. Adali, G. D. Pearlson, and J. J. Pekar, "A method for making group inferences from functional MRI data using independent component analysis, " Human Brain Mapping, vol. 14, no. 3, pp. 140-151, Nov. 2001.
8. R. Jenssen, "Kernel entropy component analysis, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 32, no. 5, pp. 847-860, 2010.
9. S. S. Haykin, Neural Networks and Learning Machines. Englewood Cliffs, NJ: Prentice Hall, 2009.
10. A. Zien, G. Rätsch, S. Mika, B. Schölkopf, T. Lengauer, and K.-R. Müller, "Engineering support vector machine kernels that recognize translation initiation sites, " Bioinformatics, vol. 16, no. 9, pp. 799-807, 2000.
11. M. P. Brown, W. N. Grundy, D. Lin, N. Cristianini, C. W. Sugnet, T. S. Furey, M. Ares, and D. Haussler, "Knowledge-based analysis of microarray gene expression data by using support vector machines, " in Proc. Natl. Acad. Sci. U. S. A., 2000, vol. 97, no. 1, pp. 262-267.
12. A. Zien, J. Fluck, R. Zimmer, and T. Lengauer, "Microarrays: How many do you need?" in Proc. 6th Annu. Int. Conf. Computational Biology, RECOMB '02, ACM , pp. 321-330.
13. B. Blankertz, G. Dornhege, M. Krauledat, K. R. Müller, and G. Curio, "The noninvasive Berlin Brain-Computer Interface: fast acquisition of effective performance in untrained subjects, " NeuroImage, vol. 37, no. 2, pp. 539-550, 2007.
14. J. D. Haynes and G. Rees, "Decoding mental states from brain activity in humans, " Nat. Rev. Neurosci., vol. 7, no. 7, pp. 523-34, 2006.
15. J. C. Snyder, M. Rupp, K. Hansen, K.-R. Müller, and K. Burke, "Finding density functionals with machine learning, " Phys. Rev. Lett., vol. 108, no. 25, p. 253002, June 2012.
16. M. Rupp, A. Tkatchenko, K.-R. Müller, and O. A. von Lilienfeld, "Fast and accurate modeling of molecular atomization energies with machine learning, " Phys. Rev. Lett., vol. 108, no. 5, p. 058301, Jan. 2012.
17. S. Sonnenburg, A. Zien, P. Philips, and G. Rätsch, "Poims: Positional oligomer importance matrices-understanding support vector machine-based signal detectors, " Bioinformatics, vol. 24, no. 13, pp. i6-i14, 2008.
18. I. Guyon and A. Elisseeff, "An introduction to variable and feature selection, " J. Mach. Learn. Res., vol. 3, pp. 1157-1182, Mar. 2003.
19. B. Schölkopf, A. Smola, and K.-R. Müller, "Nonlinear component analysis as a kernel eigenvalue problem, " Neural Comput., vol. 10, no. 5, pp. 1299-1319, 1998.
20. M. L. Braun, "Accurate bounds for the eigenvalues of the kernel matrix, " J. Mach. Learn. Res., vol. 7, pp. 2303-2328, Nov. 2006.
21. M. L. Braun, J. Buhmann, and K.-R. Müller, "On relevant dimensions in kernel feature spaces, " J. Mach. Learn. Res., vol. 9, pp. 1875-1908, Aug. 2008.
22. D. Baehrens, T. Schroeter, S. Harmeling, M. Kawanabe, K. Hansen, and K.-R. Müller, "How to explain individual classification decisions, " J. Mach. Learn. Res., vol. 11, pp. 1803-1831, June 2010.
23. A. J. Smola and P. J. Bartlett, Eds., Advances in Large Margin Classifiers. Cambridge, MA: MIT Press, 2000.
24. S. Mika, G. Rätsch, J. Weston, B. Schölkopf, and K. R. Müller, "Fisher discriminant analysis with kernels, " in Proc. 1999 IEEE Signal Processing Society Workshop, Neural Networks for Signal Processing IX, Copenhagen, Denmark, pp. 41-48.
25. S. Mika, G. Rätsch, J. Weston, B. Schölkopf, A. Smola, and K.-R. Müller, "Constructing descriptive and discriminative nonlinear features: Rayleigh coefficients in kernel feature spaces, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 25, no. 5, pp. 623-633, 2003.
26. M. Sugiyama, "Dimensionality reduction of multimodal labeled data by local Fisher discriminant analysis, " J. Mach. Learn. Res., vol. 8, pp. 1027-1061, May 2007.
27. F. R. Bach and M. I. Jordan, "Kernel independent component analysis, " J. Mach. Learn. Res., vol. 3, pp. 1-48, July 2002.
28. S. Harmeling, A. Ziehe, M. Kawanabe, and K.-R. Müller, "Kernel-based nonlinear blind source separation, " Neural Comput., vol. 15, no. 5, pp. 1089-1124, 2003.
29. K. Fukumizu, F. R. Bach, and A. Gretton, "Statistical consistency of kernel canonical correlation analysis, " J. Mach. Learn. Res., vol. 8, pp. 361-383, Feb. 2007.
30. F. Bießmann, F. C. Meinecke, A. Gretton, A. Rauch, G. Rainer, N. K. Logothetis, and K.-R. Müller, "Temporal kernel CCA and its application in multimodal neuronal data analysis, " Mach. Learn., vol. 79, no. 1-2, pp. 5-27, 2010.
31. L. Zwald and G. Blanchard, "On the convergence of eigenspaces in kernel principal component analysis, " in Advances in Neural Information Processing Systems 18. Cambridge, MA: MIT Press, 2006.
32. J. Shawe-Taylor, C. K. I. Williams, N. Cristianini, and J. S. Kandola, "On the eigenspectrum of the gram matrix and the generalization error of kernel-PCA, " IEEE Trans. Inform. Theory, vol. 51, no. 7, pp. 2510-2522, 2005.
33. C. E. Rasmussen and C. K. I. Williams, Gaussian Processes for Machine Learning (Adaptive Computation and Machine Learning Series). Cambridge, MA: MIT Press, 2005.
34. S. Geman, E. Bienenstock, and R. Doursat, "Neural networks and the bias/variance dilemma, " Neural Comput., vol. 4, no. 1, pp. 1-58, 1992.
35. B. Efron and R. Tibshirani, An Introduction to the Bootstrap. Boca Raton, FL: CRC Press, 1994.
36. K. Kersting, C. Plagemann, P. Pfaff, and W. Burgard, "Most likely heterscedastic Gaussian process regression, " in Proc. Int. Conf. Machine Learning, pp. 393-400, 2007.
37. M. Lázaro-Gredilla and M. Titsias, "Variational heteroscedastic Gaussian process regression, " in Proc. 28th Int. Conf. Machine Learning (ICML 2011), pp. 841-848.
38. I. T. Jolliffe, Principal Component Analysis, 2nd ed. New York: Springer-Verlag, 2002.
39. K. I. Kim, M. O. Franz, and B. Schölkopf, "Iterative kernel principal component analysis for image modeling, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 27, no. 9, pp. 1351-1366, 2005.
40. S. Günter, N. N. Schraudolph, and S. V. N. Vishwanathan, "Fast iterative kernel principal component analysis, " J. Mach. Learn. Res., vol. 8, pp. 1893-1918, Aug. 2007.
41. I. Guyon and A. Elisseeff, "An introduction to variable and feature selection, " J. Mach. Learn. Res., vol. 3, pp. 1157-1182, Mar. 2003.
42. J. C. Platt, "Probabilities for SV machines, " in Advances in Large Margin Classifiers. Cambridge, MA: MIT Press, 2000.
43. Z. D. Pozun, K. Hansen, D. Sheppard, M. Rupp, K.-R. Müller, and G. Henkelman, "Optimizing transition states via kernel-based machine learning, " J. Chem. Phys., vol. 136, no. 17, p. 174101, 2012.
44. O. L. Mangasarian and W. H. Wolberg, "Cancer diagnosis via linear programming, " SIAM News, vol. 23, no. 5, pp. 1-18, 1990.
45. G. Montavon, K. Hansen, S. Fazli, M. Rupp, F. Biegler, A. Ziehe, A. Tkatchenko, O. A. von Lilienfeld, and K.-R. Müller, "Learning invariant representations of molecules for atomization energy prediction, " in Proc. NIPS, 2012, Lake Tahoe, NV.
46. G. Montavon, M. Braun, and K.-R. Müller, "Kernel analysis of deep networks, " J. Mach. Learn. Res., vol. 12, pp. 2563-2581, Sept. 2011.
47. R. A. Horn and C. R. Johnson, Matrix Analysis. Cambridge, U. K. : Cambridge Univ. Press, 1985.