Fast iterative kernel principal component analysis

Journal of Machine Learning Research

Volumn 8, Issue , 2007, Pages 1893-1918

  Günter, Simon ^a   Schraudolph, Nicol N ^a   Vishwanathan, S V N ^a  

^a AUSTRALIAN NATIONAL UNIVERSITY   (Australia)

Author keywords

Gain vector adaptation; Kernel hebbian algorithm; Online learning; Step size adaptation; Stochastic meta descent

Indexed keywords

ALGORITHMS; CONVERGENCE OF NUMERICAL METHODS; EIGENVALUES AND EIGENFUNCTIONS; HILBERT SPACES; ITERATIVE METHODS; OPTICAL RESOLVING POWER;

GAIN VECTOR ADAPTATION; KERNEL HEBBIAN ALGORITHMS; ONLINE LEARNING; STEP SIZE ADAPTATION; STOCHASTIC META-DESCENT;

PRINCIPAL COMPONENT ANALYSIS;

EID: 34548170925     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (28)

1. Stephen Boyd and Lieven Vandenberghe. Convex Optimization. Cambridge University Press, Cambridge, England, 2004.
2. Liang-Hwe Chen and Shyang Chang. An adaptive learning algorithm for principal component analysis. IEEE Transaction on Neural Networks, 6(5): 1255-1263, 1995.
3. Christian Darken and John E. Moody. Towards faster stochastic gradient search. In John E. Moody, Stephen J. Hanson, and Richard Lippmann, editors, Advances in Neural Information Processing Systems, volume 4, pages 1009-1016. Morgan Kaufmann Publishers, 1992.
4. Athinodoros S. Georghiades, Peter N. Belhumeur, and David J. Kriegman. From few to many: Illumination cone models for face recognition under variable lighting and pose. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(6):643-660, 2001. ISSN 0162-8828. doi:http://doi. ieeecomputersociety.org/10.1109/34.927464.
5. Andreas Griewank. Evaluating Derivatives: Principles and Techniques of Algorithmic Differentiation. Frontiers in Applied Mathematics. SIAM, Philadelphia, 2000.
6. Thorsten Joachims. Making large-scale SVM learning practical. In Bernhard Schölkopf, Chris J. C. Burges, and Alex J. Smola, editors, Advances in Kernel Methods-Support Vector Learning, pages 169-184, Cambridge, MA, 1999. MIT Press.
7. Juha Karhunen. Optimization criteria and nonlinear PCA neural networks. In IEEE World Congress on Computational Intelligence, volume 2, pages 1241-1246, 1994.
8. Juha Karhunen and Jyrki Joutsensalo. Representation and separation of signals using nonlinear PCA type learning. Neural Networks, 7(1):113-127, 1994.
9. Kwang In Kim, Matthias O. Franz, and Bernhard Schölkopf. Iterative kernel principal component analysis for image modeling. IEEE Trans. Pattern Analysis and Machine Intelligence, 27(9): 1351-1366, 2005.
10. YannLeCun. MNIST handwritten digit database, 1998. URL http://www.research.att.com/~yann/ocr/mnist/.
11. Yann LeCun, Bernhard E. Boser, John S. Denker, Donnie Henderson, R. E. Howard, Wayne E. Hubbard, and Lawrence D. Jackel. Backpropagation applied to handwritten zip code recognition. Neural Computation, 1:541-551, 1989.
12. Marina Meila. Comparing clusterings: An axiomatic view. In Proc. 22nd Intl. Conf. Machine Learning (ICML), pages 577-584, New York, NY, USA, 2005. ACM Press.
13. Sebastian Mika, Bernhard Schölkopf, Alex J. Smola, Klaus-Robert Müller, Matthias Scholz, and Gunnar Rätsch. Kernel PCA and de-noising in feature spaces. In Michael S. Kearns, Sara A. Solla, and David A. Cohn, editors, Advances in Neural Information Processing Systems, volume 11, pages 536-542. MIT Press, 1999.
14. David C. Munson, Jr. A note on Lena. IEEE Trans. Image Processing, 5(1), 1996.
15. Andrew Y Ng, Michael I. Jordan, and Yair Weiss. On spectral clustering: Analysis and an algorithm. In Thomas G. Dietterich, Suzanna Becker, and Zoubin Ghahramani, editors, Advances in Neural Information Processing Systems, volume 14, 2002.
16. Erkki Oja and Juha Karhunen. On stochastic approximation of the eigenvectors and eigenvalues of the expectation of a random matrix. Journal of Mathematical Analysis and Applications, 106(1): 69-84, February 1985.
17. Barak A. Pearlmutter. Fast exact multiplication by the Hessian. Neural Computation, 6(1):147-160, 1994.
18. John Piatt. Fast training of support vector machines using sequential minimal optimization. In Bernhard Schölkopf, Chris J. C. Burges, and Alex J. Smola, editors, Advances in Kernel Methods - Support Vector Learning, pages 185-208, Cambridge, MA, 1999. MIT Press.
19. Herbert Robbins and Sutton Monro. A stochastic approximation method. Annals of Mathematical Statistics, 22:400-407, 1951.
20. Terrence D. Sanger. Optimal unsupervised learning in a single-layer linear feedforward network. Neural Networks, 2:459-473, 1989.
21. Bernhard Schölkopf and Alex J. Smola. Learning with Kernels. MIT Press, Cambridge, MA, 2002.
22. Bernhard Schölkopf, Alex J. Smola, and Klaus-Robert Müller. Nonlinear component analysis as a kernel eigenvalue problem. Neural Computation, 10:1299-1319, 1998.
23. Nicol N. Schraudolph. Fast curvature matrix-vector products for second-order gradient descent. Neural Computation, 14(7):1723-1738, 2002.
24. Nicol N. Schraudolph. Local gain adaptation in stochastic gradient descent. In Proc. Intl. Conf. Artificial Neural Networks, pages 569-574, Edinburgh, Scotland, 1999. IEE, London.
25. Nicol N. Schraudolph, Simon Günter, and S. V. N. Vishwanathan. Fast iterative kernel PCA. In Bernhard Schölkopf, John Platt, and Thomas Hofmann, editors, Advances in Neural Information Processing Systems, volume 19, Cambridge MA, June 2007. MIT Press.
26. Therdsak Tangkuampien and David Suter. Human motion de-noising via greedy kernel principal component analysis filtering. In Proc. Intl. Conf. Pattern Recognition, 2006.
27. S. V. N. Vishwanathan, Nicol N. Schraudolph, and Alex J. Smola. Step size adaptation in reproducing kernel Hubert space. Journal of Machine Learning Research, 7:1107-1133, June 2006.
28. Luca Zanni, Thomas Serafini, and Gaetano Zanghirati. Parallel software for training large scale support vector machines on multiprocessor systems. Journal of Machine Learning Research, 7: 1467-1492, July 2006.