Regularized Principal Manifolds

Journal of Machine Learning Research

Volumn 1, Issue 3, 2001, Pages 179-209

  Smola, Alexander J ^a   Mika, Sebastian ^b   Schölkopf, Bernhard ^a,c   Williamson, Robert C ^a  

^a AUSTRALIAN NATIONAL UNIVERSITY   (Australia)

^b FRAUNHOFER FIRST   (Germany)

^c Barnhill Technologies   (Germany)

Author keywords

Clustering; Entropy numbers; Generative topographic map; Kernel pca; Kernels; Principal curves; Regularization; Support vector machines; Uniform convergence

Indexed keywords

EID: 0037845137     PISSN: 15324435     EISSN: None     Source Type: Journal    
DOI: None     Document Type: Article

References (45)

1. M. A. Aizerman, É. M. Braverman, and L. I. Rozonoér. Theoretical foundations of the potential function method in pattern recognition learning. Automation and Remote Control, 25:821-837, 1964.
2. M. Anthony and P. Bartlett. A Theory of Learning in Artificial Neural Networks. Cambridge University Press, 1999.
3. P. Bartlett, T. Linder, and G. Lugosi. The minimax distortion redundancy in empirical quantizer design. IEEE Transactions on Information Theory, 44(5):1802-1813, 1998.
4. K. Bennett. Combining support vector and mathematical programming methods for induction. In B. Schölkopf, C. J. C. Burges, and A. J. Smola, editors, Advances in Kernel Methods - SV Learning, pages 307-326, Cambridge, MA, 1999. MIT Press.
5. C. M. Bishop. Training with noise is equivalent to Tikhonov regularization. Neural Computation, 7:108-116, 1995.
6. C. M. Bishop, M. Svensén, and C. K. I. Williams. GTM: The generative topographic mapping. Neural Computation, 10(1):215-234, 1998.
7. B. E. Boser, I. M. Guyon, and V. N. Vapnik. A training algorithm for optimal margin classifiers. In D. Haussler, editor, Proceedings of the 5th Annual ACM Workshop on Computational Learning Theory, pages 144-152, Pittsburgh, PA, July 1992. ACM Press.
8. P. Bradley and O. Mangasarian. Massive data discrimination via linear suppport vector machines. Mathematical Programming Technical Report 98-05, University of Wisconsin Madison, 1998.
9. P. S. Bradley, O. L. Mangasarian, and W. N. Street. Clustering via concave minimization. In Advances in Neural Information Processing Systems, volume 9, pages 368-374, Cambridge, MA, 1997. MIT Press.
10. B. Carl and I. Stephani. Entropy, compactness, and the approximation of operators. Cambridge University Press, Cambridge, UK, 1990.
11. S. Chen, D. Donoho, and M. Saunders. Atomic decomposition by basis pursuit. Siam Journal of Scientific Computing, 20(1):33-61, 1999.
12. A. P. Dempster, N. M. Laird, and D. B. Rubin. Maximum Likelihood from Incomplete Data via the EM Algorithm. Journal of the Royal Statistical Society B, 39(1):1-22, 1977.
13. T. T. Frieß and R. F. Harrison. Linear programming support vector machiens for pattern classification and regression estimation and the set reduction algorithm. TR RR-706, University of Sheffield, Sheffield, UK, 1998.
14. A. Gersho and R. M. Gray. Vector quantization and signal compression. Kluwer Academic Publishers, Boston, 1991.
15. F. Girosi. An equivalence between sparse approximation and support vector machines. Neural Computation, 10(6):1455-1480, 1998.
16. F. Girosi, M. Jones, and T. Poggio. Regularization theory and neural networks architectures. Neural Computation, 7(2):219-269, 1995.
17. M. Hamermesh. Group theory and its applications to physical problems. Addison Wesley, Reading, MA, 2 edition, 1962. Reprint by Dover, New York, NY.
18. T. Hastie and W. Stuetzle. Principal curves. Journal of the American Statistical Association, 84(406):502-516, 1989.
19. P. J. Huber. Robust Statistics. John Wiley and Sons, New York, 1981.
20. IBM Corporation. IBM optimization subroutine library guide and reference. IBM Systems Journal, 31, 1992. SC23-0519.
21. N. Kambhatla and T. K. Leen. Fast non-linear dimension reduction. In J. D. Cowan, G. Tesauro, and J. Alspector, editors, Advances in Neural Information Processing Systems 6. Proceedings of the 1993 Conference, pages 152-159, San Francisco, CA, 1994. Morgan Kaufmann.
22. N. Kambhatla and T. K. Leen. Dimension reduction by local principal component analysis. Neural Computation, 9(7):1493-1516, 1997.
23. B. Kégl, A. Krzyzak, T. Linder, and K. Zeger. Learning and design of principal curves. IEEE Transactions on Pattern Analysis and Machine Intelligence, pages 281-297, 2000.
24. G. S. Kimeldorf and G. Wahba. Some results on Tchebycheffian spline functions. J. Math. Anal Applic., 33:82-95, 1971.
25. S. P. Lloyd. Least squares quantization in PCM. IEEE Trans. Information Theory, 28:129-137, 1982.
26. O. L. Mangasarian. Nonlinear Programming. McGraw-Hill, New York, NY, 1969.
27. J. Mercer. Functions of positive and negative type and their connection with the theory of integral equations. Philosophical Transactions of the Royal Society, London, A 209:415-446, 1909.
28. V. A. Morozov. Methods for Solving Incorrectly Posed Problems. Springer Verlag, 1984.
29. B. A. Murtagh and M. A. Saunders. MINOS 5.1 user's guide. Technical Report SOL 83-20R, Stanford University, CA, USA, 1983. Revised 1987.
30. W. H. Press, S. A. Teukolsky, W. T. Vetterling, and B. P. Flannery. Numerical Recipes in C: The Art of Scientific Computing (2nd ed.). Cambridge University Press, Cambridge, 1992. ISBN 0-521-43108-5.
31. S. Roweis and Z. Ghahramani. A unifying review of linear gaussian models. Neural Computation, 11(2), 1999.
32. B. Schölkopf, A. Smola, and K.-R. Müller. Nonlinear component analysis as a kernel eigenvalue problem. Neural Computation, 10:1299-1319, 1998.
33. A. Smola, B. Schölkopf, and K.-R. Müller. The connection between regularization operators and support vector kernels. Neural Networks, 11:637-649, 1998.
34. A. J. Smola. Learning with Kernels. PhD thesis, Technische Universität Berlin, 1998. GMD Research Series No. 25.
35. A. J. Smola, A. Elisseeff, B. Schölkopf, and R. C. Williamson. Entropy numbers for convex combinations and MLPs. In A. J. Smola, P. L. Bartlett, B. Schölkopf, and D. Schuurmans, editors, Advances in Large Margin Classifiers, pages 369-387, Cambridge, MA, 2000. MIT Press.
36. A. N. Tikhonov and V. Y. Arsenin. Solution of Ill-Posed Problems. Winston, Washington, DC, 1977.
37. R. J. Vanderbei. LOQO user's manual -version 3.10. Technical Report SOR-9708, Princeton University, Statistics and Operations Research, 1997. Code available at http://www.princeton.edu/̃rvdb/.
38. V. Vapnik. Statistical Learning Theory. Wiley, NY, 1998.
39. V. N. Vapnik. Estimation of Dependences Based on Empirical Data. Springer-Verlag, Berlin, 1982.
40. G. Wahba. Smoothing and ill-posed problems. In M. Golberg, editor, Solutions methods for integral equations and applications, pages 183-194. Plenum Press, New York, 1979.
41. G. Wahba. Spline Models for Observational Data, volume 59 of CBMS-NSF Regional Conference Series in Applied Mathematics. SIAM, Philadelphia, 1990.
42. J. Weston, A. Gammerman, M. Stitson, V. Vapnik, V. Vovk, and C. Watkins. Support vector density estimation. In B. Schölkopf, C. J. C. Burges, and A. J. Smola, editors, Advances in Kernel Methods -Support Vector Learning, pages 293-306, Cambridge, MA, 1999. MIT Press.
43. C. K. I. Williams. Prediction with Gaussian processes: From linear regression to linear prediction and beyond. In M. I. Jordan, editor, Learning and Inference in Graphical Models. Kluwer, 1998.
44. R. C. Williamson, A. J. Smola, and B. Schölkopf. Generalization performance of regularization networks and support vector machines via entropy numbers of compact operators. Technical Report 19, NeuroCOLT, http://www.neurocolt.com, 1998. Accepted for publication in IEEE Transactions on Information Theory.
45. A. Yuille and N. Grzywacz. The motion coherence theory. In Proceedings of the International Conference on Computer Vision, pages 344-354, Washington, D.C., December 1988. IEEE Computer Society Press.