Local dimensionality reduction for non-parametric regression

Neural Processing Letters

Volumn 29, Issue 2, 2009, Pages 109-131

  Hoffmann, Heiko ^a,b   Schaal, Stefan ^c   Vijayakumar, Sethu ^a  

^a UNIVERSITY OF EDINBURGH   (United Kingdom)

^b University of Southern California   (United States)

^c UNIVERSITY OF SOUTHERN CALIFORNIA   (United States)

Author keywords

Correlation; Dimensionality reduction; Factor analysis; Incremental learning; Kernel function; Locally weighted regression; Partial least squares; Principal component analysis; Principal component regression; Reduced rank regression

Indexed keywords

CURVE FITTING; DATA REDUCTION; FACE RECOGNITION; INTERFERENCE SUPPRESSION; LEARNING SYSTEMS; LEAST SQUARES APPROXIMATIONS; REGRESSION ANALYSIS;

CORRELATION; DIMENSIONALITY REDUCTION; FACTOR ANALYSIS; INCREMENTAL LEARNING; KERNEL FUNCTION; LOCALLY-WEIGHTED REGRESSION; PARTIAL LEAST SQUARES; PRINCIPAL COMPONENT REGRESSION; REDUCED-RANK REGRESSION;

PRINCIPAL COMPONENT ANALYSIS;

EID: 62649105137     PISSN: 13704621     EISSN: 1573773X     Source Type: Journal    
DOI: 10.1007/s11063-009-9098-0     Document Type: Article

References (53)

1. B Abraham G Merola 2005 Dimensionality reduction approach to multivariate prediction Comput Stat Data Anal 48 5 16
2. CG Atkeson AW Moore S Schaal 1997 Locally weighted learning Artif Intell Rev 11 11 73
3. CG Atkeson AW Moore S Schaal 1997 Locally weighted learning for control Artif Intell Rev 11 75 113
4. M Belkin P Niyogi 2003 Laplacian eigenmaps for dimensionality reduction and data representation Neural Comp 15 6 1373 1396
5. A Bell T Sejnowski 1997 The independent components of natural scenes are edge filters Vis Res 37 23 3327 3338
6. Bellman RE (1961) Adaptive control processes: a guided tour. Princeton University Press, Princeton, NJ
7. Beyer K, Goldstein J, Ramakrishnan R, Shaft U (1999) When is 'nearest neighbor' meaningful? In: Proceedings of 7th international conference on database theory, Jerusalem, Israel, pp 217-235
8. Bishop CM (2006) Pattern recognition and machine learning. Springer, New York
9. Cressie N (1993) Statistics for spatial data. Wiley, New York
10. S de Jong 1993 SIMPLS: an alternative approach to partial least squares regression Chemometr Intell Lab Syst 18 251 263
11. Diamantaras KI, Kung SY (1996) Principal component neural networks. Wiley, New York
12. D'Souza A, Vijayakumar S, Schaal S (2001) Are internal models of the entire body learnable? Society for Neuroscience, Abstracts
13. Everitt BS (1984) An introduction to latent variable models. Chapman and Hall, London
14. Fan J, Gijbels I (1996) Local polynomial modeling and its applications. Chapman and Hall, London, UK
15. IE Frank JH Friedman 1993 A statistical view of some chemometrics regression tools Technometrics 35 2 109 135
16. J Geweke 1996 Bayesian reduced rank regression in econometrics J Econometr 75 1 121 146
17. Ghahramani Z, Beal M (2000) Variational inference for bayesian mixtures of factor analysers. In: Solla S, Leen T, Müller KR (eds) Advances in neural information processing systems, vol 12. MIT Press, Cambridge, MA, pp 449-455
18. Ghahramani Z, Hinton GE (1997) The EM algorithm for mixtures of factor analyzers. Tech. Rep. CRG-TR-96-1. Department of Computer Science, University of Toronto, Canada
19. Hinton G, Roweis ST (2003) Stochastic neighbor embedding. In: Becker S, Thrun S, Obermayer K (eds) Advances in neural information processing systems, vol 15. MIT Press, Cambridge, MA, pp 857-864
20. AE Hoerl RW Kennard 1970 Ridge regression: biased estimation for nonorthogonal problems Technometrics 12 1 55 67
21. Hoffmann H (2005) Unsupervised learning of visuomotor associations, MPI series in biological cybernetics, vol 11. Logos Verlag Berlin, PhD thesis (2004), Bielefeld University, Germany
22. Hoffmann H, Möller R (2003) Unsupervised learning of a kinematic arm model. In: Kaynak O, Alpaydin E, Oja E, Xu L (eds) Artificial neural networks and neural information processing-ICANN/ICONIP 2003, LNCS, vol 2714. Springer, Berlin, pp 463-470
23. H Hoffmann W Schenck R Möller 2005 Learning visuomotor transformations for gaze-control and grasping Biol Cybernetics 93 2 119 130
24. AJ Izenman 1975 Reduced-rank regression for the multivariate linear model J Multivar Anal 5 2 248 264
25. MI Jordan DE Rumelhart 1992 Forward models: supervised learning with a distal teacher Cogn Sci 16 307 354
26. M Kawato 1999 Internal models for motor control and trajectory planning Curr Opin Neurobiol 9 718 727
27. Kustra R (1996) Delve census-house dataset. Available at http://www.cs.toronto.edu/~delve/data/datasets.html
28. G Matheron 1963 Principles of geostatistics Econ Geol 58 8 1246 1266
29. R Möller 2002 Interlocking of learning and orthonormalization in RRLSA Neurocomputing 49 429 433
30. JR Movellan JL [tmp] McClelland 1993 Learning continuous probability distributions with symmetric diffusion networks Cogn Sci 17 463 496
31. E Oja 1982 A simplified neuron model as a principal component analyzer J Math Biol 15 3 267 273
32. E Oja 1989 Neural networks, principle components, and subspaces Int J Neural Syst 1 1 61 68
33. BA Olshausen DJ Field 1996 Emergence of simple-cell receptive field properties by learning a sparse code for natural images Nature 381 607 609
34. S Ouyang Z Bao GS Liao 2000 Robust recursive least squares learning algorithm for principal component analysis IEEE Trans Neural Netw 11 1 215 221
35. Rasmussen CE, Williams CKI (2006) Gaussian processes for machine learning. MIT Press, Cambridge, MA
36. ST Roweis LK Saul 2000 Nonlinear dimensionality reduction by locally linear embedding Science 290 2323 2326
37. TD Sanger 1989 Optimal unsupervised learning in a single-layer linear feedforward neural network Neural Netw 2 459 473
38. S Schaal D Sternad 2001 Origins and violations of the 2/3 power law in rhythmic 3d movements Exp Brain Res 136 60 72
39. Schaal S, Vijayakumar S, Atkeson CG (1998) Local dimensionality reduction. In: Jordan M, Kearns M, Solla S (eds) Advances in neural information processing systems, vol 10. MIT Press, Cambridge, MA, pp 633-639
40. Schölkopf B, Smola AJ (2002) Learning with kernels. MIT Press, Cambridge, MA
41. JB Tenenbaum V de Silva JC Langford 2000 A global geometric framework for nonlinear dimensionality reduction Science 290 2319 2323
42. ME Tipping 2001 Sparse bayesian learning and the relevance vector machine J Mach Learn Res 1 211 244
43. ME Tipping CM Bishop 1999 Probabilistic principal component analysis J Roy Stat Soc Ser B 61 3 611 622
44. Vapnik VN (1995) The nature of statistical learning theory. Springer-Verlag, New York
45. Vijayakumar S, Schaal S (2000a) Fast and efficient incremental learning for high-dimensional movement systems. In: Proceedings of the international conference on robotics and automation, San Francisco, CA
46. Vijayakumar S, Schaal S (2000b) Locally weighted projection regression: an O(n) algorithm for incremental real time learning in high dimensional space. In: Proceedings of the 17th international conference on machine learning, Montreal, Canada, pp 1079-1086
47. S Vijayakumar A D'Souza T Shibata J Conradt S Schaal 2002 Statistical learning for humanoid robots Auton Robots 12 1 55 69
48. S Vijayakumar A D'Souza S Schaal 2005 Incremental online learning in high dimensions Neural Comput 17 2602 2634
49. N Vlassis Y Motomura B Kröse 2002 Supervised dimension reduction of intrinsically low-dimensional data Neural Comput 14 191 215
50. JT Webster RF Gunst RL Mason 1974 Latent root regression analysis Technometrics 16 4 513 522
51. Weinberger KQ, Sha F, Saul LK (2004) Learning a kernel matrix for nonlinear dimensionality reduction. In: Proceedings of the 21st international conference on machine learning, Washington, DC
52. S Wold A Ruhe H Wold WJ Dunn III 1984 The collinearity problem in linear regression. The partial least squares (PLS) approach to generalized inverses SIAM J Sci Stat Comput 5 3 735 743
53. AL van den Wollenberg 1977 Redundancy analysis an alternative for canonical correlation analysis Psychometrika 42 2 207 219