Constructive Incremental Learning from only Local Information

Neural Computation

Volumn 10, Issue 8, 1998, Pages 2047-2084

  Schaal, Stefan ^a,b   Atkeson, Christopher G ^c,d  

^a University of Southern California ^*  (United States)

^b JAPAN SCIENCE AND TECHNOLOGY AGENCY   (Japan)

^c Georgia Institute of Technology   (United States)

^d ADVANCED TELECOMMUNICATIONS RESEARCH INSTITUTE INTERNATIONAL   (Japan)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 0001108227     PISSN: 08997667     EISSN: None     Source Type: Journal    
DOI: 10.1162/089976698300016963     Document Type: Article

References (93)

1. Aha, D. (1997). Lazy learning. Artificial Intelligence Review, 11, 1-5.
2. An, C. H., Atkeson, C. G., & Hollerbach, J. M. (1988). Model-based control of a robot manipulator. Cambridge, MA: MIT Press.
3. Atkeson, C. G., Moore, A. W., & Schaal, S. (1997a). Locally weighted learning. Artificial Intelligence Review, 11, 11-73.
4. Atkeson, C. G., Moore, A. W., & Schaal, S. (1997b). Locally weighted learning for control. Artificial Intelligence Review, 11, 75-113.
5. Atkeson, C. G. (1989a). Learning arm kinematics and dynamics. Annual Review Neuroscience, 12, 157-183.
6. Atkeson, C. G. (1989b). Using local models to control movement. In D. Touretzky, (Ed.), Advances in neural information processing systems, 1 (79-86). San Mateo, CA: Morgan Kaufmann.
7. Atkeson, C. G., & Schaal, S. (1995). Memory-based neural networks for robot learning. Neurocomputing, 9, 243-269.
8. Belsley, D. A., Kuh, E., & Welsch, R. E. (1980). Regression diagnostics: Identifying influential data and sources of collinearity. New York: Wiley.
9. Bishop, C. M. (1996). Neural networks for pattern recognition. New York: Oxford University Press.
10. Breiman, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (1984). Classification and regression trees. Belmont, CA: Wadsworth International Group.
11. Bruske, J., & Sommer, G. (1995). Dynamic cell structure learns perfectly topology preserving map. Neural Computation, 7, 845-865.
12. Cannon, M., & Slotine, J. E. (1995). Space-frequency localized basis function networks for nonlinear system estimation and control. Neurocomputing, 9, 3, 293-342.
13. Carpenter, G. A., & Grossberg, S. (1987). A massively parallel architecture for a self-organizing neural pattern recognition machine. Computer Vision, Graphics, and Image Processing, 37, 54-115.
14. Churchland, P. S., & Sejnowski, T. J. (1992). The computational brain. Cambridge, MA: MIT Press.
15. Cleveland, W. S. (1979). Robust locally weighted regression and smoothing scatterplots. Journal of the American Statistical Association, 74, 829-836.
16. Cleveland, W. S., Devlin, S. J., & Grosse, E. (1988). Regression by local fitting: Methods, properties, and computational algorithms. Journal of Econometrics, 37, 87-114.
17. Cleveland, W. S., & Devlin, S. J. (1988). Locally weighted regression: An approach to regression analysis by local fitting. Journal of the American Statistical Association, 83, 596-610.
18. Cleveland, W. S., & Loader, C. (1995). Smoothing by local regression: Principles and methods (Tech. Rep.). Murray Hill, NJ: AT&T Bell Laboratories.
19. Daugman, J., & Downing, C. (1995). Gabor wavelets for statistical pattern recognition. In M. A. Arbib (Ed.), The handbook of brain theory and neural networks (pp. 414-420). Cambridge, MA: MIT Press.
20. de Boor, C. (1978). A practical guide to splines. New York: Springer-Verlag.
21. Deco, G., & Obradovic, D. (1996). An information-theoretic approach to neural computation. New York: Springer-Verlag.
22. Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977). Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society B, 39, 1-38.
23. Duda, R. O., & Hart, P. E. (1973). Pattern classification and scene analysis. New York: Wiley.
24. Fahlman, S. E., & Lebiere, C. (1990). The cascade-correlation learning architecture. In D. S. Touretzky (Ed.), Advances in neural information processing systems, 2. San Mateo, CA: Morgan Kaufmann.
25. Fan, J., & Gijbels, I. (1992). Variable band-width and local linear regression smoothers. Annals of Statistics, 20, 2008-2036.
26. Fan, J., & Gijbels, I. (1995). Data-driven bandwidth selection in local polynomial fitting: Variable bandwidth and spatial adaptation. Journal of the Royal Statistical Society B, 57, 371-395.
27. Fan, J., & Gijbels, I. (1996). Local polynomical modelling and its applications. London: Chapman & Hall.
28. Farmer, J. D., & Sidorowich, J. J. (1987). Predicting chaotic time series. Phys. Rev. Lett., 59 (8), 845-848.
29. Farmer, J. D., & Sidorowich, J. J. (1988). Exploiting chaos to predict the future and reduce noise. In Y. C. Lee (Ed.), Evolution, learning, and cognition (p. 27). Singapore: World Scientific.
30. Field, D. J. (1994). What is the goal of sensory coding? Neural Computation, 6, 559-601.
31. Frean, M. (1990). The upstart algorithm: A method for constructing and training feedforward neural networks. Neural Computation, 2, 198-209.
32. Friedman, J. H. (1984). A variable span smoother (Tech. Rep. No. 5). Stanford: Department of Statistics, Stanford University.
33. Friedman, J. H. (1991). Multivariate adaptive regression splines. Annals of Statistics, 19, 1-141.
34. Friedman, J. H., & Stützle, W. (1981). Projection pursuit regression. Journal of the American Statistical Association, Theory and Models, 76, 817-823.
35. Fritzke, B. (1994). Growing cell structures - A self-organizing network of unsupervised and supervised learning. Neural Networks, 7, 1441-1460.
36. Fritzke, B. (1995). Incremental learning of locally linear mappings. In Proceedings of the International Conference on Artificial Neural Networks, Paris, October 9-13.
37. Furlanello, C., & Giuliani, D. (1995). Combining local PCA and radial basis function networks for speaker normalization. In F. Girosi, J. Makhoul, E. Manolakas, & E. Wilson (Eds.), Proceedings of the 1995 IEEE Workshop on Neural Networks for Signal Processing V (pp. 233-242). New York: IEEE.
38. Furlanello, C., Giuliani, D., & Trentin, E. (1995). Connectionist speaker normalization with generalized resource allocating networks. In D. Tesauro, D. S. Touretzky, & T. K. Leen (Eds.), Advances in neural information processing systems, 7 (pp. 867-874). Cambridge, MA: MIT Press.
39. Geman, S., Bienenstock, E., & Doursat, R. (1992). Neural networks and the bias/variance dilemma. Neural Computation, 4, 1-58.
40. Georgopoulos, A. P. (1991). Higher order motor control. Annual Review of Neuroscience, 14, 361-377.
41. Hájek, J. (1969). A course in nonparametric statistics. San Francisco: Holden-Day.
42. Hastie, T., & Loader, C. (1993). Local regression: Automatic kernel carpentry. Statistical Science, 8, 120-143.
43. Hastie, T. J., & Tibshirani, R. J. (1990). Generalized additive models. London: Chapman and Hall.
44. Hastie, T. J., & Tibshirani, R. J. (1994). Nonparametric regression and classification: Part I: Nonparametric regression. In V. Cherkassky, J. H. Friedman, & H. Wechsler (Eds.), From statistics to neural networks: Theory and pattern recognition applications. ASI Proceedings, subseries F, Computer and Systems Sciences. Berlin: Springer-Verlag.
45. Hubel, D. H., & Wiesel, T. N. (1959). Receptive fields of of single neurons in the cat's striate cortex. Journal of Neurophysiology, 148, 574-591.
46. Imamizu, H., Uno, Y., & Kawato, M. (1995). Internal representations of the motor apparatus: Implications from generalization in visuomotor learning. Journal of Experimental Psychology, 21, 1174-1198.
47. Jacobs, R. A., Jordan, M. I., Nowlan, S. J., & Hinton, G. E. (1991). Adaptive mixtures of local experts. Neural Computation, 3, 79-87.
48. Jordan, M. I., & Jacobs, R. (1994). Hierarchical mixtures of experts and the EM algorithm. Neural Computation, 6, 181-214.
49. Jutten, C., & Chentouf, R. (1995). A new scheme for incremental learning. Neural Processing Letters, 2, 1-4.
50. Kwok, T.-Y., & Yeung, D.-Y. (1995). Constructive feedforward neural networks for regression problems: A survey (Tech. Rep. No. HKUST-CS95-43). Clear Water Bay, Kowloon, Hong Kong: Department of Computer Science, Hong Kong University of Science and Technology.
51. Lee, C., Rohrer, W. R., & Sparks, D. L. (1988). Population coding of saccadic eye movement by neurons in the superior colliculus. Nature, 332, 357-360.
52. Littmann, E., & Ritter, H. (1993). Generalization abilities of cascade network architectures. In S. J. Hanson, J. Cowan, & C. L. Giles (Eds.), Advances in neural information processing systems, 5 (pp. 188-195). San Mateo, CA: Morgan Kaufmann.
53. Ljung, L., & Söderström, T. (1986). Theory and practice of recursive identification. Cambridge, MA: MIT Press.
54. Lowe, D. G. (1995). Similarity metric learning for a variable-kernel classifier. Neural Computation, 7, 72-85.
55. Martinetz, T., & Schulten, K. (1994). Topology representing networks. Neural Networks, 7, 507-522.
56. Merzenich, M. M., Kaas, J. H., Nelson, R. J., Sur, M., & Felleman, D. (1983). Topographic reorganization of somatosensory cortical areas 3b and 1 in adult monkeys following restricted deafferentation. Neuroscience, 8, 33-55.
57. Millington, P. J. (1991). Associative reinforcement learning for optimal control. Master's thesis, Massachusetts Institute of Technology.
58. Moody, J., & Darken, C. (1988). Learning with localized receptive fields. In D. Touretzky, , G. Hinton, & T. Sejnowski (Eds.), Proceedings of the 1988 Connectionist Summer School (pp. 133-143). San Mateo, CA: Morgan Kaufmann.
59. Moore, A. (1991). Fast, robust adaptive control by learning only forward models. In J. E. Moody, S. J. Hanson, & R. P. Lippmann (Eds.), Advances in neural information processing systems, 4. San Mateo, CA: Morgan Kaufmann.
60. Mountcastle, V. B. (1957). Modality and topographic properties of single neurons of cat's somatic sensory cortex. Journal of Neurophysiology, 20, 408-434.
61. Myers, R. H. (1990). Classical and modern regression with applications. Boston: PWS-KENT.
62. Nadaraya, E. A. (1964). On estimating regression. Theor. Prob. Appl., 9, 141-142.
63. Olshausen, B. A., & Field, D. J. (1996). Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature, 381, 607-609.
64. Ormoneit, D., & Tresp, V. (1995). Improved Gaussian mixture density estimates using Bayesian penalty terms and network averaging (Tech. Rep. No. FKI-205-95. Munich: Theoretical Computer Science and Foundations of Artificial Intelligence, Technische Universität München.
65. Orr, M. J. L. (1995). Regularization in the selection of radial basis function centers. Neural Computation, 7, 606-623.
66. Papoulis, A. (1991). Probability, random variables, and stochastic processes. New York: McGraw-Hill.
67. Perrone, M. P., & Cooper, L. N. (1993). When networks disagree: Ensemble methods for hybrid neural networks. In R. J. Mammone (Ed.), Neural networks for speech and image processing. London: Chapman-Hall.
68. Platt, J. (1991). A resource-allocating network for function interpolation. Neural Computation, 3, 213-225.
69. Poggio, R., & Girosi, F. (1990). Regularization algorithms for learning that are equivalent to multilayer networks. Science, 247, 978-982.
70. Powell, M. J. D. (1987). Radial basis functions for multivariate interpolation: A review. In J. C. Mason & M. G. Cox (Eds.), Algorithms for approximation (pp. 143-167). Oxford: Clarendon Press.
71. Quartz, S. R., & Sejnowski, T. J. (1997). The neural basis of cognitive development: A constructivist manifesto. Behavioral and Brain Sciences 20, 537-596.
72. Reilly, D. L., Cooper, L. N., & Elbaum, C. (1982). A neural model for category learning. Biological Cybernetics, 45, 35-41.
73. Ritter, H., & Schulten, K. (1986). Topology conserving mappings for learning motor tasks. In J. S. Denker (Ed.), Neural networks for computing (pp. 376-380). AIP Conference Proceedings, Snowbird, Utah.
74. Sanger, T. D. (1991). A tree-structured adaptive network for function approximation in high-dimensional spaces. IEEE Transactions on Neural Networks, 2, 285-293.
75. Sanger, T. D., Sutton, R. S., & Matheus, C. J. (1992). Iterative construction of sparse polynomial approximations. In S. J. Hanson, J. E. Moody, & R. P. Lippmann (Eds.), Advances in neural information processing systems, 4 (pp. 1064-1071). San Mateo, CA: Morgan-Kaufmann.
76. Schaal, S. (1997). Learning from demonstration. In M. C. Mozer, M. Jordan, & T. Petsche (Eds.), Advances in neural information processing systems, 9 (pp. 1040-1046). Cambridge, MA: MIT Press.
77. Schaal, S., & Atkeson, C. G. (1994a). Robot juggling: An implementation of memory-based learning. Control Systems Magazine, 14, 57-71.
78. Schaal, S., & Atkeson, C. G. (1994b). Assessing the quality of learned local models. In J. Cowan, G. Tesauro, & J. Alspector (Eds.), Advances in neural information processing systems, 6 (pp. 160-167). San Mateo, CA: Morgan Kaufmann.
79. Schaal, S., & Atkeson, C. G. (1996). From isolation to cooperation: An alternative of a system of experts. In D. S. Touretzky, M. C. Mozer, & M. E. Hasselmo (Eds.), Advances in neural information processing systems, 8 (pp. 605-611). Cambridge, MA: MIT Press.
80. Schaal, S., & Atkeson, C. G. (1997). Receptive field weighted regression (Tech. Rep. No. TR-H-209). Kyoto: ATR Human Information Processing Laboratories.
81. Scott, D. W. (1992). Multivariate density estimation. New York: Wiley.
82. Shadmehr, R., Brashers-Krug, T., & Mussa-Ivaldi, F. A. (1995). Interference in learning internal models of inverse dynamics in humans. In G. Tesauro, D. S. Touretzky, & K. T. Leen (Eds.), Advances in neural information processing systems, 7 (pp. 1117-1124). San Mateo, CA: Morgan Kaufmann.
83. Shadmehr, R., & Mussa-Ivaldi, F. A. (1994). Adaptive representation of dynamics during learning of a motor task. Journal of Neuroscience, 14, 3208-3224.
84. Shin, Y., & Ghosh, J. (1995). Ridge polynomial networks. IEEE Transactions on Neural Networks, 6, 610-622.
85. Stone, M. (1974). Cross-validatory choice and assessment of statistical predictors. Journal of the Royal Statistical Society, B36, 111-147.
86. Sutton, R. S. (1992a). Gain adaptation beats least squares. In Proceedings of Seventh Yale Workshop on Adaptive and Learning Systems (pp. 161-166). New Haven, CT.
87. Sutton, R. S. (1992b). Adapting bias by gradient descent: An incremental version of Delta-Bar-Delta. In Proceedings of the Tenth National Conference on Artificial Intelligence (pp. 171-176). Cambridge, MA: MIT Press.
88. van der Smagt, P., & Groen, F. (1995). Approximation with neural networks: Between local and global approximation. In Proceedings of the 1995 International Conference on Neural Networks, 2 (pp. 1060-1064). Perth, Australia.
89. Vijayakumar, S., & Schaal, S. (1997). Local dimensionality reduction for locally weighted learning. In IEEE International Symposium on Computational Intelligence in Robotics and Automation (pp. 220-225). Monterey, CA, July 10-11.
90. Wahba, G. (1990). Spline models for observational data. Philadelphia: Society for Industrial and Applied Mathematics.
91. Wahba, G., & Wold, S. (1975). A completely automatic french curve: Fitting spline functions by cross-validation. Communications in Statistics, 4 (1), 1-17.
92. Watson, G. S. (1964). Smooth regression analysis. Sankhaya: The Indian Journal of Statistics A, 26, 359-372.
93. Xu, L., Jordan, M. I., & Hinton, G. E. (1995). An alternative model for mixture of experts. In G. Tesauro, D. S. Touretzky, & T. K. Leen (Eds.), Advances in neural information processing systems (pp. 633-640). Cambridge, MA: MIT Press.