Structurally adaptive modular networks for nonstationary environments

IEEE Transactions on Neural Networks

Volumn 10, Issue 1, 1999, Pages 152-160

  Ramamurti, Viswanath ^a   Ghosh, Joydeep ^a  

^a The University of Texas at Austin   (United States)

Author keywords

Localized gating; Mixture of experts; Model complexity; Network growing; On line learning; Pruning

Indexed keywords

COMPUTER SIMULATION; EXPERT SYSTEMS; LEARNING SYSTEMS; MATHEMATICAL MODELS; ONLINE SYSTEMS; PARAMETER ESTIMATION; PROBLEM SOLVING;

LOCALIZED GATING; ONLINE LEARNING; STRUCTURALLY ADAPTIVE MODULAR NETWORKS;

NEURAL NETWORKS;

EID: 0032786303     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/72.737501     Document Type: Article

References (38)

1. N. E. Sharkey and A. J. C. Sharkey, "Understanding catastrophic interference in neural nets," Dept. Comput. Sci., Univ. Sheffield, U.K., Tech. Rep. CS-94-4, 1994.
2. R. M. French, "Semidistributed representations and catastrophic forgetting in connectionist networks," Connection Sci., vol. 4, no. 3, pp. 365-377, 1992.
3. J. C. Platt, "A resource allocation network for function interpolation," Neural Comput., vol. 3, no. 2, pp. 213-225, 1991.
4. A. Nag and J. Ghosh, "Flexible resource allocating network for noisy data," in SPIE Conf. Applicat. Sci. Comput. Intell., Orlando, Fl., vol. 3390, Apr. 1998, pp. 551-559.
5. R. A. Jacobs, M. I. Jordan, S. J. Nowlan, and G. E. Hinton, "Adaptive mixtures of local experts," Neural Comput., vol. 3, pp. 79-87, 1991.
6. M. I. Jordan and R. A. Jacobs, "Hierarchical mixture of experts and the {EM} algorithm," Neural Comput., vol. 6, pp. 181-214, 1994.
7. A. J. Zeevi, R. Meir, and V. Maiorov, "Error bounds for functional approximation and estimation using mixtures of experts," IEEE Trans. Inform. Theory, vol. 44, pp. 1010-1025, 1998.
8. W. S. Chaer, R. H. Bishop, and J. Ghosh, "A mixture-of-experts framework for adaptive Kalman filtering," IEEE Trans. Syst., Man, Cybern., vol. 27(B), pp. 452-64, June 1997.
9. R. Murray-Smith and T. A. Johansen, "Local learning in local model networks," in Multiple Model Approaches to Modeling and Control, Roderick Murray-Smith and Tor Arne Johansen, Eds. London, U.K." Taylor and Francis, 1997, pp. 185-210.
10. S. Schaal and C. G. Atkeson, "From isolation to cooperation: An alternative view of a system of experts," in Advances in Neural Inform. Processing Syst. 8, D. Touretzky, M. Mozer, and M. Hasselmo, Eds. Cambridge, MA: MIT Press, 1996, pp. 605-611.
11. K. Tumer and J. Ghosh, "Analysis of decision boundaries in linearly combined neural classifiers," Pattern Recognition, vol. 29, no. 2, pp. 341-348, Feb. 1996.
12. M. P. Perrone and L. N. Cooper, "Learning from what's been learned: Supervised learning in multineural network systems," in Proc. World Congr. Neural Networks. Hillsdale, NJ: INNS Press, 1993, pp. III:354-357.
13. A. J. C. Sharkey, "On combining artificial neural networks," Connection Sci., vol. 8, no. 3 and 4, pp. 299-314, Dec. 1996.
14. L. Xu, M. I. Jordan, and G. E. Hinton, "An alternative model for mixture of experts," in Advances in Neural Inform. Processing Syst. 7, G. Tesauro, D. S. Touretzky, and T. K. Leen, Eds. Cambridge, MA: MIT Press, 1995, pp. 633-640.
15. V. Ramamurti and J. Ghosh, "Structurally adaptive localized mixtures of experts for nonstationary environments," Univ. Texas Tech. Rep. UT-CVIS-TR-98-001, 1988. Available WWW: http://www.lans.ece.utexas.edu/techreps.html
16. S. Haykin, Neural Networks: A Comprehensive Foundation, 2nd ed. Englewood Cliffs, NJ: Prentice-Hall, 1999.
17. A. S. Weigend, M. Mangeas, and A. N. Srivastava, "Nonlinear gated experts for time series: Discovering regimes and avoiding overfilling," Int. J. Neural Syst., vol. 6, pp. 373-399, 1995.
18. V. Ramamurti and J. Ghosh, "On the use of localized gating in mixtures of experts networks," in SPIE Conf. Applicat. Sci. Comput. Intell., SPIE Proc. vol. 3390, invited paper, Orlando, Fl., Apr. 1998, pp. 24-35.
19. W. S. Chaer, R. H. Bishop, and J. Ghosh, "Hierarchical adaptive Kalman filtering for interplanetary orbit determination," IEEE Trans. Aerospace Electronic Syst., vol. 34, pp. 883-896, 1998.
20. V. Ramamurti and J. Ghosh, "Structural adaptation in mixlure of experts," in Proc. ICPR 96, Track D, 1996, pp. 704-708.
21. R. Reed, "Pruning algorithms - A survey," IEEE Trans. Neural Networks, vol. 4, pp. 740-747, 1993.
22. J. Ghosh and K. Turner, "Structural adaptation and generalization in supervised feedforward networks," J. Artificial Neural Networks, vol. 1, no. 4, pp. 431-458, 1994.
23. S. E. Fahlman and C. Lebiere, "The cascade-correlation learning architecture," in Advances in Neural Information Processing Systems-2. San Mateo, CA: Morgan Kaufmann, 1990, pp. 525-532.
24. Y. Shin and J. Ghosh, "Ridge polynomial networks," IEEE Trans. Neural Networks, vol. 6, pp. 610-622, May 1995.
25. J. Fristch, M. Finke, and A. Waibel, "Adaptively growing hierarchical mixtures of experts," in Advances in Neural Information Processing Systems-9, M. I. Jordan, M. C. Mozer, and T. Petsche, Eds. Cambridge, MA: MIT Press, 1997, pp. 459-65.
26. S. R. Waterhouse and A. J. Robinson, "Constructive algorithms for hierarchical mixture of experts," in Advances in Neural Inform. Processing Syst. 8, D. Touretzky, M. Mozer, and M. Hasselmo, Eds. Cambridge, MA: MIT Press, 1996, pp. 584-590.
27. L. Prechelt, "PROBEN1 - A set of benchmarks and benchmarking rules for neural network training algorithms," Fakultät für Informatik, Univ. Karlsruhe, Germany, Tech. Rep. 21/94, Sept. 1994. Available via anonymous FTP: /pub/papers/techreports/1994/1994-21.ps.Z on ftp.ira.uka.de
28. K. S. Narendra, J. Balakrishnan, and K. Ciliz, "Adaptation and learning using multiple models, switching and tuning," IEEE Contr. Syst. Mag., June 1995, pp. 37-51.
29. S. Haykin, Adaptive Filter Theory, 2nd ed. Englewood Cliffs, NJ: Prentice-Hall, 1997.
30. R. O. Duda and P. E. Hart, Pattern Classification and Scene Analysis. New York: Wiley, 1973.
31. G. Hans and C. Traven, "A neural network approach to statistical pattern classification by "semiparametric" estimation of probability density functions," IEEE Trans. Neural Networks, vol. 2, pp. 366-377, May 1991.
32. J. Kohlmorgen, K. R. Muller, and K. Pawelzik, "Segmentation and identification of drifting dynamical systems," in Neural Networks for Signal Processing, J. Principe, L. Giles, N. Morgan, and E. Wilson, Eds. Piscataway, NJ: IEEE, 1997, pp. 326-335.
33. S. Roberts and L. Tarassenko, "A probabilistic resource allocating network for novelty detection," Neural Comput., vol. 6, pp. 271-284, 1994.
34. V. Tresp and R. Hofmann, "Missing and noisy data in nonlinear time-series prediction," in Neural Networks for Signal Processing V, F. Girosi, J. Makhoul, E. Manolakos, and E. Wilson, Eds. Piscataway, NJ: IEEE, 1995, pp. 1-10.
35. V. Ramamurti and J. Ghosh, "Advances in using hierarchical mixture of experts for signal classification," in Proc. ICASSP'96, pp. 3569-3572.
36. _, "Regularization and error bars for the mixture of experts network," in Proc. ICNN'97, pp. 221-226.
37. A. S. Weigend and N. A. Gershenfeld, Eds., Time Series Prediction: Forecasting the Future and Understanding the Past. Reading, MA: Addison-Wesley, 1993.
38. P. D. Turney, "The identification of context-sensitive features: A formal definition of context for concept learning," in Proc. ICML-96 Wkshp. Learning Context-Sensitive Domains, Bari, Italy, July 3, 1996, pp. 53-59.