A Survey of Transfer Between Connectionist Networks

Connection Science

Volumn 8, Issue 2, 1996, Pages 163-184

  Pratt, Lorien ^a   Jennings, Barbara ^b  

^a Evolving Systems Inc   (United States)

^b Department of Geology and Geological Engineering   (United States)

Author keywords

Network learning; Transfer

Indexed keywords

EID: 0030491430     PISSN: 09540091     EISSN: None     Source Type: Journal    
DOI: 10.1080/095400996116866     Document Type: Article

References (62)

1. Abu-Mostafa, Y.S. (1994) Learning from hints. Journal of Complexity, 10, 165-178.
2. Agarwal, A., Mammone, R.J. & Naik, D.K. (1992) An on-line training algorithm to overcome catastrophic forgetting. In Intelligence Engineering Systems through Artificial Neural Networks, Vol. 2. New York: ASME Press, pp. 239-244.
3. Barnard, E. (1992) Optimization for training neural nets. IEEE Transactions on Neural Networks, 3, 232-240.
4. Barron, R.L., Mucciardi, A.N., Cook, F.J., Craig, J.N. & Barron, A.R. (1984) Adaptive learning networks: Development and application in the United States of algorithms related to GMDH. In S.J. Farlow (Ed.), Self-Organizing Methods in Modeling. New York: Marcel Dekker.
5. Baxter, J. (1996) Learning internal representations. In D.S. Touretzky, M.C. Mozer & M.E. Hasselmo (Eds), Advances in Neural Information Processing Systems, Vol. 8. Cambridge, MA: MIT Press.
6. Berenji, H.R. & Khedkar, P. (1992) Learning and tuning fuzzy logic controllers through reinforcements. IEEE Transactions on Neural Networks, 3, 724-740.
7. Carpenter, G.A. & Grossberg, S. (1991) Pattern Recognition by Self-organizing Neural Networks. Cambridge, MA: MIT Press.
8. Carroll, M. & Ellis, M. (1995) Designing and Coding Reusable C + +. Reading, MA: Addison-Wesley.
9. Caruana, R.A. (1993) Multitask learning: A knowledge-based source of inductive bias. In Proceedings of the 10th International Conference on Machine Learning, University of Massachusetts, pp. 41-48.
10. de Sa, V.R. (1994) Learning classification with unlabeled data. In J. Cowan, G. Tesauro & J. Alspector (Eds), Advances in Neural Information Processing Systems, Vol. 6. San Francisco, CA: Morgan Kaufmann, pp. 112-119.
11. Durbin, R., Miall, C. & Mitchison, G. (Eds) (1989) The Computing Neuron. Reading, MA: Addison-Wesley.
12. Elman, J. (1989) Representation and structure in connectionist models. Technical Report 8903. Center for Research in Language, University of California, San Diego, CA.
13. Fahlman, S.E. & Lebiere, C. (1990) The cascade-correlation learning architecture. In D.S. Touretzky (Ed.), Advances in Neural Information Processing Systems 2. San Mateo, CA: Morgan Kaufmann, pp. 524-532.
14. Fu. L. (1991) Rule learning by searching on adapted nets. In Proceedings of the 9th National Conference on Artificial Intelligence (AAAI-91). Anaheim, CA, pp. 590-595.
15. Hall, R.P. (1988) Computational approaches to analogical reasoning: A comparative analysis. Artificial Intelligence, 39, 39-71.
16. Hertz, J.A., Palmer, R.G. & Krogh, A.S. (1991) Introduction to the Theory of Neural Computation. Redwood City, CA: Addison-Wesley.
17. Holyoak, K.J. & Barnden, J.A. (1994) Advances in Connectionist and Neural Computation Theory, Volume 2: Analogical Connections. Norwood, NJ: Ablex.
18. Intrator, N. & Edelman, S. (1996) Making a low-dimensional representation suitable for diverse tasks. Connection Science, 8, 205-223.
19. Jackson, S.A. & Sharkey, N.E. (1995) Adaptive generalization in dynamic neural networks. In L. Niklasson & M. Boden (Eds), Current Trends in Connectionism. Hillsdale, NJ: Lawrence Erlbaum.
20. Jacobs, R.A., Jordan, M.I., Nowlan, S.J. & Hinton, G.E. (1991) Adaptive mixtures of local experts. Neural Computation, 3, 79-87.
21. Judd, S. (1988) On the complexity of learning in shallow neural networks. Journal of Complexity, 4, 177-192.
22. Kruschke, J.K. (1993) Human category learning: Implications for backpropagation models. Connection Science, 5, 3-36.
23. Kruschke, J.K. (1996) Dimensional relevance shifts in category learning. Connection Science, 8, 225-247.
24. Kubat, M. (1996) Second tier for decision trees. In Proceedings of the 13th International Conference on Machine Learning. Bari, Italy: Morgan Kaufmann.
25. Lando, M. & Edelman, S. (1995) Receptive field spaces and class-based generalization from a single view in face recognition. Network, 6, 551-576.
26. Martin, G. (1988) The effects of old learning on new in Hopfield and backpropagation nets. Technical Report ACA-HI-019. Microelectronics and Computer Technology Corporation (MCC).
27. McCloskey, M. & Cohen, N.J. (1989) Catastrophic interference in connectionist networks: The sequential learning problem. The Psychology of Learning and Motivation, 24.
28. Murre, J.M.J. (1996) Hypertransfer in neural networks. Connection Science, 8, 249-258.
29. Naik, D.K., Mammone, R.J. & Agarwal, A. (1992) Meta-neural network approach to learning by learning. In Intelligence Engineering Systems through Artificial Neural Networks, Vol. 2. New York: ASME Press, pp. 245-252.
30. Osgood, C.E. (1949) The similarity paradox in human learning: A resolution. Psychological Review, 56, 132-143.
31. Piatetsky-Shapiro, G. & Frawley, W.J. (Eds) (1991) Knowledge Discovery in Databases. Menlo Park, CA: AAAI Press/MIT Press.
32. Poggio, T., Beymer, D. & Sashua, A. (1993) Example based image analysis and synthesis. Technical Report AI Memo No. 1431, CBCL Paper No. 80. Massachusetts Institute of Technology, Center for Biological and Computational Learning.
33. Pratt, L.Y. (1993a) Discriminability-based transfer between neural networks. In C. Giles, S.J. Hanson & J.D. Cowan (Eds), Advances in Neural Information Processing Systems 5. San Mateo, CA: Morgan Kaufmann, pp. 204-211.
34. Pratt, L.Y. (1993b) Non-literal transfer among neural network learners. In R.J. Mammone (Ed.), Artificial Neural Networks for Speech and Vision. London: Chapman Hall, Chapter 11.
35. Pratt, L.Y. (1993c) Transferring previously learned back-propagation neural networks to new learning tasks. Technical Report ML-TR-37. Rutgers University Computer Science Department, New Brunswick, NJ.
36. Pratt, L.Y. (1994) Experiments on the transfer of knowledge between neural networks. In S. Hanson, G. Drastal & R. Rivest (Eds), Computational Learning Theory and Natural Learning Systems, Constraints and Prospects, Vol. 1. Cambridge, MA: MIT Press, Chapter 19, pp. 523-560.
37. Robins, A. (1996a) Transfer in cognition. Connection Science, 8, 185-203.
38. Robins, A. (1996b) Consolidation in neural networks and in the sleeping brain. Connection Science, 8, 259-275.
39. Rumelhart, D.E. & McClelland, J.L. (1986) On learning the past tenses of English verbs. In D.E. Rumelhart & J.L. McClelland (Eds), Parallel Distributed Processing: Explorations in the Microstructure of Cognition, Vol. 2. Cambridge, MA: MIT Press (Bradford Books), pp. 216-271.
40. Rumelhart, D.E., Hinton, G.E. & Williams, R.J. (1987) Learning internal representations by error propagation. In D.E. Rumelhart & J.L. McClelland (Eds), Parallel Distributed Processing: Explorations in the Microstructure of Cognition, Vol. 1. Cambridge, MA: MIT Press (Bradford Books), pp. 318-362.
41. Sharkey, N.B. & Sharkey, A.J.C. (1993) Adaptive generalization. Artificial Intelligence Review, 7, 313-328.
42. Sharkey, N.E., Sharkey, A.J.C. & Jackson, S.A. (1996) Are SRNs sufficient for modeling language acquisition. In P. Broeder & J. Murre (Eds), Models of Language Learning: Inductive and Deductive Approaches. Cambridge, MA: MIT Press.
43. Shrager, J. & Blumenthal, D. (1992) Transfer in recurrent networks: A preliminary report and request for advice. Electronic mail message 〈92Jun29.011830pdt.38019@huh.parc.xerox.com〉, archived by Connectionists-Request@cs.cmu.edu.
44. Shultz, T.R., Tetewsky, S.J. & Takane, Y. (1995) Training regimens and function compatibility: Implications for understanding the effects of knowledge on concept learning. In Proceedings of the 17th Annual Conference of the Cognitive Science Society.
45. Silver, D.L. & Mercer, R.E. (1995) Toward a model of consolidation: The retention and transfer of neural net task knowledge. In Proceedings of the INNS World Congress on Neural Networks, Vol. III. Hillsdale, NJ: Lawrence Erlbaum, pp. 164-169.
46. Silver, D.L. & Mercer, R.E. (1996) The parallel transfer of task knowledge using dynamic learning rates based on a measure of relatedness. Connection Science, 8, 277-294.
47. Singh, S.P. (1992) Transfer of learning by composing solutions for elemental sequential tasks. Machine Learning, 8, 323-339.
48. Sony Corporation (1992) Neural network with normalized learning constant for high-speed stable learning. US Patent number 5 093 899 awarded 3/92.
49. Suddarth, S. (1990) Rule-injection hints as a means of improving network performance and learning time. In Proceedings of the EUROSIP Workshop on Neural Networks. Berlin: Springer-Verlag.
50. Suddarth, S.C. & Holden, A. (1991) Symbolic neural systems and the use of hints for developing complex systems. International Journal of Man-Machine Studies, 35, 291-312.
51. Sutton, R.S. (1992) Adapting bias by gradient descent: An incremental version of delta-bar-delta. In Proceedings of AAAI-92, pp. 171-176.
52. Tenorio, M.F. & Lee, W.-T. (1989) Self-organizing neural network for optimum supervised learning. Technical Report TR-EE 89-30. Purdue University School of Electrical Engineering, West Lafayette, IN.
53. Thrun, S. (1996) Explanation-based Neural Network Learning: A Lifelong Learning Approach. Boston, MA: Kluwer.
54. Thrun, S.B. & Mitchell, T.M. (1993) Lifelong robot learning. In L. Steels (Ed.), Proceedings of the NATO ASI: The Biology and Technology of Intelligent Autonomous Agents.
55. Thrun, S. & Mitchell, T.M. (1995) Learning one more thing. In Proceedings of IJCAI-95. Canada, Montreal, pp. 930-976.
56. Thrun, S. & O'Sullivan, J. (1995) Clustering learning tasks and the selective cross-task transfer of knowledge. Technical Report CMU-CS-95-209. Carnegie Mellon University, School of Computer Science, Pittsburgh, PA.
57. Toates, F.M. (1980) Animal Behavior: A Systems Approach. New York: Wiley.
58. Towell, G.G. & Shavlik, J.W. (1992) Interpretation of artificial neural networks: Mapping knowledge-based neural networks into rules. In Advances in Neural Information Processing Systems 4. San Mateo, CA: Morgan Kaufmann, pp. 977-984.
59. Towell, G.G., Shavlik, J.W. & Noordewier, M.O. (1990) Refinement of approximate domain theories by knowledge-based neural networks. In Proceedings of AAAI-90. San Mateo, CA: Morgan Kaufmann, pp. 861-866.
60. Waibel, A. (1989) Modular construction of time-delay neural networks for speech recognition. Neural Computation, 1, 39-46.
61. Watrous, R. & Towell, G. (1995) A patient-adaptive neural network ECG patient monitoring algorithm. In Proceedings Computers in Cardiology. Vienna, Austria, pp. 229-232.
62. Wynne-Jones, M. (1992) Node splitting: A constructive algorithm for feed-forward neural networks. In J.E. Moody, S.J. Hanson & R.P. Lippmann (Eds), Advances in Neural Information Processing Systems 4. San Mateo, CA: Morgan Kaufmann, pp. 1072-1079.