Knowledge transfer in deep block-modular neural networks

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Volumn 9222, Issue , 2015, Pages 268-279

  Terekhov, Alexander V ^a   Montone, Guglielmo ^a   O’Regan, J Kevin ^a  

^a UNIVERSITÉ PARIS DESCARTES   (France)

Author keywords

Deep learning; Knowledge transfer; Modular; Neural networks

Indexed keywords

BIOMIMETICS; INFORMATION MANAGEMENT; KNOWLEDGE MANAGEMENT; NEURAL NETWORKS;

DEEP LEARNING; DEEP NEURAL NETWORKS; EXISTING CAPACITY; KNOWLEDGE TRANSFER; MODULAR; MODULAR ARCHITECTURES; MODULAR NEURAL NETWORKS; SPECIALIZED TOOLS;

NETWORK ARCHITECTURE;

EID: 84947087639     PISSN: 03029743     EISSN: 16113349     Source Type: Book Series    
DOI: 10.1007/978-3-319-22979-9_27     Document Type: Conference Paper

References (23)

1. Anderson, M.L.: Neural reuse: A fundamental organizational principle of the brain. Behavioral and Srain Sciences 33(04), 245–266 (2010)
2. Bastien, F., Lamblin, P., Pascanu, R., Bergstra, J., Goodfellow, I.J., Bergeron, A., Bouchard, N., Bengio, Y.: Theano: new features and speed improvements. In: Deep Learning and Unsupervised Feature Learning NIPS 2012 Workshop (2012)
3. Bergstra, J., Breuleux, O., Bastien, F., Lamblin, P., Pascanu, R., Desjardins, G., Turian, J., Warde-Farley, D., Bengio, Y.: Theano: a CPU and GPU math expression compiler. In: Proceedings of the Python for Scientific Computing Conference (SciPy), June 2010. oral Presentation
4. Bradley, D.M., Bagnell, J.A.: Differential sparse coding (2008)
5. Collobert, R., Weston, J.: A unified architecture for natural language processing: Deep neural networks with multitask learning. In: Proceedings of the 25th International Conference on Machine Learning, pp. 160–167. ACM (2008)
6. Dehaene, S., Cohen, L.: Cultural recycling of cortical maps. Neuron 56(2), 384–398 (2007)
7. Erhan, D., Manzagol, P.A., Bengio, Y., Bengio, S., Vincent, P.: The difficulty of training deep architectures and the effect of unsupervised pre-training. In: International Conference on Srtificial Intelligence and Statistics, pp. 153–160 (2009)
8. Girosi, F., Jones, M., Poggio, T.: Regularization theory and neural networks architectures. Neural Computation 7(2), 219–269 (1995)
9. Glorot, X., Bengio, Y.: Understanding the difficulty of training deep feedforward neural networks. In: International Conference on Artificial Intelligence and Statistics, pp. 249–256 (2010)
10. Glorot, X., Bordes, A., Bengio, Y.: Deep sparse rectifier networks. In: Proceedings of the 14th International Conference on Artificial Intelligence and Statistics, JMLR W&CP, vol. 15, pp. 315–323 (2011)
11. Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv:1312.6211 (2013)
12. Graves, A., Mohamed, A.R., Hinton, G.: Speech recognition with deep recurrent neural networks. In: 2013 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 6645–6649. IEEE (2013)
13. Gutstein, S., Fuentes, O., Freudenthal, E.: Knowledge transfer in deep convolutional neural nets. International Journal on Artificial Intelligence Tools 17(03), 555–567 (2008)
14. Hinton, G., Osindero, S., Teh, Y.W.: A fast learning algorithm for deep belief nets. Neural Computation 18(7), 1527–1554 (2006)
15. Jordan, M.I., Jacobs, R.A.: A competitive modular connectionist architecture. In: Advances in Neural Information Processing Systems, pp. 767–773 (1991)
16. LeCun, Y., Bengio, Y.: Convolutional networks for images, speech, and time series. In: The Handbook of Brain Theory and Neural Networks, vol. 3361, p. 310 (1995)
17. Lee, H., Grosse, R., Ranganath, R., Ng, A.Y.: Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations. In: Proceedings of the 26th Annual International Conference on Machine Learning, pp. 609–616. ACM (2009)
18. Liu, X., Gao, J., He, X., Deng, L., Duh, K., Wang, Y.Y.: Representation learning using multi-task deep neural networks for semantic classification and information retrieval
19. Lu, B.L., Ito, M.: Task decomposition and module combination based on class relations: a modular neural network for pattern classification. IEEE Transactions on Neural Networks 10(5), 1244–1256 (1999)
20. McCloskey, M., Cohen, N.J.: Catastrophic interference in connectionist networks: The sequential learning problem. Psychology of Learning and Motivation 24, 109–165 (1989)
21. Salakhutdinov, R., Hinton, G.: An efficient learning procedure for deep boltzmann machines. Neural Computation 24(8), 1967–2006 (2012)
22. Schmidhuber, J.: Deep learning in neural networks: An overview. Neural Networks 61, 85–117 (2015)
23. Sutskever, I., Vinyals, O., Le, Q.V.: Sequence to sequence learning with neural networks. In: Advances in Neural Information Processing Systems, pp. 3104–3112 (2014)