Neural Modularity Helps Organisms Evolve to Learn New Skills without Forgetting Old Skills

PLoS Computational Biology

Volumn 11, Issue 4, 2015, Pages

  Ellefsen, Kai Olav ^a   Mouret, Jean Baptiste ^b,c   Clune, Jeff ^d  

^a NORWEGIAN UNIVERSITY OF SCIENCE AND TECHNOLOGY   (Norway)

^b SORBONNE UNIVERSITÉ   (France)

^c UNIVERSITÉ PIERRE ET MARIE CURIE   (France)

^d UNIVERSITY OF WYOMING   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

LEARNING ALGORITHMS; SEPARATION;

CATASTROPHIC FORGETTING; LEARN+; LEARNING INTERFERENCE; LEARNING MODULES; MODULAR NEURAL NETWORKS; NEURAL LEARNING ALGORITHMS; NEURAL-NETWORKS; REINFORCEMENT LEARNINGS; SENSORY PROCESSING; SUBFIELDS;

REINFORCEMENT LEARNING;

ARTICLE; CATASTROPHIC FORGETTING; CONTROLLED STUDY; EVOLUTIONARY ALGORITHM; INFORMATION PROCESSING; KNOWLEDGE; LEARNING; MENTAL PERFORMANCE; NERVE CELL NETWORK; NEUROMODULATION; REINFORCEMENT; REWARD; SKILL; TASK PERFORMANCE; ANIMAL; ARTIFICIAL NEURAL NETWORK; BIOLOGICAL MODEL; BIOMIMETICS; COMPUTER SIMULATION; HUMAN; INHIBITION (PSYCHOLOGY); LONG TERM MEMORY; PHYSIOLOGY; PROCEDURES; RECALL;

ANIMALIA;

ANIMALS; BIOMIMETICS; COMPUTER SIMULATION; HUMANS; INHIBITION (PSYCHOLOGY); LEARNING; MENTAL RECALL; MODELS, NEUROLOGICAL; NERVE NET; NEURAL NETWORKS (COMPUTER); RETENTION (PSYCHOLOGY); TASK PERFORMANCE AND ANALYSIS;

EID: 84929492825     PISSN: 1553734X     EISSN: 15537358     Source Type: Journal    
DOI: 10.1371/journal.pcbi.1004128     Document Type: Article

References (78)

1. French R, (1999) Catastrophic forgetting in connectionist networks. Trends Cogn Sci 3: 128–135. doi: 10.1016/S1364-6613(99)01294-2 10322466
2. Mermillod M, Bugaiska A, Bonin P, (2013) The Stability-Plasticity Dilemma: Investigating the Continuum from Catastrophic Forgetting to Age-Limited Learning Effects. Front Psychol 4: 504. doi: 10.3389/fpsyg.2013.00504 23935590
3. Ajemian R, D’Ausilio A, Moorman H, Bizzi E, (2013) A theory for how sensorimotor skills are learned and retained in noisy and nonstationary neural circuits. Proc Natl Acad Sci U S A 110: E5078–87. doi: 10.1073/pnas.1320116110 24324147
4. Haykin SS, (2009) Neural networks and learning machines. New York: Prentice Hall, 3 edition.
5. Floreano D, Mattiussi C, (2008) Bio-inspired artificial intelligence: theories, methods, and technologies. The MIT Press, 659 pp.
6. Widrow B, Lehr M, (1990) 30 years of adaptive neural networks: perceptron, Madaline, and backpropagation. Proc IEEE 78: 1415–1442. doi: 10.1109/5.58323
7. Rummelhart D, Hinton G, Williams R, (1986) Learning representations by back-propagating errors. Nature 323: 533–535. doi: 10.1038/323533a0
8. Soltoggio A, (2008) Neural Plasticity and Minimal Topologies for Reward-Based Learning. In: Hybrid Intelligent Systems, 2008 HIS’08 Eighth International Conference on. IEEE, pp. 637–642.
9. McCloskey M, Cohen NJ, (1989) Catastrophic interference in connectionist networks: The sequential learning problem. Psychol Learn Motiv 24: 109–165. doi: 10.1016/S0079-7421(08)60536-8
10. Ratcliff R, (1990) Connectionist models of recognition memory: constraints imposed by learning and forgetting functions. Psychol Rev 97: 285–308. doi: 10.1037/0033-295X.97.2.285 2186426
11. Barnes JM, Underwood BJ, (1959) Fate of first-list associations in transfer theory. J Exp Psychol 58: 97–105. doi: 10.1037/h0047507 13796886
12. Aaltonen T, Adelman J, Akimoto T, Albrow MG, Alvarez González B, et al. (2009) Measurement of the top-quark mass with dilepton events selected using neuroevolution at CDF. Phys Rev Lett 102: 152001. doi: 10.1103/PhysRevLett.103.152001 19518620
13. Anemone R, Emerson C, Conroy G, (2011) Finding fossils in new ways: An artificial neural network approach to predicting the location of productive fossil localities. Evol Anthropol 20: 169–180. doi: 10.1002/evan.20324 22034235
14. Ramesh AN, Kambhampati C, Monson JRT, Drew PJ, (2004) Artificial intelligence in medicine. Ann R Coll Surg Engl 86: 334–338. doi: 10.1308/147870804290 15333167
15. Seipone T, Bullinaria J, (2005) The evolution of minimal catastrophic forgetting in neural systems. In: Proc Annu Conf Cogn Sci Soc. Cognitive Science Society, pp. 1991–1996.
16. Ans B, Rousset S, (2000) Neural networks with a self-refreshing memory: knowledge transfer in sequential learning tasks without catastrophic forgetting. Conn Sci 12: 1–19. doi: 10.1080/095400900116177
17. Alon U, (2006) An Introduction to Systems Biology: Design Principles of Biological Circuits. CRC press.
18. Carroll SB, (2001) Chance and necessity: the evolution of morphological complexity and diversity. Nature 409: 1102–1109. doi: 10.1038/35055637 11234024
19. Wagner GP, Pavlicev M, Cheverud JM, (2007) The road to modularity. Nat Rev Genet 8: 921–931. doi: 10.1038/nrg2267 18007649
20. Hintze A, Adami C, (2008) Evolution of complex modular biological networks. PLoS Comput Biol 4: e23. doi: 10.1371/journal.pcbi.0040023 18266463
21. Mountcastle VB, (1997) The columnar organization of the neocortex. Brain 120: 701–722. doi: 10.1093/brain/120.4.701 9153131
22. Lipson H, (2007) Principles of modularity, regularity, and hierarchy for scalable systems. J Biol Phys Chem 7: 125. doi: 10.4024/40701.jbpc.07.04
23. Clune J, Mouret JB, Lipson H (2013) The evolutionary origins of modularity. Proc R Soc London Ser B Biol Sci 280.
24. Burrell BD, Sahley CL, (2001) Learning in simple systems. Curr Opin Neurobiol 11: 757–764. doi: 10.1016/S0959-4388(01)00281-1 11741030
25. Soltoggio A, Bullinaria JA, Mattiussi C, Dürr P, Floreano D, (2008) Evolutionary Advantages of Neuromodulated Plasticity in Dynamic, Reward-based Scenarios. Artif Life 11: 569–576.
26. Cherniak C, Mokhtarzada Z, Rodriguez-Esteban R, Changizi K, (2004) Global optimization of cerebral cortex layout. Proc Natl Acad Sci U S A 101: 1081–1086. doi: 10.1073/pnas.0305212101 14722353
27. Ahn YY, Jeong H, Kim BJ, (2006) Wiring cost in the organization of a biological neuronal network. Physica A 367: 531–537. doi: 10.1016/j.physa.2005.12.013
28. Striedter G, (2005) Principles of brain evolution. Sinauer Associates Sunderland, MA.
29. Kortge C (1990) Episodic memory in connectionist networks. In: Proc Annu Conf Cogn Sci Soc. pp. 764–771.
30. Lewandowsky S, Li SC, Dempster F, Brainerd C, (1993) Catastrophic interference in neural networks: causes, solutions and data. In: New Perspectives on Interference and Inhibition in Cognition, Academic Press. pp. 329–361.
31. French R (1991) Using semi-distributed representations to overcome catastrophic forgetting in connectionist networks. In: Proceedings of the 13th Annual Congitive Science Society Conference. pp. 173–178.
32. French R (1994) Dynamically constraining connectionist networks to produce distributed, orthogonal representations to reduce catastrophic interference. In: Proc Annu Conf Cogn Sci Soc. pp. 335–340.
33. French R, (1997) Pseudo-recurrent connectionist networks: An approach to the’sensitivity-stability’ dilemma. Conn Sci 9: 353–379. doi: 10.1080/095400997116595
34. Robins A, (1995) Catastrophic forgetting, rehearsal and pseudorehearsal. Conn Sci 7: 123–146. doi: 10.1080/09540099550039318
35. McClelland JL, McNaughton BL, O’Reilly RC, (1995) Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory. Psychol Rev 102: 419. doi: 10.1037/0033-295X.102.3.419 7624455
36. Ans B, Rousset S, (1997) Avoiding catastrophic forgetting by coupling two reverberating neural networks. C R Acad Sci III 320: 989–997. doi: 10.1016/S0764-4469(97)82472-9
37. Xin Y, (1999) Evolving artificial neural networks. Proc IEEE 87: 1423–1447. doi: 10.1109/5.784219
38. Holland JH, (1975) Adaptation in natural and artificial systems: An introductory analysis with applications to biology, control, and artificial intelligence. U Michigan Press.
39. Stanley KO, Miikkulainen R, (2002) Evolving neural networks through augmenting topologies. Evol Comput 10: 99–127. doi: 10.1162/106365602320169811 12180173
40. Koza JR, (2003) Genetic programming IV: Routine human-competitive machine intelligence. Kluwer.
41. Hornby GS, Lohn JD, Linden DS, (2011) Computer-automated evolution of an x-band antenna for NASA’s space technology 5 mission. Evol Comput 19: 1–23. doi: 10.1162/EVCO_a_00005 20583909
42. Gomez FJ, Miikkulainen R (2003) Active guidance for a finless rocket using neuroevolution. In: Proc Genet Evol Comput Conf. pp. 2084–2095.
43. Clune J, Beckmann BE, Ofria C, Pennock RT (2009) Evolving coordinated quadruped gaits with the HyperNEAT generative encoding. In: Proc Congr Evol Comput. pp. 2764–2771.
44. Hornby GS, Takamura S, Yamamoto T, Fujita M, (2005) Autonomous evolution of dynamic gaits with two quadruped robots. IEEE Trans Robot 21: 402–410. doi: 10.1109/TRO.2004.839222
45. Hebb DO, (1949) The Organization of Behavior. New York: Wiley and Sons, 335 pp.
46. Niv Y, Joel D, Meilijson I, Ruppin E, (2002) Evolution of reinforcement learning in uncertain environments: a simple explanation for complex foraging behaviors. Adapt Behav 10: 5–24. doi: 10.1177/10597123020101001
47. Ellefsen KO (2013) Balancing the Costs and Benefits of Learning Ability. In: European Conference of Artificial Life. pp. 292–299.
48. Urzelai J, Floreano D, (2001) Evolution of adaptive synapses: Robots with fast adaptive behavior in new environments. Evol Comput 9: 495–524. doi: 10.1162/10636560152642887 11709106
49. Blynel J, Floreano D, (2002) Levels of dynamics and adaptive behavior in evolutionary neural controllers. In: 7th International Conference on Simulation on Adaptive Behavior (SAB’2002). MIT Press, pp. 272–281.
50. Hinton G, Nowlan S, (1987) How learning can guide evolution. Complex Systems 1: 495–502.
51. Nolfi S, Parisi D, Elman JL, (1994) Learning and Evolution in Neural Networks. Adapt Behav 3: 5–28. doi: 10.1177/105971239400300102
52. Mayley G, (1996) Landscapes, Learning Costs, and Genetic Assimilation. Evol Comput 4: 213–234. doi: 10.1162/evco.1996.4.3.213
53. Sasaki T, Tokoro M, (1999) Evolving Learnable Neural Networks under Changing Environments with Various Rates of Inheritance of Acquired Characters: Comparison between Darwinian and Lamarckian Evolution. Artif Life 5: 203–223. doi: 10.1162/106454699568746 10648951
54. Jay TM, (2003) Dopamine: a potential substrate for synaptic plasticity and memory mechanisms. Prog Neurobiol 69: 375–390. doi: 10.1016/S0301-0082(03)00085-6 12880632
55. Abbott LF, Regehr WG, (2004) Synaptic computation. Nature 431: 796–803. doi: 10.1038/nature03010 15483601
56. Pigliucci M, (2008) Is evolvability evolvable? Nat Rev Genet 9: 75–82. doi: 10.1038/nrg2278 18059367
57. Kashtan N, Alon U, (2005) Spontaneous evolution of modularity and network motifs. Proc Natl Acad Sci U S A 102: 13773–13778. doi: 10.1073/pnas.0503610102 16174729
58. Hoverstad BA, (2011) Noise and the evolution of neural network modularity. Artif Life 17: 33–50. doi: 10.1162/artl_a_00016 21087148
59. Espinosa-Soto C, Wagner A, (2010) Specialization can drive the evolution of modularity. PLoS Comput Biol 6: e1000719. doi: 10.1371/journal.pcbi.1000719 20360969
60. Clune J, Beckmann BE, McKinley PK, Ofria C, (2010) Investigating whether HyperNEAT produces modular neural networks. In: Genet Evol Comput Conf. ACM, pp. 635–642.
61. Verbancsics P, Stanley KO, (2011) Constraining connectivity to encourage modularity in HyperNEAT. In: Genet Evol Comput Conf. ACM, pp. 1483–1490.
62. Dunlap AS, Stephens DW, (2009) Components of change in the evolution of learning and unlearned preference. Proc R Soc B Biol Sci 276: 3201–8. doi: 10.1098/rspb.2009.0602
63. Tanaka K, (1996) Inferotemporal cortex and object vision. Annu Rev Neurosci 19: 109–139. doi: 10.1146/annurev.ne.19.030196.000545 8833438
64. Bengio Y, (2009) Learning Deep Architectures for AI. Foundations and Trends in Machine Learning 2: 1–127. doi: 10.1561/2200000006
65. Newman MEJ, (2006) Modularity and community structure in networks. Proc Natl Acad Sci U S A 103: 8577–8582. doi: 10.1073/pnas.0601602103 16723398
66. Guimerà R, Sales-Pardo M, Amaral LAN, (2004) Modularity from fluctuations in random graphs and complex networks. Phys Rev E Stat Nonlin Soft Matter Phys 70: 025101. doi: 10.1103/PhysRevE.70.025101 15447530
67. Leicht EA, Newman MEJ, (2008) Community structure in directed networks. Phys Rev Lett: 118703–118707.
68. Cohen PR, (1995) Empirical Methods for Artificial Intelligence. Cambridge, MA, USA: MIT Press, 405 pp.
69. Tonelli P, Mouret JB (2011) On the relationships between synaptic plasticity and generative systems. In: Genet Evol Comput Conf. pp. 1531–1538.
70. Stanley KO, Miikkulainen R, (2003) A taxonomy for artificial embryogeny. Artif Life 9: 93–130. doi: 10.1162/106454603322221487 12906725
71. Clune J, Stanley KO, Pennock RT, Ofria C, (2011) On the performance of indirect encoding across the continuum of regularity. IEEE Trans Evol Comput 15: 346–367. doi: 10.1109/TEVC.2010.2104157
72. Deb K, Pratap A, Agarwal S, Meyarivan T, (2002) A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Trans Evol Comput 6: 182–197. doi: 10.1109/4235.996017
73. Mouret JB, Doncieux S, (2012) Encouraging Behavioral Diversity in Evolutionary Robotics: an Empirical Study. Evol Comput 1: 91–113. doi: 10.1162/EVCO_a_00048
74. Risi S, Vanderbleek SD, Hughes CE, Stanley KO (2009) How novelty search escapes the deceptive trap of learning to learn. In: Genet Evol Comput Conf. pp. 153–160.
75. Mouret JB, Doncieux S, (2010) SFERES v2: Evolvin’ in the Multi-Core World. In: Proc Congr Evol Comput. IEEE, 2, pp. 4079–4086.
76. Tonelli P, Mouret JB, (2013) On the Relationships between Generative Encodings, Regularity, and Learning Abilities when Evolving Plastic, Artificial Neural Networks. PLoS One 8: e79138. doi: 10.1371/journal.pone.0079138 24236099
77. Brandes U, Delling D, Gaertler M, Gorke R, Hoefer M, et al. (2008) On modularity clustering. IEEE Trans Knowl Data Eng 20: 172–188. doi: 10.1109/TKDE.2007.190689
78. Fortunato S, (2010) Community detection in graphs. Phys Rep 486: 75–174. doi: 10.1016/j.physrep.2009.11.002