Development of goal-directed action selection guided by intrinsic motivations: An experiment with children

Experimental Brain Research

Volumn 232, Issue 7, 2014, Pages 2167-2177

  Taffoni, Fabrizio ^a   Tamilia, Eleonora ^a   Focaroli, Valentina ^b   Formica, Domenico ^a   Ricci, Luca ^a   Di Pino, Giovanni ^c   Baldassarre, Gianluca ^d   Mirolli, Marco ^d   Guglielmelli, Eugenio ^a   Keller, Flavio ^b  

^a Laboratory of Biomedical Robotics and Biomicrosystems University Campus Bio Medico   (Italy)

^b University Campus Bio Medico   (Italy)

^c UNIVERSITY CAMPUS BIO MEDICO   (Italy)

^d INSTITUTE OF COGNITIVE SCIENCES AND TECHNOLOGIES   (Italy)

Author keywords

Action selection; Action outcome contingency; Curiosity; Intrinsic motivation; Novelty detection

Indexed keywords

ARTICLE; CHILD; CHILD BEHAVIOR; CONTROLLED STUDY; DECISION MAKING; EXPLORATORY BEHAVIOR; GOAL ATTAINMENT; GROUPS BY AGE; HUMAN; HUMAN EXPERIMENT; LEARNING; MOTIVATION; NORMAL HUMAN; PRESCHOOL CHILD; PRIORITY JOURNAL; SKILL; STIMULUS RESPONSE; AGE; CHILD DEVELOPMENT; FEMALE; MALE; PHYSIOLOGY; PSYCHOMOTOR PERFORMANCE; RECREATION; TIME;

AGE FACTORS; CHILD; CHILD DEVELOPMENT; CHILD, PRESCHOOL; CHOICE BEHAVIOR; EXPLORATORY BEHAVIOR; FEMALE; GOALS; HUMANS; LEARNING; MALE; MOTIVATION; PLAY AND PLAYTHINGS; PSYCHOMOTOR PERFORMANCE; TIME FACTORS;

EID: 84902550180     PISSN: 00144819     EISSN: 14321106     Source Type: Journal    
DOI: 10.1007/s00221-014-3907-z     Document Type: Article

References (45)

1. Baldassarre G (2011) What are intrinsic motivations? A biological perspective. In: IEEE international conference on development and learning (ICDL), vol 2, pp 1-8
2. Baldassarre G, Mirolli M (2013) Intrinsically motivated learning in natural and artificial systems. Springer, Berlin
3. Baldassarre G, Mannella F, Fiore VG, Redgrave P, Gurney K, Mirolli M (2013) Intrinsically motivated action-outcome learning and goal-based action recall: a system-level bio-constrained computational model. Neural Netw 41:168-187
4. Balleine BW, Dickinson A (1998) Goal-directed instrumental action: contingency and incentive learning and their cortical substrates. Neuropharmacology 37(4):407-419 (Pubitemid 28306930)
5. Balleine BW, Daw ND, O'Doherty JP (2008) Multiple forms of value learning and the function of dopamine. In: Glimcher PW, Camerer CF, Poldrack RA, Fehr E (eds) Neuroeconomics: decision making and the Brain. Academic Press, New York, pp 367-385
6. Baranes A, Oudeyer PY (2013) Active learning of inverse models with intrinsically motivated goal exploration in robots. Robot Auton Syst 61(1):49-73
7. Barto A, Mirolli M, Baldassarre G (2013) Novelty or surprise? Front Psychol 4:907. doi:1.10.3389/fpsyg.2013.00907
8. Berlyne DE (1960) Conflict, arousal, and curiosity. McGraw-Hill Book, New York
9. Botvinick MM, Niv Y, Barto AC (2009) Hierarchically organized behavior and its neural foundations: a reinforcement learning perspective. Cognition 113(3):262-280
10. Chentanez N, Barto AG, Singh SP (2004) Intrinsically motivated reinforcement learning. In: Saul LK, Weiss Y, Bottou L (eds) Advances in neural information processing systems 17. MIT Press, Cambridge, pp 1281-1288
11. De Charms R (1968) Personal causation: the internal affective determinants of behavior. Academic Press, New York
12. Di Pino G, Porcaro C, Tombini M, Assenza G, Pellegrino G, Tecchio F, Rossini P (2012) A neurally-interfaced hand prosthesis tuned interhemispheric communication. Restor Neurol Neurosci 30(5):407-418
13. Elsner B, Hommel B (2004) Contiguity and contingency in actioneffect learning. Psychol Res 68(2-3):138-154
14. Ferreri F, Ponzo D, Vollero L, Guerra A, Di Pino G, Petrichella S, Benvenuto A, Tombini M, Rossini L, Denaro L et al (2014) Does an intraneural interface short-term implant for robotic hand control modulate sensorimotorcortical integration? An EEG-TMS co-registration study on a human amputee. Restor Neurol Neurosci 32(2):281-292
15. Gopnik A, Meltzoff A, Kuhl P (1999) The scientist in the crib: what early learning tells us about the mind. Harper-Collins, New York, p 1
16. Gopnik A, Glymour C, Sobel DM, Schulz LE, Kushnir T, Danks D (2004) A theory of causal learning in children: causal maps and bayes nets. Psychol Rev 111(1):3
17. Gottlieb J, Oudeyer PY, Lopes M, Baranes A (2013) Information-seeking, curiosity, and attention: computational and neural mechanisms. Trends Cogn Sci 17(11):585-593
18. Hickling AK, Wellman HM (2001) The emergence of children's causal explanations and theories: evidence from everyday conversation. Dev Psychol 37(5):668
19. Oudeyer PY, Kaplan F (2007) What is intrinsic motivation? a typology of computational approaches. Front Neurosci 1:6
20. Keen R (2011) The development of problem solving in young children: a critical cognitive skill. Annu Rev Psychol 62:1-21
21. Kenward B, Folke S, Holmberg J, Johansson A, Gredebäck G (2009) Goal directedness and decision making in infants. Dev Psychol 45(3):809
22. Kumaran D, Maguire EA (2007) Which computational mechanisms operate in the hippocampus during novelty detection? Hippocampus 17(9):735-748 (Pubitemid 47493455)
23. Mascalzoni E, Regolin L, Vallortigara G, Simion F (2013) The cradle of causal reasoning: newborns preference for physical causality. Dev Sci 16(3):327-335
24. Mirolli M, Baldassarre G (2013) Functions and mechanisms of intrinsic motivations: the knowledge versus competence distinction. In: Baldassarre G, Mirolli M (eds) Intrinsically motivated learning in natural and artificial systems. Springer, Berlin, pp 49-72
25. Mirolli M, Santucci V, Baldassarre G (2013) Phasic dopamine as a prediction error signal of intrinsic and extrinsic reinforcements: a computational model. Neural Netw 39:40-51
26. Moll H, Tomasello M (2010) Infant cognition. Curr Biol 20(20):R872-R875
27. Nehmzow U, Gatsoulis Y, Kerr E, Condell J, Siddique N, McGinnity TM (2013) Novelty detection as an intrinsic motivation for cumulative learning robots. In: Baldassarre G, Mirolli M (eds) Intrinsically motivated learning in natural and artificial systems. Springer, New York, pp 185-207
28. Piaget J, Cook MT (1952) The origins of intelligence in children. W.W. Norton, New York
29. Polich J (2007) Updating p300: an integrative theory of p3a and p3b. Clin Neurophysiol 118(10):2128-2148
30. Ranganath C, Rainer G (2003) Neural mechanisms for detecting and remembering novel events. Nat Rev Neurosci 4(3):193-202 (Pubitemid 37280184)
31. Redgrave P, Gurney K (2006) The short-latency dopamine signal: a role in discovering novel actions? Nat Rev Neurosci 7(12):967-975
32. Redgrave P, Vautrelle N, Reynolds J (2011) Functional properties of the basal ganglia's re-entrant loop architecture: selection and reinforcement. Neuroscience 198:138-151
33. Ryan RM, Deci EL (2000) Intrinsic and extrinsic motivations: classic definitions and new directions. Contemp Educ Psychol 25(1):54-67
34. Santucci VG, Baldassarre G, Mirolli M (2013) Which is the best intrinsic motivation signal for learning multiple skills? Front Neurorobot 7:22. doi:10.3389/fnbot.2013.00022
35. Schembri M, Mirolli M, Baldassarre G (2007) Evolving internal reinforcers for an intrinsically motivated reinforcement-learning robot. In: IEEE 6th international conference on development and learning (ICDL 2007), pp 282-287
36. Schmidhuber J (1991) Curious model-building control systems. In: IEEE international joint conference on neural networks, pp 1458-1463
37. Singh S, Lewis RL, Barto AG, Sorg J (2010) Intrinsically motivated reinforcement learning: an evolutionary perspective. IEEE Trans Auton Ment Dev 2(2):70-82
38. Smith L, Gasser M (2005) The development of embodied cognition: six lessons from babies. Artif Life 11(1-2):13-29 (Pubitemid 40394094)
39. Spelke ES, Breinlinger K, Macomber J, Jacobson K (1992) Origins of knowledge. Psychol Rev 99(4):605
40. Taffoni F, Formica D, Zompanti A, Mirolli M, Balsassarre G, Keller F, Guglielmelli E (2012a) A mechatronic platform for behavioral studies on infants. In: 4th IEEE RAS and EMBS international conference on biomedical robotics and biomechatronics (BioRob), pp 1874-1878
41. Taffoni F, Vespignani M, Formica D, Cavallo G, Di Sorrentino EP, Sabbatini G, Truppa V, Mirolli M, Baldassarre G, Visalberghi E et al (2012b) A mechatronic platform for behavioral analysis on nonhuman primates. J Integr Neurosci 11(01):87-101
42. Taffoni F, Formica D, Schiavone G, Scorcia M, Tomassetti A, di Sorrentino EP, Sabbatini G, Truppa V, Mannella F, Fiore V et al (2013) The "mechatronic board": a tool to study intrinsic motivations in humans, monkeys, and humanoid robots. In: Baldassarre G, Mirolli M (eds) Intrinsically motivated learning in natural and artificial systems. Springer, New York, pp 411-432
43. Téglás E, Vul E, Girotto V, Gonzalez M, Tenenbaum JB, Bonatti L (2011) Pure reasoning in 12-month-old infants as probabilistic inference. Science 332(6033):1054-1059
44. von Hofsten C (2004) An action perspective on motor development. Trends Cogn Sci 8(6):266-272
45. White RW (1959) Motivation reconsidered: the concept of competence. Psychol Rev 66(5):297