Synching models with infants: A perceptual-level model of infant audio-visual synchrony detection

Cognitive Systems Research

Volumn 6, Issue 3 SPEC. ISS., 2005, Pages 205-228

  Berthouze, Luc ^c   Prince, Christopher G ^a   Hollich, George J ^b  

^a UNIVERSITY OF MINNESOTA DULUTH   (United States)

^b PURDUE UNIVERSITY   (United States)

^c NONE

Author keywords

[No Author keywords available]

Indexed keywords

COMMUNICATION CHANNELS (INFORMATION THEORY); LEARNING SYSTEMS; SENSORY PERCEPTION; SPEECH ANALYSIS; SPEECH PROCESSING; SYNCHRONIZATION; VISUAL COMMUNICATION;

OSCILLOSCOPE; SENSORY CHANNELS; SYNCHING MODELS; SYNCHRONY DETECTION;

COGNITIVE SYSTEMS;

ALGORITHM; ARTICLE; AUDITORY STIMULATION; BRAIN FUNCTION; CHILD DEVELOPMENT; COGNITIVE DEVELOPMENT; COMPUTER ANALYSIS; CONCEPT FORMATION; CONTROLLED STUDY; FEMALE; HUMAN; HUMAN EXPERIMENT; INFANT; LEARNING; MALE; MOTION; NORMAL DISTRIBUTION; PERCEPTION; PRIORITY JOURNAL; QUANTITATIVE ANALYSIS; SENSORIMOTOR FUNCTION; SPEECH; VISUAL STIMULATION;

EID: 19044398715     PISSN: None     EISSN: 13890417     Source Type: Journal    
DOI: 10.1016/j.cogsys.2004.11.006     Document Type: Article

References (50)

1. Arsenio A. & Fitzpatrick P. (2003, December). Exploiting cross-modal rhythm for robot perception of objects. Paper presented at the 2nd international conference on computational intelligence, robotics, and autonomous systems, Singapore
2. D.H. Ashmead R.K. Clifton & E.E. Perris Precision of auditory localization in human infants Developmental Psychology 23 1987 641-647
3. L.E. Bahrick Increasing specificity in perceptual development: Infants' detection of nested levels of multimodal stimulation Journal of Experimental Child Psychology 79 2001 253-270
4. L.E. Bahrick & R. Lickliter Intersensory redundancy guides attentional selectivity and perceptual learning in infancy Developmental Psychology 36 2000 190-201
5. L.E. Bahrick & J.S. Watson Detection of intermodal proprioceptive-visual contingency as a potential basis of self-perception in infancy Developmental Psychology 21 1985 963-973
6. D.A. Baldwin Infants' ability to consult the speaker for clues to word reference Journal of Child Language 20 1993 394-419
7. D.A. Baldwin Understanding the link between joint attention and language In C. Moore & P.J. Dunham (Eds.), Joint attention: Its origins and role in development 1995 Lawrence Erlbaum Hillsdale, NJ
8. D.A. Baldwin & E.M. Markman Establishing word-object relations: A first step Child Development 60 1989 381-398
9. Bergeson T. Houston D. & Pisoni D.B. (2004, May). Audiovisual speech perception in normal-hearing infants and hearing-impaired infants with cochlear implants. Paper presented at the 14th biennial international conference on infant studies, Chicago, IL
10. P. Bloom How children learn the meanings of words 2000 MIT Press Cambridge, MA
11. Butz T. & Thiran J.-P. (2002). Feature space mutual information in speech-video sequences. Paper presented at ICME, Lausanne, Switzerland
12. V.R. de Sa & D.H. Ballard Category learning through multimodality sensing Neural Computation 10 1998 1097-1117
13. B. Dodd Lip reading in infants: Attention to speech presented in- and out-of-synchrony Cognitive Psychology 11 1979 478-484
14. J. Driver Enhancement of selective listening by illusory mislocation of speech sounds due to lipreading Nature 381 1996 66-68
15. J.W. Fisher III T. Darrell W.T. Freeman & P. Viola Learning joint statistical models for audio-visual fusion and segregation In T.K. Leen T.G. Dietterich & V. Tresp (Eds.), Advances in neural information processing systems Vol. 13 2001 MIT Press Cambridge, MA
16. Fitzpatrick P. & Arsenio A. (2004, August). Feel the beat: Using cross-modal rhythm to integrate perception of objects, others, and self. Paper presented at the fourth international workshop on epigenetic robotics: Modeling cognitive development in robotic systems, Genoa, Italy
17. G. Gergely & J.S. Watson Early socio-emotional development: contingency perception and the social-biofeedback model In P. Rochat (Ed.), Early social cognition: Understanding others in the first months of life 1999 Lawrence Erlbaum Mahwah, NJ 101-136
18. L.J. Gogate & L.E. Bahrick Intersensory redundancy facilitates learning of arbitrary relations between vowel sounds and objects in seven-month-old infants Journal of Experimental Child Psychology 69 1998 133-149
19. L.J. Gogate & L.E. Bahrick Intersensory redundancy and 7-month-old infants' memory for arbitrary syllable-object relations Infancy 2 2001 219-231
20. L.J. Gogate L.E. Bahrick & J.D. Watson A study of multimodal motherese: The role of temporal synchrony between verbal labels and gestures Child Development 71 2000 878-894
21. A. Gopnik & A.N. Meltzoff Words, thoughts, and theories 1997 MIT Press Cambridge, MA
22. J. Hershey & J. Movellan Audio-vision: Using audio-visual synchrony to locate sounds In S.A. Solla T.K. Leen & K.-R. Müller (Eds.), Advances in neural information processing systems Vol. 12 2000 MIT Press Cambridge, MA 813-819
23. G. Hollich K. Hirsh-Pasek & R.M. Golinkoff Breaking the language barrier: An emergentist coalition model for the origins of word learning Monographs of the Society for Research in Child Development 65 3, Serial No. 262 2000
24. G.J. Hollich P.W. Jusczyk & R.S. Newman Infants' use of visual information in speech segmentation Journal of the Acoustical Society America 110 2001 2703
25. Hollich G.J. Mislivec E.J. Helder N.A. & Prince C.G. (2004, October). Are you synching what I'm synching? Modeling infants' real-time detection of audiovisual contingencies between face and voice. Poster presented at the international conference on development and learning (ICDL 04), La Jolla, CA
26. P.W. Jusczyk & R.N. Aslin Infants' detection of the sound patterns of words in fluent speech Cognitive Psychology 29 1995 1-23
27. Krichmar J.L. Nitz D.A. & Edelman G.M. (2004, October). Object recognition, adaptive behavior and learning in brain-based devices. In Proceedings of the third international conference on development and learning (ICDL), La Jolla, CA
28. D.J. Lewkowicz Developmental changes in infants' bisensory response to synchronous durations Infant Behavior and Development 9 1986 335-353
29. D.J. Lewkowicz Perception of auditory-visual temporal synchrony in human infants Journal of Experimental Psychology: Human Perception and Performance 22 1996 1094-1106
30. D.J. Lewkowicz The development of intersensory temporal perception: An epigenetic systems/limitation view Psychological Bulletin 126 2000 281-308
31. M. Lungarella G. Metta R. Pfeifer & G. Sandini Developmental robotics: A survey Connection Science 15 2003 151-190
32. Magyar J. & Gergely G. (1998, April). The obscure object of desire: "Nearly, but clearly not, like me". Perception of self-generated contingencies in normal and autistic children. Poster presented at the international conference on infant studies, Atlanta, GA, USA
33. Mislivec E.J. (2004). Audio-visual synchrony for face location and segmentation. Undergraduate Research Opportunity Project, University of Minnesota Duluth. Available from: http://www.cprince.com/PubRes/ SenseStream/
34. R.S. Newman & P.W. Jusczyk The cocktail party effect in infants Perception and Psychophysics 58 1996 1145-1156
35. H.J. Nock G. Iyengar & C. Neti Speaker localization using audio-visual synchrony: An empirical study In E.M. Bakker T.S. Huang M.S. Lew N. Sebe & X.S. Zhou (Eds.), Image and video retrieval, second international conference, CIVR 2003 Lecture Notes in Computer Science Vol. 2728 2003 Springer Berlin
36. H.J. Nock G. Iyengar & C. Neti Multimodal processing by finding common cause Communications of the ACM 47 2004 51-56
37. J. Pickens T. Field T. Nawrocki A. Martinez D. Soutullo & J. Gonzalez Full-term and preterm infants' perception of face-voice synchrony Infant Behavior and Development 17 1994 447-455
38. T. Poggio & E. Bizzi Generalization in vision and motor control Nature 431 2004 768-774
39. Prince C.G. Helder N.A. Mislivec E.J. Ang B.J. Lim M.S. & Hollich G.J. (2003, October). Taking contingency seriously in sensory-based models of learning in infants. Poster presented at the 2003 meeting of the cognitive development society, Park City, UT
40. Prince, C.G., Hollich, G.J., Helder, N.A., Mislivec, E.J., Reddy, A., Salunke, S., & Memon, N. (2004, August). Taking synchrony seriously: A perceptual-level model of infant synchrony detection. Paper presented at the fourth international workshop on epigenetic robotics: modeling cognitive development in robotic systems, Genoa, Italy. Available from: http://www.cprince.com/PubRes/EpiRob04/
41. Prince, C.G., & Mislivec, E.J. (2004). Investigating the sensory grounding hypothesis: A sensory-augmented connectionist word-learning model. Unpublished manuscript
42. Z.W. Pylyshyn Computation and cognition: Towards a foundation for cognitive science 1984 MIT Press Cambridge, MA
43. L.D. Rosenblum & H.M. Saldaña An audiovisual test of kinematic primitives for visual speech perception Journal of Experimental Psychology: Human Perception and Performance 22 1996 318-331
44. T.R. Shultz Computational developmental psychology 2003 MIT Press Cambridge, MA
45. M. Slaney M. Covell FaceSync: A linear operator for measuring synchronization of video facial images and audio tracks Proceedings of neural information processing society Vol. 13 2001 MIT Press Cambridge, MA
46. E.S. Spelke Infants' intermodal perception of events Cognitive Psychology 8 1976 553-560
47. Viola P. & Jones M.J. (2001). Robust Real-time Object Detection. Compaq Cambridge Research Laboratory, Technical Report Series CRL 2001/ 01, February 2001
48. A. Vouloumanos & J.F. Werker Tuned to the signal: The privileged status of speech for young infants Developmental Science 7 2004 270-276
49. Vuppla, K. (2004). Evaluation of two synchrony detection implementations. Masters Thesis, Computer Science Department, University of Minnesota Duluth. Available from: http://www.cprince.com/PubRes/ VupplaThesis04/
50. J. Weng J. McClelland A. Pentland O. Sporns I. Stockman M. Sur et al. Autonomous mental development by robots and animals Science 291 2001 599-600