A multimodal learning interface for grounding spoken language in sensory perceptions

ICMI'03: Fifth International Conference on Multimodal Interfaces

Volumn , Issue , 2003, Pages 164-171

  Yu, Chen ^a   Ballard, Dana H ^a  

^a University of Rochester   (United States)

Author keywords

Language Acquisition; Machine Learning; Multimodal Integration

Indexed keywords

ARTIFICIAL INTELLIGENCE; COGNITIVE SYSTEMS; LINGUISTICS; MARKOV PROCESSES; MATHEMATICAL MODELS; MODAL ANALYSIS; NATURAL LANGUAGE PROCESSING SYSTEMS; SENSORY PERCEPTION; USER INTERFACES;

COGNITIVE SIMULATIONS; LANGUAGE ACQUISITION; MULTIMODAL INTEGRATION; SENSORY PERCEPTIONS;

LEARNING SYSTEMS;

EID: 18844382379     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/958462.958465     Document Type: Conference Paper

References (22)

1. R. Adams and L. Bischof. Seeded region growing. IEEE Transaction on Pattern Analysis and Machine Intelligence, 16(6):641-647, June 1994.
2. C. C. Aggarwal and P. S. Yu. Finding generalized projected clusters in high dimensional spaces. In SIGMOD, Dallas, Texas, USA, 2000. ACM.
3. D. H. Ballard and C. Yu. A multimodal learning interface for word acquisition. In Proceedings of the International Conference on Acoustics, Speech and Signal Processing, Hong Kong, April 2003.
4. P. F. Brown, S. Pietra, V. Pietra, and R. L. Mercer. The mathematics of statistical machine translation:parameter estimation. Computational Linguistics, 19(2), 1993.
5. L. Campbell, D. Becker, A. Azarbayejani, A. Bobick, and A. Pentland. Invariant features for 3-d gesture recognition. In Second International Workshop on Face and Gesture Recognition, pages 157-162, Killington, VT, Oct. 1996.
6. J. Hartigan. Clustering Algorithms. Wiley, New York, 1975.
7. M. Hayhoe. Visual routines: A functional account of vision. Visual Cognition, 7:43-64, 2000.
8. M. Land, N. Mennie, and J. Rusted. The roles of vision and eye movements in the control of activities of daily living. Perception, 28:1311-1328, 1999.
9. R. P. Lippmann. Review of neural networks for speech recognition. Neural computation, 1(1):1-38, 1989.
10. B. W. Mel. Seemore: Combining color, shape, and texture histogramming in a neurally inspired approach to visual object recognition. Neural Computation, 9:777-804, 1997.
11. T. Oates, L. Firoiu, and P. R. Cohen. Clustering time series with hidden markov models and dynamic time warping. In Proceedings of the IJCAI-99 Workshop on Neural, Symbolic and Reinforcement Learning Methods for Sequence Learning, pages 17-21, 1999.
12. S. Oviatt. Multimodal interfaces. In J. Jacko and A. Sears, editors, Handbook of Human-Computer Interaction. Lawrence Erlbaum, New Jersey, 2002.
13. L. R. Rabiner and B. Juang. A tutorial on hidden markov models and selected applications in speech recognition. Proceedings of the IEEE, 77(2):257-286, 1989.
14. T. Robinson. An application of recurrent nets to phone probability estimation. IEEE Transactions on Neural Networks, 5(2):298-305, 1994.
15. D. Roy and A. Pentland. Learning words from sights and sounds: A computational model. Cognitive Science, 26(1):113-146, 2002.
16. D. D. Salvucci and J. Anderson. Tracking eye movement protocols with cognitive process models. In Proceedings of the Twentieth Annual Conference of the Cognitive Science Society, pages 923-928, LEA: Mahwah, NJ, 1998.
17. B. Schiele and J. L. Crowley. Recognition without correspondence using multidimensional receptive field histograms. International Journal of Computer Vision, 36(1):31-50, 2000.
18. J. M. Siskind. Grounding language in perception, artificial Intelligence Review, 8:371-391, 1995.
19. P. Smyth. Clustering sequences with hidden markov models. In M. C. Mozer, M. I. Jordan, and T. Petsche, editors, Advances in Neural Information Processing Systems, page 648. The MIT Press, 1997.
20. M. J. Swain and D. Ballard. Color indexing. International Journal of Computer Vision, 7:11-32, 1 1991.
21. S. Wang and J. M. Siskind. Image segmentation with ratio cut. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2003.
22. C. Yu and D. H. Ballard. Learning to recognize human action sequences. In IEEE Proceedings of the 2nd International Conference on Development and Learning, pages 28-34, Boston, MA, 2002.