Audio-visual group recognition using diffusion maps

IEEE Transactions on Signal Processing

Volumn 58, Issue 1, 2010, Pages 403-413

  Keller, Yosi ^a   Coifman, Ronald R ^b   Lafon, Stéphane ^c   Zucker, Steven W ^d  

^a BAR ILAN UNIVERSITY   (Israel)

^b YALE UNIVERSITY   (United States)

^c GOOGLE INC   (United States)

^d Yale University   (United States)

Author keywords

Dimensionality reduction; Laplacian eigenmaps; Multisensor; Sensor fusion; Speech recognition

Indexed keywords

APRIORI; AUDIO-VISUAL; BELTRAMI; DATA SOURCE; DATA-DRIVEN; DETERMINISTIC MODELS; DIFFUSION MAPS; DIMENSIONALITY REDUCTION; IN-DIFFUSION; INPUT CHANNELS; LAPLACIAN EIGENMAPS; MULTI SENSOR; MULTISENSORY DATA; PERFORMANCE IMPROVEMENTS; PHYSICAL SYSTEMS; SAMPLING DENSITIES; SENSOR FUSION; SPECTRAL DIFFUSION;

DIFFUSION; LAPLACE TRANSFORMS; SENSORS;

SPEECH RECOGNITION;

EID: 73049098610     PISSN: 1053587X     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSP.2009.2030861     Document Type: Article

References (27)

1. J. Driver, "Enhancement of selective listening by illusory mislocation of speech sounds due to lip-reading," Nature, no.381, pp. 66-68, 1996.
2. Y. Gutfreund, W. Zheng, and E. I. Knudsen, "Gated visual input to the central auditory system," Science, no.297, pp. 1556-1559, 2002.
3. T. Hastie, R. Ribshirani, and J. Friedman, The Elements of Statistical Learning: Data Mining, Inference and Prediction. New York: Springer, 2002.
4. R. Coifman, S. Lafon, A. Lee, M. Maggioni, B. Nadler, F.Warner, and S. W. Zucker, "Geometric diffusions as a tool for harmonic analysis and structure definition of data: Diffusion maps," Proc. Nat. Acad. Sci., vol.102, no.21, pp. 7426-7431, May 2005. (Pubitemid 40740988)
5. R. Coifman, S. Lafon, A. Lee, M. Maggioni, B. Nadler, F.Warner, and S. Zucker, "Geometric diffusions as a tool for harmonics analysis and structure definition of data: Multiscale methods," Proc. Nat. Acad. Sci., vol.102, no.21, pp. 7432-7437, May 2005. (Pubitemid 40740989)
6. R. Coifman and S. Lafon, "Diffusion maps," Appl. Comput. Harmon. Anal. (Special Issue on Diffusion Maps and Wavelets), vol.22, pp. 5-30, Jul. 2006.
7. A. Gueziec, X. Pennec, and N. Ayache, "Medical image registration using geometric hashing," IEEE Comput. Sci. Eng. Mag. (Special Issue on Geometric Hashing), vol.4, pp. 29-41, Oct.-Dec. 1997.
8. H. Li, B. S. Manjunath, and S. K. Mitra, "A contour-based approach to multisensor image registration," IEEE Trans. Image Process., vol.4, pp. 320-334, Mar. 1995.
9. R. Sharma and M. Pavel, "Registration of video sequences from multiple sensors," in Proc. Image Registr. Workshop (NASA GSFC), 1997, pp. 361-366.
10. P. Viola, I. Wells, and W. M. , "Alignment by maximization of mutual information," in Proc. 5th Int. Conf. Comput. Vision, Jun. 1995, pp. 16-23.
11. U. Ozertem and D. Erdogmus, "Information regularized sensor fusion: Application to localization with distributed motion sensors," J. VLSI Signal Process. Syst., vol.49, no.2, pp. 291-299, 2007.
12. J. Ham, D. Lee, and L. Saul, "Semisupervised alignment of manifolds," in Proc. 10th Int. Workshop Artif. Intell. Statist., Jan. 6-8, 2005, pp. 120-127.
13. E. Kidron, Y. Schechner, and M. Elad, "Cross-modal localization via sparsity," IEEE Trans. Signal Process., vol.55, pp. 1390-1404, Apr. 2007.
14. M. E. Sargin, Y. Yemez, E. Erzin, and A. M. Tekalp, "Audiovisual synchronization and fusion using canonical correlation analysis," IEEE Trans. Multimedia, vol.9, no.7, pp. 1396-1403, 2007.
15. S. Lafon, Y. Keller, and R. R. Coifman, "Data fusion and multi-cue data matching by diffusion maps," IEEE Trans. Pattern Anal. Machine Intell., vol. 28, no. 11, pp. 1784-1797, 2006.
16. F. R. Bach, G. R. G. Lanckriet, and M. I. Jordan, "Multiple kernel learning, conic duality, and the smo algorithm," in Proc. ACM 21st Int. Conf. Machine Learn. (ICML '04), New York, 2004, p. 6.
17. F. Bach, "Exploring large feature spaces with hierarchical multiple kernel learning," in Proc. NIPS, 2008, pp. 105-112.
18. G. R. Lanckriet, M. Deng, N. Cristianini, M. I. Jordan, and W. S. Noble, "Kernel-based data fusion and its application to protein function prediction in yeast," in Proc. Pacific Symp. Biocomput., Jan. 2004, pp. 300-311.
19. P. Pavlidis, J. Weston, J. Cai, and W. N. Grundy, "Gene functional classification from heterogeneous data," in Proc. 5th Annu. Int. Conf. Comput. Mol. Biol., New York, 2001, pp. 242-248.
20. D. Dewasurendra, P. Bauer, and K. Premaratne, "Evidence filtering," IEEE Trans. Signal Process., vol. 55, pp. 5796-5805, Dec. 2007.
21. Y. Zhu, E. Song, J. Zhou, and Z. You, "Optimal dimensionality reduction of sensor data in multisensor estimation fusion," IEEE Trans. Signal Process., vol. 53, pp. 1631-1639, May 2005.
22. M. Belkin and P. Niyogi, "Laplacian eigenmaps for dimensionality reduction and data representation," Neural Comput., vol. 6, no. 15, pp. 1373-1396, Jun. 2003.
23. M. Balasubramanian, E. L. Schwartz, J. B. Tenenbaum, V. de Silva, and J. C. Langford, "The Isomap algorithm and topological stability," Science, vol. 295, no. 5552, p. 7, 2002.
24. C. Fowlkes, S. Belongie, F. Chung, and J. Malik, "Spectral grouping using the nyström method," IEEE Trans. Pattern Anal. Machine Intell., vol. 26, no. 2, pp. 214-225, 2004.
25. R. R. Coifman, Y. Shkolnisky, F. J. Sigworth, and A. Singer, "Graph laplacian tomography from unknown random projections," IEEE Trans. Image Process., vol. 10, pp. 1891-1899, Oct. 2008.
26. A. Singer, "From graph to manifold laplacian: The convergence rate," Appl. Comput. Harmon. Anal., vol. 1, no. 21, pp. 135-144, 2006.
27. I. Guyon and A. Elisseeff, "An introduction to variable and feature selection," J. Machine Learn. Res., vol. 3, pp. 1157-1182, 2003.