Analysis of multimodal sequences using geometric video representations

Signal Processing

Volumn 86, Issue 12, 2006, Pages 3534-3548

  Monaci, Gianluca ^a   Escoda, Òscar Divorra ^a   Vandergheynst, Pierre ^a  

^a EPFL   (Switzerland)

Author keywords

Audiovisual association; Cross modal localization; Geometric video representation; Multimodal data processing; Sparse redundant decomposition

Indexed keywords

AUDIOVISUAL ASSOCIATION; CROSS MODAL LOCALIZATION; GEOMETRIC VIDEO REPRESENTATION; MULTIMODAL DATA PROCESSING; SPARSE REDUNDANT DECOMPOSITION;

FACE RECOGNITION; MODAL ANALYSIS; MOTION ESTIMATION; SIGNAL PROCESSING;

DATA PROCESSING;

EID: 33749427593     PISSN: 01651684     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.sigpro.2006.02.044     Document Type: Article

References (25)

1. J. Hershey, J. Movellan, Audio-vision: Using audio-visual synchrony to locate sounds, in: Proceedings of NIPS, vol. 12, 1999.
2. M. Slaney, M. Covell, FaceSync: a linear operator for measuring synchronization of video facial images and audio tracks, in: Proceedings of NIPS, vol. 13, 2000.
3. H.J. Nock, G. Iyengar, C. Neti, Speaker localisation using audio-visual synchrony: an empirical study, in: Proceedings of the 10th ACM International Conference on Multimedia, 2002.
4. J.W. Fisher III, T. Darrell, W.T. Freeman, P. Viola, Learning joint statistical models for audio-visual fusion and segregation, in: Proceedings of NIPS, vol. 13, 2000.
5. Fisher III J.W., and Darrell T. Speaker association with signal-level audiovisual fusion. IEEE Trans. Multimedia 6 3 (2004) 406-413
6. Butz T., and Thiran J.-P. From error probability to information theoretic (multi-modal) signal processing. Signal Processing 85 5 (2005) 875-902
7. P. Besson, M. Kunt, T. Butz, J.-P. Thiran, A multimodal approach to extract optimized audio features for speaker detection, in: Proceedings of EUSIPCO, 2005.
8. P. Smaragdis, M. Casey, Audio/visual independent components, in: Proceedings of ICA, 2003, pp. 709-714.
9. Wang Y., Ostermann J., and Zhang Y.-Q. Digital Video Processing and Communications (2001), Prentice-Hall, Englewood Cliffs, NJ
10. Daubechies I. Time-frequency localization operators: a geometric phase space approach. IEEE Trans. Inform. Theory 34 4 (1988) 605-612
11. Mallat S., and Zhang Z. Matching pursuits with time-frequency dictionaries. IEEE Trans. Signal Process. 41 12 (1993) 3397-3415
12. Chen S.S., Donoho D.L., and Saunders M.A. Atomic decomposition by basis pursuit. SIAM J. Sci. Comput. 20 1 (1998) 33-61
13. Burt P.J., and Adelson E.H. The Laplacian pyramid as a compact image code. IEEE Trans. Comm. 31 4 (1983) 532-540
14. L. Peotta, L. Granai, P. Vandergheynst, Very low bit rate image coding using redundant dictionaries, in: Proceedings of the SPIE, Wavelets: Applications in Signal and Image Processing X, vol. 5207, 2003, pp. 228-239.
15. O. Divorra Escoda, Toward sparse and geometry adapted video approximations, Ph.D. Thesis, EPFL, Lausanne. Available: 〈http://lts2www.epfl.ch/〉, June 2005 (online).
16. O. Divorra Escoda, P. Vandergheynst, A Bayesian approach to video expansions on parametric over-complete 2-D dictionaries, in: Proceedings of IEEE MMSP, 2004, pp. 490-493.
17. Rabiner L., and Juang B.-H. Fundamentals of Speech Recognition (1993), Prentice-Hall, Englewood Cliffs, NJ
18. R. Gribonval, E. Bacry, S. Mallat, P. Depalle, X. Rodet, Analysis of sound signals with high resolution matching pursuit, in: Proceedings of IEEE TFTS, 1996, pp. 125-128.
19. Anderson T.W. An Introduction to Multivariate Statistical Analysis. second ed. (1984), Wiley, New York
20. G. Monaci, O. Divorra Escoda, P. Vandergheynst, Analysis of multimodal signals using redundant representations, in: Proceedings of IEEE ICIP, 2005.
21. Cover T.M., and Thomas J.A. Elements of Information Theory (1991), Wiley, New York
22. Patterson E.K., Gurbuz S., Tufekci Z., and Gowdy J.N. Moving-talker, speaker-independent feature study, and baseline results using the CUAVE multimodal speech corpus. J. Appl. Signal Process. 11 (2002) 1189-1201
23. R. Gribonval, E. Bacry, J. Abadia, Matching pursuit software and documentation, 〈http://www.cmap.polytechnique.fr/bacry/LastWave/packages/mp/mp.html〉.
24. G. Monaci, Multimodal web page, 〈http://lts2www.epfl.ch/monaci/multimodal.html〉.
25. P. Jost, P. Vandergheynst, P. Frossard, Tree-based pursuit: algorithm and properties, EPFL-ITS Technical Report 2005.13, Lausanne. Available: 〈http://lts2www.epfl.ch/〉, May 2005 (online).