Learning multimodal dictionaries

IEEE Transactions on Image Processing

Volumn 16, Issue 9, 2007, Pages 2272-2283

  Monaci, Gianluca ^a   Jost, Philippe ^a   Vandergheynst, Pierre ^a   Mailhé, Boris ^b   Lesage, Sylvain ^b   Gribonval, Rémi ^b  

^a EPFL   (Switzerland)

^b CAMPUS DE BEAULIEU   (France)

Author keywords

Audiovisual source localization; Dictionary learning; Multimodal data processing; Sparse representation

Indexed keywords

COMPUTER VISION; DATA STRUCTURES; EIGENVALUES AND EIGENFUNCTIONS; ITERATIVE METHODS; LEARNING ALGORITHMS; SENSORY PERCEPTION;

AUDIOVISUAL SOURCE LOCALIZATION; DICTIONARY LEARNING; MULTIMODAL DATA PROCESSING; SPARSE REPRESENTATION;

DATA PROCESSING;

ALGORITHM; ARTICLE; ARTIFICIAL INTELLIGENCE; AUTOMATED PATTERN RECOGNITION; BOOK; COMPUTER ASSISTED DIAGNOSIS; IMAGE ENHANCEMENT; IMAGE SUBTRACTION; INFORMATION RETRIEVAL; METHODOLOGY;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; DICTIONARIES AS TOPIC; IMAGE ENHANCEMENT; IMAGE INTERPRETATION, COMPUTER-ASSISTED; INFORMATION STORAGE AND RETRIEVAL; PATTERN RECOGNITION, AUTOMATED; SUBTRACTION TECHNIQUE;

EID: 34548231808     PISSN: 10577149     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIP.2007.901813     Document Type: Article

References (36)

1. F. Maes, A. Collignon, D. Vandermeulen, G. Marchal, and P. Suetens, "Multimodality image registration by maximization of mutual information," IEEE Trans. Med. Imag., vol. 16, no. 2, pp. 187-198, Feb. 1997.
2. T. Butz and J.-P. Thiran, "From error probability to information theoretic (multi-modal) signal processing," Signal Process., vol. 85, no. 5, pp. 875-902, 2005.
3. I. R. Farah, M. B. Ahmed, and M. R. Boussema, "Multispectral satellite image analysis based on the method of blind separation and fusion of sources," in Proc. Int. Geoscience and Remote Sensing Symp., 2003, vol. 6, pp. 3638-3640.
4. K. C. Partington, "A data fusion algorithm for mapping sea-ice concentrations from special sensor microwave/imager data," IEEE Trans. Geosci. Remote Sens., vol. 38, no. 4, pp. 1947-1958, Apr. 2000.
5. E. Martínez-Montes, P. A. Valdés-Sosa, F. Miwakeichi, R. I. Goldman, and M. S. Cohen, "Concurrent EEG/fMR1 analysis by multiway partial least squares," NeuroImage, vol. 22, pp. 1023-1034, 2004.
6. C. Carmona-Moreno, A. Belward, J. Malingreau, M. Garcia-Alegre, A. Hartley, M. Antonovskiy, V. Buchshiaber, and V. Pivovarov, "Characterizing inter-annual variations in global fire calendar using data from earth observing satellites," Global Change Biol., vol. 11, no. 9, pp. 1537-1555, 2005.
7. G. Potamianos, C. Neti, G. Gravier, A. Garg, and A. W. Senior, "Recent advances in the automatic recognition of audiovisual speech," Proc. IEEE, vol. 91, no. 9, pp. 1306-1326, Sep. 2003.
8. S. Lucey, T. Chen, S. Sridharan, and V. Chandran, "Integration strategies for audio-visual speech processing: Applied to text-dependent speaker recognition," IEEE Trans. Multimedia, vol. 7, no. 3, pp. 495-506, Sep. 2005.
9. E. Cosatto, J. Ostermann, H. Graf, and J. Schroeter, "Lifelike talking faces for interactive services," Proc. IEEE, vol. 91, no. 9, pp. 1406-1429, Sep. 2003.
10. J. Hershey and J. Movellan, "Audio-vision: Using audio-visual synchrony to locate sounds," Adv. Neural Inf. Process. Syst., vol. 12, pp. 813-819, 1999.
11. M. Slaney and M. Covell, "FaceSync: A linear operaior for measuring synchronization of video facial images and audio tracks," Adv. Neural Inf. Process. Syst., vol. 13, pp. 814-820, 2000.
12. P. Smaragdis and M. Casey, "Audio/visual independent components," in Proc. ICA, Apr. 2003, pp. 709-714.
13. J. W. Fisher, III and T. Darrell, "Speaker association with signal-level audiovisual fusion," IEEE Trans. Multimedia, vol. 6, no. 3, pp. 406-413, Jun. 2004.
14. E. Kidron, Y. Schechner, and M. Elad, "Cross-modal localization via sparsity," IEEE Trans. Signal Process., vol. 55, no. 4, pp. 1390-1404, Apr. 2007.
15. H. J. Nock, G. Iyengar, and C. Neti, "Speaker localisation using audio-visual synchrony: An empirical study," in Proc. Int. Conf. Image and Video Retrieval, 2003, pp. 488-199.
16. G. Monaci and P. Vandergheynst, "Audiovisual Gestalts," in Proc. IEEE CVPR Workshop, 2006, p. 200.
17. G. Monaci, O. D. Escoda, and P. Vandergheynst, "Analysis of mullimodal sequences using geometric video representations," Signal Process. vol. 86, no. 12, pp. 3534-3548, 2006.
18. J. Driver, "Enhancement of selective listening by illusory mislocation of speech sounds due to lip-reading," Nature, vol. 381, pp. 66-68, 1996.
19. M. T. Wallace, G. E. Roberson, W. D. Hairston, B. E. Stein, J. W. Vaughan, and J. A. Schirillo, "Unifying multisensory signals across time and space," Exp. Brain Res., vol. 158, pp. 252-258, 2004.
20. S. Watkins, L. Shams, S. Tanaka, J.-D. Haynes, and G. Rees, "Sound alters activity in human V1 in association with illusory visual perception," NeuroImage, vol. 31, no. 3, pp. 1247-1256, 2006.
21. A. Violentyev, S. Shimojo, and L. Shams, "Touch-induced visual illusion," Neuroreport, vol. 10, no. 16, pp. 1107-1110, 2005.
22. J.-P. Bresciani, F. Dammeier, and M. Emst, "Vision and touch are automatically integrated for the perception of sequences of events," J. Vis., vol. 6, no. 5, pp. 554-564, 2006.
23. E. K. Patterson, S. Gurbuz, Z. Tufekci, and J. N. Gowdy, "Moving-talker, speaker-independent feature study, and baseline results using the CUAVE multimodal speech corpus," EURASIP J. Appl. Signal Process., vol. 2002, no. 11, pp. 1189-1201, 2002.
24. G. Monaci, O. D. Escoda, and P. Vandergheynst, "Analysis of multimodal signals using redundant representations," in Proc. IEEE Int. Conf. Image Processing, 2005, vol. 3, pp. 46-49.
25. R. Gribonval, "Sparse decomposition of stereo signals with matching pursuit and application to blind separation of more than two sources from a stereo mixture," in Proc. IEEE Int. Conf. Image Processing 2002, vol. 3, pp. 3057-3060.
26. J. Tropp, A. Gilbert, and M. J. Strauss, "Simultaneous sparse approximation via greedy pursuit," in Proc. IEEE ICASSP, 2005, vol. 5, pp. 721-724.
27. M. Lewicki and T. Sejnowski, "Learning overcomplete representations," Neural Comput., vol. 12, no. 2, pp. 337-365, 2000.
28. S. Abdallah and M. Plumbley, "If edges are the independent components of natural images, what are the independent components of natural sounds?," in Proc. ICA, 2001, pp. 534-539.
29. P. Jost, P. Vandergheynst, S. Lesage, and R. Gribonval, "MoTIF: An efficient algorithm for learning translation invariant dictionaries," in Proc. IEEE ICASSP, 2006, vol. 5, pp. 857-860.
30. A. Bell and T. Sejnowski, "The "independent components" of natural scenes are edge filters," Vis. Res., vol. 37, no. 23, pp. 3327-3338, 1997.
31. B. A. Olshausen and D. J. Field, "Sparse coding with an overcomplete basis set: A strategy employed by V1?," Vis. Res., vol. 37, pp. 3311-3327, 1997.
32. M. Lewicki and B. A. Olshausen, "A probabilistic framework for the adaptation and comparison of image codes," J. Opt. Soc. Amer. A, vol. 16, no. 7, pp. 1587-1601, 1999.
33. K. Kreutz-Delgado, J. Murray, B. Rao, K. Engan, T. Lee, and T. Sejnowski, "Dictionary learning algorithms for sparse representation," Neural Comput., vol. 15, pp. 349-396, 2003.
34. B. A. Olshausen, "Learning sparse, overcomplete representations of time-varying natural images," in Proc. IEEE Int. Conf. Image Processing, 1, 2003, pp. 41-44.
35. D. Dong and J. Atick, "Temporal decorrelation: A theory of lagged and nonlagged responses in the lateral geniculate nucleus," Network: Comput. Neural Syst., vol. 6, pp. 159-178, 1995.
36. O. Divorra Escoda, "Toward sparse and geometry adapted video approximations" Ph.D. dissertation, Swiss Fed. Inst. Technol., Lausanne, Switzerland, Jun. 2005 [Online]. Available: http://lts2www.epfl.ch/.