Multimodal exemplar-based voice conversion using lip features in noisy environments

Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH

Volumn , Issue , 2014, Pages 1159-1163

  Masaka, Kenta ^a   Aihara, Ryo ^a   Takiguchi, Tetsuya ^a   Ariki, Yasuo ^a  

^a KOBE UNIVERSITY   (Japan)

Author keywords

Image features; Multimodal; Noise robustness; Non negative matrix factorization; Voice conversion

Indexed keywords

ACOUSTIC NOISE; COST FUNCTIONS; FACE RECOGNITION; FACTORIZATION; SIGNAL PROCESSING; SPEECH COMMUNICATION;

IMAGE FEATURES; MULTI-MODAL; NOISE ROBUSTNESS; NONNEGATIVE MATRIX FACTORIZATION; VOICE CONVERSION;

SPEECH PROCESSING;

EID: 84910091291     PISSN: 2308457X     EISSN: 19909772     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (28)

1. D. D. Lee and H. S. Seung, "Algorithms for non-negative matrix factorization, " Neural Information Processing System, pp. 556- 562, 2001.
2. T. Virtanen, "Monaural sound source separation by non-negative matrix factorization with temporal continuity and sparseness cri- Teria, " IEEE Trans. Audio, Speech, Lang. Process., vol. 15, no. 3, pp. 1066-1074, 2007.
3. M. N. Schmidt and R. K. Olsson, "Single-channel speech sepa- ration using sparse non-negative matrix factorization, " in Inter- speech, 2006.
4. J. F. Gemmeke, T. Viratnen, and A. Hurmalainen, "Exemplar- based sparse representations for noise robust automatic speech recognition, " IEEE Trans. Audio, Speech and Language Processing, vol. 19, no. 7, pp. 2067-2080, 2011.
5. R. Takashima, T. Takiguchi, and Y. Ariki, "Exemplar-based voice conversion in noisy environment, " in SLT, pp. 313-317, 2012.
6. Y. Stylianou, O. Cappe, and E. Moilines, "Continuous probabilis- Tic transform for voice conversion, " IEEE Trans. Speech and Audio Processing, vol. 6, no. 2, pp. 131-142, 1998.
7. G. Potamianos and H. P. Graf, "Discriminative training of HMM stream exponents for audio-visual speech recognition, " in ICASSP, pp. 3733-3736, 1998.
8. A. Verma, T. Faruquie, C. Neti, S. Basu, and A. Senior, "Late inte- gration in audio-visual continuous speech recognition, " in ASRU, 1999.
9. M. J. Tomlinson, M. J. Russell, and N. M. Brooke, "Integrat- ing audio and visual information to provide highly robust speech recognition, " in ICASSP, pp. 821-824, 1996.
10. Y. Komai, N. Yang, T. Takiguchi, and Y. Ariki, "Robust aam- based audio-visual speech recognition against face direction changes, " ACM Multimedia, pp. 1161-1164, 2012.
11. T. F. Cootes, G. J. Edwards, and C. J. Taylor, "Active appearance models, " IEEE Transactions on Pattern Analysis and Machine Intelligence, pp. 681-685, 2001.
12. C. Veaux and X. Robet, "Intonation conversion from neutral to expressive speech, " in Interspeech, pp. 2765-2768, 2011.
13. R. Aihara, R. Takashima, T. Takiguchi, and Y. Ariki, "GMM- based emotional voice conversion using spectrum and prosody features, " American Journal of Signal Processing, vol. 2, no. 5, 2012.
14. K. Nakamura, T. Toda, H. Saruwatari, and K. Shikano, "Speaking- Aid systems using GMM-based voice conversion for electrolaryn- geal speech, " Speech Communication, vol. 54, no. 1, pp. 134-146, 2012.
15. R. Aihara, R. Takashima, T. Takiguchi, and Y. Ariki, "Individuality-preserving voice conversion for articulation disor- ders based on Non-negative Matrix Factorization, " in ICASSP, pp. 8037-8040, 2013.
16. K. Nakamura, T. Toda, H. Saruwatari, and K. Shikano, "Speak- ing aid system for total laryngectomees using voice conversion of body transmitted artificial speech, " in Interspeech, pp. 148-151, 2006.
17. A. Kain and M.W. Macon, "Spectral voice conversion for text-to- speech synthesis, " in ICASSP, vol. 1, pp. 285-288, 1998.
18. M. Abe, S. Nakamura, K. Shikano, and H. Kuwabara, "Esophageal speech enhancement based on statistical voice con- version with Gaussian mixture models, " in ICASSP, pp. 655-658, 1988.
19. H. Valbret, E. Moulines, and J. P. Tubach, "Voice transformation using PSOLA technique, " Speech Communication, vol. 11, no. 2- 3, pp. 175-187, 1992.
20. T. Toda, A. Black, and K. Tokuda, "Voice conversion based on maximum likelihood estimation of spectral parameter trajectory, " IEEE Trans. Audio, Speech, Lang. Process., vol. 15, no. 8, pp. 2222-2235, 2007.
21. E. Helander, T. Virtanen, J. Nurminen, and M. Gabbouj, "Voice conversion using partial least squares regression, " IEEE Trans. Audio, Speech, Lang. Process., vol. 18, Issue:5, pp. 912-921, 2010.
22. C. H. Lee and C. H. Wu, "Map-based adaptation for speech con- version using adaptation data selection and non-parallel training, " in Interspeech, pp. 2254-2257, 2006.
23. T. Toda, Y. Ohtani, and K. Shikano, "Eigenvoice conversion based on Gaussian mixture model, " in Interspeech, pp. 2446-2449, 2006.
24. D. Saito, K. Yamamoto, N. Minematsu, and K. Hirose, "One-to- many voice conversion based on tensor representation of speaker space, " in Interspeech, pp. 653-656, 2011.
25. K. Masaka, R. Aihara, T. Takiguchi, and Y. Ariki, "Mutimodal voice conversion using non-negative matrix factorization in noisy environments, " in ICASSP 2014, 2014.
26. A. Kurematsu, K. Takeda, Y. Sagisaka, S. Katagiri, H. Kuwabara, and K. Shikano, "ATR Japanese speech database as a tool of speech recognition and synthesis, " Speech Communication, vol. 9, pp. 357-363, 1990.
27. S. Sako, K. Tokuda, T. Masuko, T. Kobayashi, and T. Kitamura, "HMM-based text-to-audio-visual speech synthesis - image-based approach, " ICSLP, vol.III, pp.25-28, 2000.
28. N. Kitaoka, T. Yamada, S. Tsuge, C. Miyajima, K. Yamamoto, T. Nishiura, M. Nakayama, Y. Denda, M. Fujimoto, T. Takiguchi, S. Tamura, S. Matsuda, T. Ogawa, S. Kuroiwa, K. Takeda, and S. Nakamura, "CENSREC-1-C: An evaluation framework for voice activity detection under noisy environments, " Acoustical Science and Technology, Vol. 30 (2009), No. 5, pp. 363-371, 2009.