Exemplar-based sparse representation with residual compensation for voice conversion

IEEE Transactions on Audio, Speech and Language Processing

Volumn 22, Issue 10, 2014, Pages 1506-1521

  Wu, Zhizheng ^a,b   Virtanen, Tuomas ^c   Chng, Eng Siong ^a,b   Li, Haizhou ^a,d  

^a NANYANG TECHNOLOGICAL UNIVERSITY   (Singapore)

^b NANYANG TECHNOLOGICAL UNIVERSITY   (Singapore)

^c TAMPERE UNIVERSITY OF TECHNOLOGY   (Finland)

^d INSTITUTE FOR INFOCOMM RESEARCH   (Singapore)

Author keywords

Exemplar; Nonnegative matrix factorization; Residual compensation; Sparse representation; Voice conversion

Indexed keywords

MATRIX ALGEBRA; MAXIMUM LIKELIHOOD; SPEECH PROCESSING;

DIMENSIONALITY REDUCTION; EXEMPLAR; NONNEGATIVE MATRIX FACTORIZATION; PARTIAL LEAST-SQUARES REGRESSION; SPARSE REPRESENTATION; SUBJECTIVE LISTENING TEST; VOICE CONVERSION; WEIGHTED LINEAR COMBINATIONS;

LEAST SQUARES APPROXIMATIONS;

EID: 84911369131     PISSN: 15587916     EISSN: None     Source Type: Journal    
DOI: 10.1109/TASLP.2014.2333242     Document Type: Article

References (54)

1. T. Kinnunen and H. Li, "An overview of text-independent speaker recognition: From features to supervectors," Speech Commun., vol. 52, no. 1, pp. 12-40, 2010.
2. A. Kain and M. W. Macon, "Spectral voice conversion for text-to-speech synthesis," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP), 1998, pp. 285-288.
3. K. Park and H. Kim, "Narrowband to wideband conversion of speech using GMM based transformation," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP), 2000, pp. 1843-1846.
4. A. Mouchtaris, J. Van der Spiegel, P. Mueller, and P. Tsakalides, "A spectral conversion approach to single-channel speech enhancement," IEEE Trans. Audio, Speech, Lang. Process., vol. 15, no. 4, pp. 1180-1193, May 2007.
5. M. Afify, X. Cui, and Y. Gao, "Stereo-based stochastic mapping for robust speech recognition," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP), 2007, pp. 377-380.
6. X. Cui, M. Afify, and B. Zhou, "Stereo-based stochastic mapping with context using probabilistic PCA for noise robust automatic speech recognition," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP), 2012, pp. 4705-4708.
7. T. Toda, A. Black, and K. Tokuda, "Statistical mapping between articulatory movements and acoustic spectrum using a gaussian mixture model," Speech Commun., vol. 50, no. 3, pp. 215-227, 2008.
8. T. Toda, K. Nakamura, H. Sekimoto, and K. Shikano, "Voice conversion for various types of body transmitted speech," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP), 2009, pp. 3601-3604.
9. M. Abe, S. Nakamura, K. Shikano, and H. Kuwabara, "Voice conversion through vector quantization," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP), 1988, pp. 655-658.
10. K. Shikano, S. Nakamura, and M. Abe, "Speaker adaptation and voice conversion by codebook mapping," in Proc. IEEE Int. Symp. Circuits Syst., 1991, pp. 594-597.
11. Y. Stylianou, O. Cappé, and E. Moulines, "Continuous probabilistic transform for voice conversion," IEEE Trans. Speech Audio Process., vol. 6, no. 2, pp. 131-142, Mar. 1998.
12. T. Toda, A. W. Black, and K. Tokuda, "Voice conversion based on maximum-likelihood estimation of spectral parameter trajectory," IEEE Trans. Audio, Speech, Lang. Process., vol. 15, no. 8, pp. 2222-2235, Nov. 2007.
13. H. Zen, Y. Nankaku, and K. Tokuda, "Continuous stochastic feature mapping based on trajectory HMMs," IEEE Trans. Audio, Speech, Lang. Process., vol. 19, no. 2, pp. 417-430, Feb. 2011.
14. H. Ye and S. Young, "Perceptually weighted linear transformations for voice conversion," in Proc. Interspeech, 2003.
15. H. Ye and S. Young, "Quality-enhanced voice morphing using maximum likelihood transformations," IEEE Trans. Audio, Speech, Lang. Process., vol. 14, no. 4, pp. 1301-1312, Jul. 2006.
16. E. Helander, T. Virtanen, J. Nurminen, and M. Gabbouj, "Voice conversion using partial least squares regression," IEEE Trans. Audio, Speech, Lang. Process., vol. 18, no. 5, pp. 912-921, Jul. 2010.
17. V. Popa, H. Silen, J. Nurminen, and M. Gabbouj, "Local linear transformation for voice conversion," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP), 2012, pp. 4517-4520.
18. M. Narendranath, H. Murthy, S. Rajendran, and B. Yegnanarayana, "Transformation of formants for voice conversion using artificial neural networks," Speech Commun., vol. 16, no. 2, pp. 207-216, 1995.
19. S. Desai, E. V. Raghavendra, B. Yegnanarayana, A. W. Black, and K. Prahallad, "Voice conversion using artificial neural networks," in Proc. IEEE Int. Conf. Acoustics, Speech, Signal Process. (ICASSP), 2009, pp. 3893-3896.
20. P. Song, Y. Bao, L. Zhao, and C. Zou, "Voice conversion using support vector regression," Electron. Lett., vol. 47, no. 18, pp. 1045-1046, 2011.
21. E. Helander, H. Silén, T. Virtanen, and M. Gabbouj, "Voice conversion using dynamic kernel partial least squares regression," IEEE Trans. Audio, Speech, Lang. Process., vol. 20, no. 3, pp. 806-817, Mar. 2012.
22. L.-H. Chen, Z.-H. Ling, Y. Song, and L.-R. Dai, "Joint spectral distribution modeling using restricted Boltzmann machines for voice conversion," in Proc. Interspeech, 2013.
23. D. Sundermann and H. Ney, "VTLN-based voice conversion," in Proc. 3rd IEEE Int. Symp. Signal Process. Inf. Technol., 2003, pp. 556-559.
24. D. Sundermann, H. Ney, and H. Hoge, "VTLN-based cross-language voice conversion," in IEEE Workshop Autom. Speech Recogn. Understand., 2003, pp. 676-681.
25. D. Erro, A. Moreno, and A. Bonafonte, "Voice conversion based on weighted frequency warping," IEEE Trans. Audio, Speech, Lang. Process., vol. 18, no. 5, pp. 922-931, Jul. 2010.
26. E. Godoy, O. Rosec, and T. Chonavel, "Voice conversion using dynamic frequency warping with amplitude scaling, for parallel or nonparallel corpora," IEEE Trans. Audio, Speech, Lang. Process., vol. 20, no. 4, pp. 1313-1323, May 2012.
27. D. Erro, E. Navas, and I. Hernaez, "Parametric voice conversion based on bilinear frequency warping plus amplitude scaling," IEEE Trans. Audio, Speech, Lang. Process., vol. 21, no. 3, pp. 556-566, Mar. 2013.
28. Z. Wu, T. Virtanen, T. Kinnunen, E. S. Chng, and H. Li, "Exemplar-based voice conversion using non-negative spectrogram deconvolution," in Proc. 8th ISCA Speech Synth. Workshop (SSW8), 2013.
29. D. Seung and L. Lee, "Algorithms for non-negative matrix factorization," Adv. Neural Inf. Process. Syst., vol. 13, pp. 556-562, 2001.
30. J. Gemmeke, T. Virtanen, and A. Hurmalainen, "Exemplar-based sparse representations for noise robust automatic speech recognition," IEEE Trans. Audio, Speech, Lang. Process., vol. 19, no. 7, pp. 2067-2080, Sep. 2011.
31. A. Hurmalainen, J. Gemmeke, and T. Virtanen, "Non-negative matrix deconvolution in noise robust speech recognition," in Proc. ICASSP, 2011, pp. 4588-4591.
32. R. Takashima, T. Takiguchi, and Y. Ariki, "Exemplar-based voice conversion in noisy environment," in Proc. IEEE Spoken Lang. Technol. Workshop (SLT), 2012, pp. 313-317.
33. B. King, C. Févotte, and P. Smaragdis, "Optimal cost function and magnitude power for NMF-based speech separation and music interpolation," in Proc. IEEE Int. Workshop Mach. Learn. Signal Process. (MLSP), 2012, pp. 1-6.
34. D. Erro, A. Moreno, and A. Bonafonte, "INCA algorithm for training voice conversion systems from nonparallel corpora," IEEE Trans. Audio, Speech, Lang. Process., vol. 18, no. 5, pp. 944-953, 2010.
35. Y. Chen, M. Chu, E. Chang, J. Liu, and R. Liu, "Voice conversion with smoothed GMM and MAP adaptation," in Proc. Eur. Conf. Speech Commun. Technol. (Eurospeech), 2003.
36. T. Fukada, K. Tokuda, T. Kobayashi, and S. Imai, "An adaptive algorithm for mel-cepstral analysis of speech," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP), 1992, pp. 137-140.
37. F. Soong and B.-H. Juang, "Line spectrum pair (LSP) and speech data compression," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP), 1984, pp. 137-140.
38. J. Yamagishi, T. Nose, H. Zen, Z.-H. Ling, T. Toda, K. Tokuda, S. King, and S. Renals, "Robust speaker-adaptive HMM-based text-to-speech synthesis," IEEE Trans. Audio, Speech, Lang. Process., vol. 17, no. 6, pp. 1208-1230, Aug. 2009.
39. J. Dines, J. Yamagishi, and S. King, "Measuring the gap between HMM-based ASR and TTS," IEEE J. Sel. Topics Signal Process., vol. 4, no. 6, pp. 1046-1058, Aug. 2010.
40. T. Virtanen, "Unsupervised learning methods for source separation in monaural music signals," in Signal Processing Methods for Music Transcription, A. Klapuri and M. Davy, Eds. New York, NY, USA: Springer, 2006, pp. 267-296.
41. J. Nam, G. J. Mysore, J. Ganseman, K. Lee, and J. S. Abel, "A super-resolution spectrogram using coupled PLCA," in Proc. Interspeech, 2010.
42. D. Bansal, B. Raj, and P. Smaragdis, "Bandwidth expansion of narrow-band speech using non-negative matrix factorization," in Proc. Interspeech, 2005.
43. H. Kawahara, I. Masuda-Katsuse, and A. de Cheveigné, "Restructuring speech representations using a pitch-adaptive time-frequency smoothing and an instantaneous-frequency-based f0 extraction: Possible role of a repetitive structure in sounds," Speech Commun., vol. 27, no. 3, pp. 187-207, 1999.
44. K. Tokuda, T. Kobayashi, T. Masuko, and S. Imai, "Mel-generalized cepstral analysis-a unified approach to speech spectral estimation," in Proc. Int. Conf. Spoken Lang. Process. (ICSLP), 1994.
45. A. B. Kain, "High resolution voice transformation," Ph.D. dissertation, OGI School of Sci. & Eng., Oregon Health and Science Univ., Beaverton, OR, USA, 2001.
46. T. Toda, T. Muramatsu, and H. Banno, "Implementation of computationally efficient real-time voice conversion," in Proc. Interspeech, 2012.
47. M. K. Wolters, K. B. Isaac, and S. Renals, "Evaluating speech synthesis intelligibility using amazon mechanical turk," in Proc. 7th ISCA Speech Synth. Workshop (SSW7).
48. F. Ribeiro, D. Florêncio, C. Zhang, and M. Seltzer, "CROWDMOS: An approach for crowdsourcing mean opinion score studies," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP), 2011, pp. 2416-2419.
49. S. Buchholz and J. Latorre, "Crowdsourcing preference tests, and how to detect cheating," in Proc. Interspeech, 2011.
50. T. Dutoit, A. Holzapfel, M. Jottrand, A. Moinet, J. Perez, and Y. Stylianou, "Towards a voice conversion system based on frame selection," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP), 2007, pp. 513-516.
51. D. Sundermann, H. Hoge, A. Bonafonte, H. Ney, A. Black, and S. Narayanan, "Text-independent voice conversion based on unit selection," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP), 2006, pp. 81-84.
52. Y. Stylianou and A. K. Syrdal, "Perceptual and objective detection of discontinuities in concatenative speech synthesis," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP), 2001, pp. 837-840.
53. D. D. Lee and H. S. Seung, "Learning the parts of objects by non-negative matrix factorization," Nature, vol. 401, no. 6755, pp. 788-791, 1999.
54. J.-Y. Pan and J.-S. Zhang, "Large margin based nonnegative matrix factorization and partial least squares regression for face recognition," Pattern Recogn. Lett., vol. 32, no. 14, pp. 1822-1835, 2011.