Data driven articulatory synthesis with deep neural networks

Computer Speech and Language

Volumn 36, Issue , 2016, Pages 260-273

  Aryal, Sandesh ^a   Gutierrez Osuna, Ricardo ^a  

^a TEXAS A AND M UNIVERSITY   (United States)

Author keywords

Articulatory synthesis; Deep learning; Electromagnetic articulography; Gaussian mixture models

Indexed keywords

AERODYNAMICS; COMMUNICATION CHANNELS (INFORMATION THEORY); MAPPING; OBJECT RECOGNITION;

ARTICULATORY SYNTHESIS; CONVENTIONAL APPROACH; DEEP LEARNING; DEEP NEURAL NETWORKS; ELECTROMAGNETIC ARTICULOGRAPHY; GAUSSIAN MIXTURE MODEL; SUBJECTIVE EVALUATIONS; TRAJECTORY OPTIMIZATION;

TRAJECTORIES;

EID: 84949568676     PISSN: 08852308     EISSN: 10958363     Source Type: Journal    
DOI: 10.1016/j.csl.2015.02.003     Document Type: Article

References (52)

1. G. Andrew, R. Arora, J. Bilmes, and K. Livescu Deep canonical correlation analysis Proceedings of ICML 2013 1247 1255
2. R. Arora, and K. Livescu Multi-view CCA-based acoustic features for phonetic recognition across speakers and domains Proceedings of ICASSP 2013 7135 7139
3. S. Aryal, and R. Gutierrez-Osuna Articulatory inversion and synthesis: towards articulatory-based modification of speech Proceedings of ICASSP 2013 7952 7956
4. S. Aryal, and R. Gutierrez-Osuna Accent conversion through cross-speaker articulatory synthesis Proceedings of ICASSP 2014 7744 7748
5. Y. Bao, H. Jiang, C. Liu, Y. Hu, and L. Dai Investigation on dimensionality reduction of concatenated features with deep neural network for LVCSR systems Proceedings of ICSP 2012 562 566
6. P. Birkholz, D. Jackèl, and B. Kroger Construction and control of a three-dimensional vocal tract model Proceedings of ICASSP 2006 873 876
7. C.P. Browman, L. Goldstein, J.A.S. Kelso, P. Rubin, and E. Saltzman Articulatory synthesis from underlying dynamics J. Acoust. Soc. Am. 75 1984 S22 S23
8. K. Cho Boltzmann machines and denoising autoencoders for image denoising 2013 arXiv:1301.3468
9. K.H. Cho Matlab code for restricted/deep Boltzmann machines and autoencoders 2013 https://github.com/kyunghyuncho/deepmat
10. K.H. Cho, T. Raiko, and A. Ilin Gaussian-Bernoulli deep Boltzmann machine Proceedings of IJCNN 2013 1 7
11. B. Denby, T. Schultz, K. Honda, T. Hueber, J.M. Gilbert, and J.S. Brumberg Silent speech interfaces Speech Commun. 52 2010 270 287
12. D. Erhan, Y. Bengio, A. Courville, P.-A. Manzagol, P. Vincent, and S. Bengio Why does unsupervised pre-training help deep learning? J. Mach. Learn. Res. 11 2010 625 660
13. D. Felps, S. Aryal, and R. Gutierrez-Osuna Normalization of articulatory data through Procrustes transformations and analysis-by-synthesis Proceedings of ICASSP 2014 3051 3055
14. D. Felps, C. Geng, and R. Gutierrez-Osuna Foreign accent conversion through concatenative synthesis in the articulatory domain IEEE Trans. Audio Speech Lang. Process. 20 2012 2301 2312
15. C. Geng, and C. Mooshammer How to stretch and shrink vowel systems: results from a vowel normalization procedure J. Acoust. Soc. Am. 125 2009 3278
16. P.K. Ghosh, and S.S. Narayanan A subject-independent acoustic-to-articulatory inversion Proceedings of ICASSP 2011 4624 4627
17. H. Hermansky, and D.J. Broad The effective second formant F2′ and the vocal tract front-cavity Proceedings of ICASSP 1989 480 483
18. G. Hinton, L. Deng, D. Yu, G.E. Dahl, A.R. Mohamed, N. Jaitly, A. Senior, V. Vanhoucke, P. Nguyen, and T.N. Sainath Deep neural networks for acoustic modeling in speech recognition: the shared views of four research groups IEEE Signal Process. Mag. 29 2012 82 97
19. G.E. Hinton A practical guide to training restricted Boltzmann machines Neural Networks: Tricks of the Trade 2012 Springer 599 619
20. S. Hiroya, and M. Honda Estimation of articulatory movements from speech acoustics using an HMM-based speech production model IEEE Trans. Speech Audio Process. 12 2004 175 185
21. A. Hu, H. Li, F. Zhang, and W. Zhang Deep Boltzmann machines based vehicle recognition Proceedings of CCDC 2014 3033 3038
22. ITU-T Recommendation G.114: One-way transmission time 2003
23. A. Ji, J. Berry, and M.T. Johnson The electromagnetic articulography mandarin accented english (EMA-MAE) corpus of acoustic and 3D articulatory kinematic data Proceedings of ICASSP 2014 7769 7773
24. T. Kaburagi, and M. Honda Determination of the vocal tract spectrum from the articulatory movements based on the search of an articulatory-acoustic database Proceedings of ICSLP 1998 433 436
25. H. Kawahara Speech representation and transformation using adaptive interpolation of weighted spectrum: vocoder revisited Proceedings of ICASSP 1997 1303 1306
26. C.T. Kello, and D.C. Plaut A neural network model of the articulatory-acoustic forward mapping trained on recordings of articulatory parameters J. Acoust. Soc. Am. 116 2004 2354 2364
27. S. Maeda Compensatory articulation during speech: evidence from the analysis and synthesis of vocal tract shapes using an articulatory model W. Hardcastle, A. Marchal, Speech Production and Speech Modelling 1990 Kluwer Academic Publisher Amsterdam 131 149
28. P. Mermelstein Articulatory model for the study of speech production J. Acoust. Soc. Am. 53 1973 1070 1082
29. T. Muramatsu, Y. Ohtani, T. Toda, H. Saruwatari, and K. Shikano Low-delay voice conversion based on maximum likelihood estimation of spectral parameter trajectory Proceedings of INTERSPEECH 2008 1076 1079
30. I.T. Nabney NETLAB: Algorithms for Pattern Recognition 2002 Springer
31. K. Nakamura, T. Toda, Y. Nankaku, and K. Tokuda On the use of phonetic information for mapping from articulatory movements to vocal tract spectrum Proceedings of ICASSP 2006 I-I
32. T. Nakashika, R. Takashima, T. Takiguchi, and Y. Ariki Voice conversion in high-order eigen space using deep belief nets Proceedings of INTERSPEECH 2013 369 372
33. S. Narayanan, E. Bresch, P.K. Ghosh, L. Goldstein, A. Katsamanis, Y. Kim, A.C. Lammert, M.I. Proctor, V. Ramanarayanan, and Y. Zhu A multimodal real-time MRI articulatory corpus for speech research Proceedings of INTERSPEECH 2011 837 840
34. I.Y. Özbek, M. Hasegawa-Johnson, and M. Demirekler Formant trajectories for acoustic-to-articulatory inversion Proceedings of INTERSPEECH 2009 2807 2810
35. R. Prabhavalkar, E. Fosler-Lussier, and K. Livescu A factored conditional random field model for articulatory feature forced transcription Proceedings of ASRU 2011 77 82
36. C. Qin, and M.A. Carreira-Perpinán An empirical investigation of the nonuniqueness in the acoustic-to-articulatory mapping Proceedings of INTERSPEECH 2007 2300 2303
37. K. Richmond, S. King, and P. Taylor Modelling the uncertainty in recovering articulation from acoustics Comput. Speech Lang. 17 2003 153 172
38. F. Rudzicz Adaptive kernel canonical correlation analysis for estimation of task dynamics from acoustics Proceedings of ICASSP 2010 4198 4201
39. D. Rumelhart, G. Hinton, and R. Williams Learning representations by back-propagating errors Nature 323 1986 533 536
40. R. Salakhutdinov, and G.E. Hinton Deep Boltzmann machines Proceedings of AISTATS Florida, USA 2009 448 455
41. T. Toda, A.W. Black, and K. Tokuda Mapping from articulatory movements to vocal tract spectrum with Gaussian mixture model for articulatory speech synthesis Proceedings of ISCA 2004 SSW5
42. T. Toda, A.W. Black, and K. Tokuda Voice conversion based on maximum-likelihood estimation of spectral parameter trajectory IEEE Trans. Audio Speech Lang. Process. 15 2007 2222 2235
43. T. Toda, A.W. Black, and K. Tokuda Statistical mapping between articulatory movements and acoustic spectrum using a Gaussian mixture model Speech Commun. 50 2008 215 227
44. T. Toda, T. Muramatsu, and H. Banno Implementation of computationally efficient real-time voice conversion Proceedings of INTERSPEECH 2012 94 97
45. A.R. Toth, and A.W. Black Cross-speaker articulatory position data for phonetic feature prediction Proceedings of INTERSPEECH 2005 2973 2976
46. B. Uria, I. Murray, S. Renals, and K. Richmond Deep architectures for articulatory inversion Proceedings of INTERSPEECH 2012 867 870
47. J.R. Westbury X-ray Microbeam Speech Production Database User's Handbook Version 1.0 1994 Waisman Center on Mental Retardation & Human Development, University of Wisconsin Madison, WI
48. A.A. Wrench A multichannel articulatory database and its application for automatic speech recognition Prodeedings of 5th Seminar of Speech Production 2000 305 308
49. Y. Xu, J. Du, L.-R. Dai, and C.-H. Lee An experimental study on speech enhancement based on deep neural networks Signal Process. Lett. IEEE 21 2014 65 68
50. Z. You, X. Wang, and B. Xu Investigation of deep Boltzmann machines for phone recognition Proceedings of ICASSP 2013 7600 7603
51. H. Zen, A. Senior, and M. Schuster Statistical parametric speech synthesis using deep neural networks Proceedings of ICASSP 2013 7962 7966
52. Y. Zhang, R. Salakhutdinov, H.-A. Chang, and J. Glass Resource configurable spoken query detection using deep Boltzmann machines Proceedings of ICASSP 2012 5161 5164