Toward invariant functional representations of variable surface fundamental frequency contours: Synthesizing speech melody via model-based stochastic learning

Speech Communication

Volumn 57, Issue , 2014, Pages 181-208

  Xu, Yi ^a   Prom On, Santitham ^a,b  

^a UNIVERSITY COLLEGE LONDON   (United Kingdom)

^b KING MONGKUT'S UNIVERSITY OF TECHNOLOGY THONBURI   (Thailand)

Author keywords

Analysis by synthesis; Parallel encoding; Prosody modeling; Simulated annealing; Target approximation

Indexed keywords

ANALYSIS BY SYNTHESIS; BASIC AND APPLIED RESEARCH; CONTEXTUAL VARIABILITY; FUNCTIONAL REPRESENTATION; FUNDAMENTAL FREQUENCY CONTOUR; PARALLEL ENCODING; PROSODY MODEL; STOCHASTIC GLOBAL OPTIMIZATION;

NATURAL FREQUENCIES; SIMULATED ANNEALING; STOCHASTIC SYSTEMS;

SPEECH PROCESSING;

EID: 84888256123     PISSN: 01676393     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.specom.2013.09.013     Document Type: Article

References (101)

1. A.S. Abramson The vowels and tones of Standard Thai: acoustical measurements and experiments Int. J. Am. Linguist. 28 2 1962 3
2. Anderson, M.D.; Pierrehumbert, J.B.; Liberman, M.Y.; 1984. Synthesis by rule of English intonation patterns. In: Proc. ICASSP 1984, San Diego, pp. 77-80.
3. A. Arvaniti, and D.R. Ladd Greek wh-questions and the phonology of intonation Phonology 26 1 2009 43 74
4. G. Bailly, and B. H (Pubitemid 40952521)
5. M.E. Beckman, and J.B. Pierrehumbert Intonational structure in Japanese and English Phonology Yearbook 3 1986 255 309
6. Black, A.W.; Hunt, A.J.; 1996. Generating F0 contours from ToBI labels using linear regression. In: Proc. ICSLP 96, Philadelphia, pp. 1385-1388.
7. Boersma, P.; Weenink, D.; 2012. Praat: doing phonetics by computer (version 5.3.35). Available from http://www.praat.org/ (last visited 12 December 2012).
8. M. Breen, L.C. Dilley, J. Kraemer, and E. Gibson Inter-transcriber reliability for two systems of prosodic annotation: ToBI (Tones and Break Indices) and RaP (Rhythm and Pitch) Corpus Linguist. Ling. 8 2 2012 277 312
9. Y.R. Chao A Grammar of Spoken Chinese 1968 University of California Press Berkeley, CA
10. Chen, G.-P.; Bailly, G.; Liu, Q.-F.; Wang, R.-H.; 2004. A superposed prosodic model for Chinese text-to-speech synthesis. In: Proceedings of International Conference of Chinese Spoken Language Processing, Hong Kong, pp. 177-180. (Pubitemid 40914877)
11. M.Y. Chen Tone Sandhi: Patterns Across Chinese Dialects 2000 Cambridge University Press Cambridge, UK
12. Y. Chen, and Y. Xu Production of weak elements in speech - Evidence from f0 patterns of neutral tone in standard Chinese Phonetica 63 1 2006 47 75 (Pubitemid 43333526)
13. W.E. Cooper, S.J. Eady, and P.R. Mueller Acoustical aspects of contrastive stress in question-answer contexts J. Acoust. Soc. Am. 77 6 1985 2142 2156 (Pubitemid 15027154)
14. S. Duanmu The phonology of Standard Chinese 2000 Oxford University Press Oxford
15. S.J. Eady, and W.E. Cooper Speech intonation and focus location in matched statements and questions J. Acoust. Soc. Am. 80 2 1986 402 416
16. B. Efron Bootstrap methods: another look at the jackknife Ann. Statist. 7 1 1979 1 26
17. Fujisaki, H.; Hirose, K.; Halle, P.; Lei, H.; 1990. Analysis and modeling of tonal features in polysyllabic words and sentences of the standard Chinese. In: Proceedings of ICSLP 1990, Kobe, pp. 841-844.
18. H. Fujisaki, C. Wang, S. Ohno, and W. Gu Analysis and synthesis of fundamental frequency contours of Standard Chinese using the command-response model Speech Commun. 47 1-2 2005 59 70 (Pubitemid 41231581)
19. J. Gandour, S. Potisuk, and S. Dechonkit Tonal coarticulations in Thai J. Phonet. 22 4 1994 477 492
20. E. Grabe, G. Kochanski, and J. Coleman Connecting intonation labels to mathematical descriptions of fundamental frequency Lang. Speech 50 3 2007 281 310
21. W. Gu, K. Hirose, and H. Fujisaki Modeling the effects of emphasis and question on fundamental frequency contours of Cantonese utterances IEEE Trans. Audio Speech 14 4 2006 1155 1170 (Pubitemid 46547613)
22. W. Gu, K. Hirose, and H. Fujisaki Analysis of tones in Cantonese speech based on the command-response model Phonetica 64 1 2007 29 62
23. K. Hadding-Koch, and M. Studdert-Kennedy An experimental study of some intonation contours Phonetica 11 1964 175 185
24. D.J. Hermes Measuring the perceptual similarity of pitch contours J. Speech Lang. Hear. Res. 41 1 1998 73 82 (Pubitemid 28081252)
25. D.J. Hirst Form and function in the representation of speech prosody Speech Commun. 46 3-4 2005 334 347 (Pubitemid 40952520)
26. D.J. Hirst The analysis by synthesis of speech melody: from data to models J. Speech Sci. 1 1 2011 55 83
27. A.T. Ho Intonation variation in a Mandarin sentence for three expressions: interrogative, exclamatory and declarative Phonetica 34 1977 446 457 (Pubitemid 8325988)
28. J.M. Howie On the domain of tone in Mandarin Phonetica 30 1974 129 148
29. M. Jilka, G. Möhler, and G. Dogil Rules for the generation of ToBI-based American English intonation Speech Commun. 28 2 1999 83 108
30. Jokisch, O.; Mixdorff, H.; Kruschke, H.; Kordon, U.; 2000. Learning the parameters of quantitative prosody models. In: Proceedings of ICSLP 2000, Beijing, pp. 645-648.
31. S. Kirkpatrick, C.D. Gelatt, and M.P. Vecchi Optimization by simulated annealing Science 220 4598 1983 671 680
32. D.H. Klatt Review of text-to-speech conversion for English J. Acoust. Soc. Am. 82 3 1987 737 793
33. G. Kochanski, and C. Shih Prosody modeling with soft templates Speech Commun. 39 3-4 2003 311 352 (Pubitemid 35432918)
34. S. Konishi, and G. Kitagawa Generalised information criteria in model selection Biometrika 83 4 1996 875 890
35. Y.-C. Kuo, Y. Xu, and M. Yip The phonetics and phonology of apparent cases of iterative tonal change in Standard Chinese C. Gussenhoven, T. Riad, Tones and Tunes Experimental Studies in Word and Sentence Prosody vol. 2 2007 Mouton de Gruyter Berlin 211 237
36. D.R. Ladd Intonational Phonology 2008 Cambridge University Press Cambridge
37. P. Ladefoged Three Areas of Experimental Phonetics 1967 Oxford University Press
38. Lee, Y.-C.; Xu, Y.; 2010. Phonetic realization of contrastive focus in Korean. In: Proceedings of Speech Prosody 2010, Chicago, pp. 100033:1-4.
39. F. Liu, and Y. Xu Parallel encoding of focus and interrogative meaning in Mandarin intonation Phonetica 62 2-4 2005 70 87 (Pubitemid 43030005)
40. F. Liu, Y. Xu, S. Prom-on, and A.C.L. Yu Morpheme-like prosodic functions: evidence from acoustic analysis and computational modeling J. Speech Sci. 3 2013 85 140
41. Mixdorff, H.; Fujisaki, H.; Chen, G.P.; Hu, Y.; 2003. Towards the automatic extraction of Fujisaki model parameters for Mandarin. In: Proceedings of Eurospeech 2003, Geneva, pp. 873-876.
42. S. Myers Surface underspecification of tone in Chichewa Phonology 15 3 1998 367 392
43. J. Ni, H. Kawai, and K. Hirose Constrained tone transformation technique for separation and combination of Mandarin tone and intonation J. Acoust. Soc. Am. 119 3 2006 1764 1782 (Pubitemid 43326042)
44. Ni, J.; Kawai, H.; 2004. Pitch targets anchor Chinese tone and intonation patterns. In: Proceedings of Speech Prosody 2004, Nara, pp. 95-98.
45. J. Ni, and K. Hirose Quantitative and structural modeling of voice fundamental frequency contours of speech in Mandarin Speech Commun. 48 8 2006 989 1008 (Pubitemid 43947270)
46. D. O'Shaughnessy, and J. Allen Linguistic modality effects on fundamental frequency in speech J. Acoust. Soc. Am. 74 4 1983 1155 1171 (Pubitemid 13020936)
47. M.D. Pell Influence of emotion and focus on prosody in matched statements and questions J. Acoust. Soc. Am. 109 4 2001 1668 1680 (Pubitemid 32275206)
48. S.-H. Peng Lexical versus 'phonological' representations of Mandarin Sandhi tones M.B. Broe, J.B. Pierrehumbert, Papers in Laboratory Phonology V: Acquisition and the Lexicon 2000 Cambridge University Press Cambridge 152 167
49. J.S. Perkell, and D.H. Klatt Invariance and Variability of Speech Processes 1986 Lawrence Erlbaum Associates Hillsdale, NJ
50. P. Perrier, D.J. Ostry, and R. Laboissière The equilibrium-point hypothesis and its application to speech motor control J. Speech Hear. Res. 39 2 1996 365 378
51. G.E. Peterson, and H.L. Barney Control methods used in a study of the vowels J. Acoust. Soc. Am. 24 2 1952 175 184
52. Pierrehumbert, J.; 1980. The phonology and phonetics of English intonation. Ph.D. thesis, MIT, Cambridge, MA.
53. J. Pierrehumbert Synthesizing intonation J. Acoust. Soc. Am. 70 4 1981 985 995 (Pubitemid 12241597)
54. S. Potisuk, J. Gandour, and M.P. Harper Contextual variations in trisyllabic sequences of Thai tones Phonetica 42 1 1997 22 42 (Pubitemid 127457311)
55. Prom-on, S.; Liu, F.; Xu, Y.; 2011. Functional modeling of tone, focus and sentence type in Mandarin Chinese. In: Proceedings of ICPhS XVII, Hong Kong, pp. 1638-1641.
56. S. Prom-on, F. Liu, and Y. Xu Post-low bouncing in Mandarin Chinese: Acoustic analysis and computational modeling J. Acoust. Soc. Am. 132 1 2012 421 432
57. S. Prom-on, Y. Xu, and B. Thipakorn Modeling tone and intonation in Mandarin and English as a process of target approximation J. Acoust. Soc. Am. 125 1 2009 405 424
58. Raidt, S.; Bailly, G.; Holm, B.; Mixdorff, H.; 2004. Automatic generation of prosody: Comparing two superpositional systems. In: Proceedings of Speech Prosody 2004, Nara, pp. 417-420.
59. Rose, P.J.; 1988. On the non-equivalence of fundamental frequency and pitch in tonal description. In: Bradley, D.; Henderson, E.J.A.; Mazaudon, M. (Eds.), Prosodic Analysis and Asian Linguistics: To Honour R.K. Sprigg. Pacific Linguistics, Canberra, pp. 55-82.
60. K.N. Ross, and M. Ostendorf A dynamical system model for generating fundamental frequency for speech synthesis IEEE T. Speech Audi. P. 7 3 1999 295 309
61. 0 contours using a superpositional model Speech Commun. 40 4 2003 535 549
62. E.L. Saltzman, and K.G. Munhall A dynamical approach to gestural patterning in speech production Ecol. Psychol. 1 1989 333 382
63. S. Shattuck-Hufnagel, and A.E. Turk A prosody tutorial for investigators of auditory sentence processing J. Psycholinguist. Res. 25 2 1996 193 247 (Pubitemid 126434983)
64. X.-N.S. Shen The Prosody of Mandarin Chinese 1990 University of California Press Berkeley, CA
65. Shih, C.; 1987. The Phonetics of the Chinese Tonal System. AT&T Bell Labs technical memo.
66. K. Silverman F0 segmental cues depend on intonation: The case of the rise after voiced stops Phonetica 43 1-3 1986 76 91
67. Silverman, K.; Beckman, M.; Pitrelli, J.; Ostendorf, M.; Wightman, C.; Price, P.; Pierrehumbert, J.; Hirschberg, J.; 1992. ToBI: A standard for labeling English prosody. In: Proceedings of ICSLP 1992, Banff, pp. 867-870.
68. Sun, X.; 2002. The determination, analysis, and synthesis of fundamental frequency. Ph.D. dissertation, Northwestern University, 2002.
69. X. Sun, and Y. Xu Perceived pitch of synthesized voice with alternate cycles J. Voice 16 4 2002 443 459 (Pubitemid 36626807)
70. Syrdal, A.K.; McGory, J.; 2000. Inter-transcriber reliability of ToBI prosodic labeling. In: Proceedings of ICSLP 2000, Beijing, pp. 235-238.
71. J. 't Hart, R. Collier, and A. Cohen A perceptual Study of Intonation - An Experimental-Phonetic Approach to Speech Melody 1990 Cambridge University Press Cambridge
72. P. Taylor Analysis and synthesis of intonation using the Tilt model J. Acoust. Soc. Am. 107 3 2000 1697 1714 (Pubitemid 30141710)
73. P. Taylor Text-to-Speech Synthesis 2009 Cambridge University Press Cambridge, UK
74. Vainio, M.; Hirst, D.J.; Suni, A.; De Looze, C.; 2009. Using functional prosodic annotation for high quality multilingual, multidialectal and multistyle speech synthesis. In: Proceedings of SPECOM'2009, St. Petersburg, pp. 164-169.
75. M. Vainio, J. Järvikivi, D. Aalto, and A. Suni Phonetic tone signals phonological quantity and word structure J. Acoust. Soc. Am. 128 3 2010 1313 1321
76. 0 generation and alignment A. Botinis, Intonation - Analysis 2000 Modeling and Technology Kluwer, Dordrecht 269 288
77. W.S.-Y. Wang, and K.-P. Li Tone 3 in Pekinese J. Speech Hear. Res. 10 3 1967 629 636
78. M. Wagner, and D.G. Watson Experimental and theoretical advances in prosody: a review Lang. Cognitive Proc. 25 7-9 2010 905 945
79. D.H. Whalen, and A.G. Levitt The universality of intrinsic F0 of vowels J. Phonetics 23 1995 349 366
80. Wightman, C.; 2002. ToBI or not ToBI. In: Proceedings of Speech Prosody 2002, Aix-en-Provence, pp. 25-29.
81. Wightman, C.; Rose, R.C.; 1999. Evaluation of an efficient prosody labeling system for spontaneous speech utterances. In: Proceedings of IEEE ASRU 1999, Keystone, pp. 333-336.
82. Wong, Y.W.; Xu, Y.; 2007. Consonantal perturbation of f0 contours of Cantonese tones. In: Proceedings of ICPhS XVI, Saarbrucken, pp. 1293-1296.
83. Wu, W.L.; Xu.; Y.; 2010. Prosodic focus in Hong Kong Cantonese without post-focus compression. In: Proceedings of Speech Prosody 2010, Chicago, pp. 100040:1-4.
84. Z. Wu Putonghua sanzizu biandiao guilü [Rules of tone sandhi in trisyllabic words in Standard Chinese] Zhongguo Yuyan Xuebao [Bulletin of Chinese Linguistics] 2 1984 70 92
85. Y. Xu Contextual tonal variations in Mandarin J. Phonetics 25 1 1997 61 83 (Pubitemid 127169072)
86. Y. Xu Consistency of tone-syllable alignment across different syllable structures and speaking rates Phonetica 55 1998 179 203 (Pubitemid 128454760)
87. 0 contours J. Phonetics 27 1999 55 105
88. Y. Xu Speech melody as articulatorily implemented communicative functions Speech Commun. 46 3-4 2005 220 251 (Pubitemid 40952514)
89. Y. Xu Speech prosody: a methodological review J. Speech Science 1 1 2011 85 115
90. Y. Xu, S.-W. Chen, and B. Wang Prosodic focus with and without post-focus compression (PFC): A typological divide within the same language family? Linguist. Rev. 29 2012 131 147
91. Y. Xu, A. Kelly, and C. Smillie Emotional expressions as communicative signals S. Hancil, D. Hirst, Prosody and Iconicity 2013 John Benjamins Publishing Co. pp. 33-60
92. Y. Xu, A. Lee, W.-L. Wu, X. Liu, and P. Birkholz Human vocal attractiveness as signaled by body size projection PLoS ONE 8 4 2013 e62397
93. Xu, Y.; Prom-on, S.; 2010-2013. PENTAtrainer1. Available from: http://www.phon.ucl.ac.uk/home/yi/PENTAtrainer1/(9 September 2013).
94. Y. Xu, and X. Sun Maximum speed of pitch change and how it may relate to speech J. Acoust. Soc. Am. 111 3 2002 1399 1413 (Pubitemid 34214080)
95. Y. Xu, and M. Wang Organizing syllables into groups - evidence from F0 and duration patterns in Mandarin J. Phonetics 37 4 2009 507 520
96. Y. Xu, and Q.E. Wang Pitch targets and their realization: Evidence from Mandarin Chinese Speech Commun. 33 4 2001 319 337 (Pubitemid 32160797)
97. Y. Xu, and C.X. Xu Phonetic realization of focus in English declarative intonation J. Phonetics 33 2 2005 159 197 (Pubitemid 40815216)
98. M. Yip Tone 2002 Cambridge University Press Cambridge
99. Yang, X.; Yang, Y.; 2012. Effects of topic structure and syntax on boundary pitch variations in Standard Chinese. In: Proceedings of Speech Prosody 2012, Shanghai, pp. 543-546.
100. Yuan, J.; Shih, C.; Kochanski, G.P.; 2002. Comparison of declarative and interrogative intonation in Chinese. In: Proceedings of Speech Prosody 2002, Aix-en-Provence, pp. 711-714.
101. J. Zhang The role of contrast specific and language specific phonetics in contour tone distribution B. Hayes, R. Kirchner, D. Steriade, Phonetically Based Phonology 2004 Cambridge University Press