Exploiting acoustic and syntactic features for automatic prosody labeling in a maximum entropy framework

IEEE Transactions on Audio, Speech and Language Processing

Volumn 16, Issue 4, 2008, Pages 797-811

  Sridhar, Vivek Kumar Rangarajan ^a   Bangalore, Srinivas ^b   Narayanan, Shrikanth S ^a  

^a University of Southern California   (United States)

^b AT AND T LABS RESEARCH   (United States)

Author keywords

Acoustic prosodic representation; Maximum entropy model; Phrasing; Prominence; Spoken language processing; Supertags; Suprasegmental information; To BI annotation

Indexed keywords

MAXIMUM ENTROPY MODEL; PHRASING; PROMINENCE; SPOKEN LANGUAGE PROCESSING; SUPERTAGS; SUPRASEGMENTAL INFORMATION; TO BI ANNOTATION;

ENTROPY; LABELING; LINGUISTICS; QUERY LANGUAGES; SOFTWARE AGENTS; SPEECH RECOGNITION; SYNTACTICS;

ACOUSTICS;

EID: 60849095980     PISSN: 15587916     EISSN: None     Source Type: Journal    
DOI: 10.1109/TASL.2008.917071     Document Type: Article

References (64)

1. I. Lehiste, Suprasegmentals. Cambridge, MA: MIT Press, 1970.
2. C. W. Wightman and M. Ostendorf, 'Automatic labeling of prosodic patterns," IEEE Trans. Speech Audio Process., vol. 2, no. 4, pp. 469-481, Oct. 1994.
3. P. Koehn, S. Abney. J. Hirschberg, and M. Collins, "Improving into national phrasing with syntactic information," in Proc. ICASSP, 2000, pp. 1289-1290.
4. M. Steedman. "Information structure and the syntax-phonology interface," Linguist. Inquiry, vol. 31, no. 4, pp. 649-689, 2000.
5. M. Liberman and A. Prince, "On stress and linguistic rhythm," Linguist. Inquiry, vol. 8, no. 2, pp. 249-336, 1977.
6. K. Silverman, M. Beckman, J. Pitrelli, M. Ostendorf, C. Wightman, P. Price, J. Pierrehumbert, and J. Hirschberg, "ToBI: A standard for labeling english prosody," in Proc. ICSLP, 1992, pp. 867-870.
7. P. Taylor, "The TILT intonation model," in Proc. ICSLP, 1998, vol. 4, pp. 1383-1386.
8. H. Fujisaki and K. Hirose, "Modelling the dynamic characteristics of voice fundamental frequency with application to analysis and synthesis of intonation," in Proc. 13th Int. Congr. Linguists, 1982, pp. 57-70.
9. E. Grabe, F. Nolan, and K. Farrar, "IViE-A comparative transcription system for intonational variation in english," in Proc. ICSLP, Sydney, Australia, 1998, pp. 1259-1262.
10. D. J. Hirst, N. Ide, and J. Vronis, "Coding fundamental frequency patterns for multilingual synthesis with INTSINT in the MULTEXT project," in Proc. 2nd ESCA/IEEE Workshop Speech Synth., Sep. 1994, pp. 77-81.
11. E. E. Shriberg, R. A. Bates, and A. Stolcke, "A prosody-only decision-tree model for disfluency detection," in Proc. Eurospeech '97, Rhodes, Greece, 1997, pp. 2383-2386.
12. Y. Liu, E. Shriberg, A. Stolcke, D. Hillard, M. Ostendorf, and M. Harper, "Enriching speech recognition with automatic detection of sentence boundaries and disfluencies," IEEE Trans. Audio, Speech, Lang. Process., vol. 14, no. 5, pp. 1526-1540, Sep. 2006.
13. J. G. Kahn, M. Lease, E. Charniak, M. Johnson, and M. Ostendorf, "Effective use of prosody in parsing conversational speech," in Proc. HLT/EMNLP, 2005, pp. 233-240.
14. E. Shriberg, R. Bates, A. Stolcke, P. Taylor, D. Jurafsky, K. Ries, N. Coccaro, R. Martin, M. Meteer, and C. Van Ess-Dykema, "Can prosody aid the automatic classification of dialog acts in conversational speech?," Lang. Speech, vol. 41, no. 3-4, pp. 439-487, 1998.
15. J. Hirschberg, "Pitch accent in context: Predicting intonational prominence from text," Artif, Intell., vol. 63, no. 1-2, pp. 305-340, 1993.
16. P. Shimei and K. McKeown, "Word informativeness and automatic pitch accent modeling," in Proc. EMNLP/VLC, College Park, MD, 1999, pp. 148-157.
17. X. Sun, "Pitch accent prediction using ensemble machine learning," in Proc. ICSLP, 2002, pp. 561-564.
18. A. Conkie, G. Riccardi, and R. C. Rose, "Prosody recognition from speech utterances using acoustic and linguistic based models of prosodic events," in Proc. Eurospeech, Budapest, Hungary, 1999, pp. 523-526.
19. S. Ananthakrishnan and S. Narayanan, "An automatic prosody recognizer using a coupled multi-stream acoustic model and a syntactic-prosodic language model," in Proc. ICASSP, Philadelphia, PA, Mar. 2005, pp. 269-272.
20. M. Hasegawa-Johnson, K. Chen, J. Cole, S. Borys, S.-S. Kim, A. Cohen, T. Zhang, J.-Y. Choi, H. Kim, T.-J. Yoon, and S. Chavara, "Simultaneous recognition of words and prosody in the Boston University Radio Speech Corpus," Speech Commun., vol. 46, pp. 418-439, 2005.
21. M. Gregory and Y. Altun, "Using conditional random fields to predict pitch accent in conversational speech," in Proc. 42nd Annu. Meeting Assoc. Comput. Linguist. (ACL), 2004, pp. 677-704.
22. I. Bulyko and M. Ostendorf, "Joint prosody prediction and unit selection for concatenative speech synthesis," in Proc. ICASSP, 2001, pp. 781-784.
23. X. Ma, W. Zhang, Q. Shi, W. Zhu, and L. Shen, "Automatic prosody labeling using both text and acoustic information," in Proc. ICASSP, Apr. 2003, vol. 1, pp. 516-519.
24. P. Taylor, S. King, S. Isard, and H. Wright, "Intonation and dialogue context as constraints for speech recognition," Lang. Speech, vol. 41, no. 34, pp. 493-512, 2000.
25. A. Stolcke, K. Ries, N. Coccaro, E. Shriberg, R. Bates, D. Jurafsky, P. Taylor, R. Martin, C. Van Ess-Dykema, and M. Meteer, "Dialogue act modeling for automatic tagging and recognition of conversational speech," Comput. Linguist., vol. 26, pp. 339-373, Sep. 2000.
26. J. Hirschberg and P. Prieto, "Training intonational phrasing rules automatically for English and Spanish text-to-speech," Speech Commun., vol. 18, no. 3, pp. 281-290, 1996.
27. K. Chen, M. Hasegawa-Johnson, and A. Cohen, "An automatic prosody labeling system using ANN-based syntactic-prosodic model and GMM-based acoustic-prosodic model," in Proc. ICASSP, 2004, pp. I-509-I-512.
28. M. E. Beckman, M. Diaz-Campos, J. T. McGory, and T. A. Morgan, "Intonation across Spanish, in the tones and break indices framework," Probus, vol. 14, pp. 9-36.
29. M. E. Beckman and J. B. Pierrehumbert, "International structure in Japanese and English," Phonology Yearbook, vol. 3, pp. 255-309.
30. M. Ostendorf, P. J. Price, and S. Shattuck-Hufnagel, "The Boston University Radio News Corpus," Boston Univ., Boston, MA, Tech. Rep. ECS-95-001, Mar. 1995.
31. J. Hirschberg and C. Nakatani, "A prosodic analysis of discourse segments in direction-giving monologues," in Proc.34th Conf. Assoc. Computat. Linguist., 1996, pp. 286-293.
32. P. J. Price, M. Ostendorf, S. Shattuck-Hufnagel, and C. Fong, 'The use of prosody in syntactic disambiguation," J. Acoust. Soc. Amer., vol. 90, no. 6, pp. 2956-2970, 1991.
33. C. W. Wightman, S. Shattuck-Hufnagel, M. Ostendorf, and P. J. Price, "Segmental durations in the vicinity of prosodic phrase boundaries," J. Acoust. Soc. Amer., vol. 91, no. 3, pp. 1707-1717, 1992.
34. K. Ross and M. Ostendorf, "Prediction of abstract prosodic labels for speech synthesis," Comput. Speech Lang., vol. 10, pp. 155-185, Oct.1996
35. A. Nenkova, J. Brenier, A. Kothari, S. Calhoun, L. Whitton, D. Beaver, and D. Jurafsky, "To memorize or to predict: Prominence labeling in conversational speech," in Pmc. NAACL-HLT'07, 2007, pp. 9-16.
36. M. Harper, B. Dorr, B. Roark, J. Hale, Z. Shafran, Y. Liu, M. Lease, M. Snover, L. Young, R. Stewart, and A. Krasnyanskaya, "Parsing speech and structural event detection," in Proc. JHU Summer Workshop, 2005, pp. 1-116, Tech. Rep.
37. M. Q. Wang and J. Hirschberg, "Automatic classification of international phrase boundaries," Comput. Speech Lang., vol. 6, pp. 175-196, 1992.
38. J. F. Pitrelli, M. E. Beckman, and J. Hirschberg, "Evaluation of prosodic transcription labeling reliability in the ToBI framework," in Proc. ICSLP, 1994, pp. 123-126.
39. E. Nöth, A. Batliner, A. KieBling, R. Kompe, and H. Niemann, "VERBMOBIL: The use of prosody in the linguistic components of a speech understanding system," IEEE Trans. Speech Audio Process., vol. 8, no. 5, pp. 519-532, Sep. 2000.
40. T.-J. Yoon, "Predicting prosodic boundaries using linguistic features," in Proc. ICSA Int. Conf. Speech Prosody, Dresden, Germany, 2006, CD-ROM.
41. M. Ostendorf, 1. Shafran, S. Shattuck-Hufnagel, L. Carmichael, and W. Byrne, "A prosodically labeled database of spontaneous speech," in Pmc. ISCA Workshop Prosody in Speech Recognition and Understanding, 2001, pp. 119-121.
42. A. W. Black and P. Taylor, "Assigning phrase breaks from part-of - speech sequences," in Pmc. Eurospeech, Rhodes, Greece, 1997, vol. 2, pp. 995-998.
43. X. Sun and T. H. Applebaum, Intonational Phrase Break Prediction Using Decision Tree and N-Gram Model, vol. 1, pp. 537-540, 2001, (Aalborg, Denmark).
44. N. M. Veilleux, M. Ostendorf, and C. W. Wightman, "Parse scoring with prosodic information," in Pmc. Int. Conf. Spoken Lang. Process., 1992, pp. 1605-1608.
45. N. M. Veilleux and M. Ostendorf, "Prosody/parse scoring and its application in atis," in HLT'93: Pmc. Workshop Human Lang. Technol., Morristown, NJ, 1993, pp. 335-340, Assoc. Comput. Linguist.
46. V. K. Rangarajan Sridhar, S. Bangalore, and S. Narayanan, "Acoustic-syntactic maximum entropy model for automatic prosody labeling," in Proc. IEEE/ACL Spoken Lang. Technol., Aruba, Dec. 2006, pp. 74-77.
47. V. K. Rangarajan Sridhar, S. Bangalore, and S. Narayanan, "Exploiting acoustic and syntactic features for prosody labeling in a maximum entropy framework," in Proc. NAACL-HLT, 2007, pp. 1-8.
48. A. Berger, S. D. Pietra, and V. D. Pietra, "A maximum entropy approach to natural language processing," Comput. Linguist., vol. 22, no. 1, pp. 39-71, 1996.
49. M. Dudik, S. Phillips, and R. E. Schapire. "Performance guarantees for regularized maximum entropy density estimation," in Proc. COLT, Banff, Canada, 2004, pp. 472-486, Springer-Verlag.
50. P. Haffner, "Scaling large margin classifiers for spoken language understanding," Speech Commun., vol. 48, no. IV, pp. 239-261, 2006.
51. S. Bangalore and A. K. Joshi, "Supertagging: An approach to almost parsing," Computat. Linguist., vol. 25, no. 2, pp. 237-265, 1999.
52. A. Joshi and Y. Schabes, A. Salomaa and G. Rozenberg, Eds., "Tree - adjoining grammars," in Handbook of Formal Lanaguages and Automata. Berlin: Springer-Verlag, 1996.
53. "A Lexicalized Tree-Adjoining Grammar for English," Univ. of Pennsylvania, Philadelphia, Tech. Rep., 2001 [Online], Available: http://www.cis.upenn.edu/xtag/gramrelease.html, XTAG
54. J. Chen and K. Vijay-Shanker, "Automated extraction of tags from the penn treebank," in Proc. 6th Int. Workshop Parsing Technologies, Trento, Italy. 2000, pp. 73-89.
55. F. Xia, M. Palmer, and A. Joshi, "A uniform method of grammar extraction and its applications," in Pmc. Empirical Methods in Natural Lang. Process., 2000, pp. 53-62.
56. S. Bangalore, A. Emami, and P. Haffner, "Factoring Global Inference by Enriching Local Representations," AT &T Labs-Research, Tech. Rep., 2005.
57. S. Bangalore and P. Haffner, "Classification of large label sets," in Pmc. Snowbird Learning Workshop, 2005, CD-ROM.
58. M. Hasegawa-Johnson, J. Cole, C. Shih, K. Chen, A. Cohen, S. Chavarria, H. Kim, T. Yoon, S. Borys, and J.-Y. Choi, "Speech recognition models of the interdependence among syntax, prosody, and segmental acoustics," in Proc. HLT/NAACL, Workshop Higher-Level Knowledge in Automatic Speech Recognition and Understanding, May 2004, pp. 56-63.
59. J. Hirschberg and O. Rainbow, "Learning prosodic features using a tree representation," in Proc. Eurospeech, Aalborg, Denmark, 2001, pp. 1175-1180.
60. E. Shriberg, L. Ferrer, S. Kajarekar, A. Venkataraman, and A. Stolcke, "Modeling prosodic feature sequences for speaker recognition," Speech Commun., vol. 46, pp. 455-472, 2005.
61. "AT& T Natural Voices Speech Synthesizer." [Online]. Available: http://www.naturalvoices.att.com.
62. A. W. Black, P. Taylor, and R. Caley, The Festival Speech Synthesis System 1998 [Online], Available: http://festvox.org/festival
63. K. Chen, M. Hasegawa-Johnson, A. Cohen, S. Borys, S.-S. Kim, J. Cole, and J.-Y. Choi, "Prosody dependent speech recognition on radio news corpus of American English," IEEE Trans. Audio, Speech, Lang. Process., vol. 14, no. 1, pp. 232-245, Jan. 2006.
64. V. K. Rangarajan Sridhar, S. Bangalore, and S. Narayanan, "Exploiting prosodic features for dialog act tagging in a discriminative modeling framework," in Proc. ICSLP, 2007.