On-line experimental methods to evaluate text-to-speech (TTS) synthesis: Effects of voice gender and signal quality on intelligibility, naturalness and preference

Computer Speech and Language

Volumn 19, Issue 2, 2005, Pages 129-146

  Stevens, Catherine ^a   Lees, Nicole ^a   Vonwiller, Julie ^b   Burnham, Denis ^a  

^a UNIVERSITY OF WESTERN SYDNEY   (Australia)

^b Appen Pty Ltd   (Australia)

Author keywords

[No Author keywords available]

Indexed keywords

ACOUSTIC WAVE INTERFERENCE; AUTOMATION; COMPUTATIONAL LINGUISTICS; SIGNAL PROCESSING; SPEECH; SPEECH COMMUNICATION;

ENVIRONMENTAL CONDITIONS; INTELLIGIBILITY; SIGNAL QUALITY; TEXT-TO-SPEECH (TTS) SYNTHESIS;

SPEECH SYNTHESIS;

EID: 10844288683     PISSN: 08852308     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.csl.2004.03.003     Document Type: Article

References (32)

1. Baayen, R.H., Piepenbrock, R., Gulikers, L., 1995. The Celex lexical database (Release 2) [CD-ROM]. Linguistic Data Consortium, University of Pennsylvania [Distributor], Philadelphia, PA.
2. Bailly, G., 2002. Close shadowing natural versus synthetic speech. Int. J. Speech Technol. 6, 11-19.
3. Bellegarda, J.R., Silverman K.E.A., 1998. Improved duration modeling of English phonemes using a root sinusoidal transformation. In: Proc. of the Intl. Conf. on Speech & Language Processing (ICSLP98), Sydney, Australia, Paper 135.
4. Benoit, C., Grice, M., Hazan, V., 1996. The SUS test: A method for the assessment of text-to-speech synthesis intelligibility using Semantically Unpredictable Sentences. Speech Commun. 18, 381-392.
5. Charleston, D.E., Boyer, R.W., 1990. Auditory search using vowel sounds. Perc. Motor Skills 70, 1289-1290.
6. Cole, M., Hood, L., McDermott, R.P., 1997. Concepts of ecological validity: their differing implications for comparative cognitive research. In: Cole, M., Engestroem, Y. (Eds.), Mind, Culture and Activity: Seminal Papers from the Laboratory of Comparative Human Cognition. Cambridge University Press, New York, pp. 49-56.
7. Cox, D.R., 1958. Planning of Experiments. Wiley, New York.
8. Cutler, A., 1976. Phoneme-monitoring reaction time as a function of preceding intonation contour. Perc. Psychophys. 20, 55-60.
9. Davis, J., 1967. Auditory search for syllables embedded within meaningful sentences. JASA 41, 1277-1282.
10. Delogu, C., Conte, S., Sementina, C., 1998. Cognitive factors in the evaluation of synthetic speech. Speech Commun. 24, 153-168.
11. Gong, L., Lai, J., 2003. To mix or not to mix synthetic speech and human speech? Contrasting impact on judge-rated task performance versus self-rated performance and attitudinal responses. Int. J. Speech Technol. 6, 123-131.
12. Greene, B.G., Logan, J.S., Pisoni, D.B., 1986. Perception of synthetic speech produced automatically by rule: intelligibility of eight text-to-speech systems. Behav. Res. Methods, Instruments, Computers 18, 100-107.
13. Hansen, M., Kollmeier, B., 1998. Perception of band-specific speech quality distortions: detection and pairwise comparison. Acustica 86, 24.
14. Hustad, K.C., Kent, R.D., Beukelman, D., 1998. DECTalk and MacinTalk speech synthesisers: intelligibility differences for three listener groups. J. Speech, Lang. Hearing Res. 41, 744-752.
15. Kahneman, D., 1973. Attention and Effort. Prentice-Hall, Englewood Cliffs, NJ.
16. Kalikow, D.N., Stevens, K.N., Elliott, L.L., 1977. Development of a test of speech intelligibility in noise using sentence materials with controlled word predictability. JASA 61, 1337-1351.
17. Koul, R.K., Allen, G.D., 1993. Segmental intelligibility and speech interference thresholds of high quality synthetic speech in presence of noise. J. Speech Hearing Res. 36, 790-798.
18. Latimer, C.R., 1972. Search time as a function of context letter frequency. Perception 1, 57-71.
19. Möbius, B., 2003. Rare events and closed domains: two delicate concepts in speech synthesis. Int. J. Speech Technol. 6, 57-71.
20. Monaghan, A.I.C., 2003. A metrical model of prosody for multilingual TTS. Int. J. Speech Technol. 6, 73-81.
21. Nass, C., Lee, K., 2001. Does computer-synthesized speech manifest personality? Experimental tests of recognition, similarity-attraction, and consistency-attraction. J. Exp. Psychol.: App. 7, 171-181.
22. Nass, C., Robles, E., Heenan, C., Bienstock, H., Treinen, M., 2003. Speech-based disclosure systems: effects of modality, gender of prompt, and gender of user. Int. J. Speech Technol. 6, 113-121.
23. Pashler, H.E., 1998. The Psychology of Attention. MIT Press, Cambridge, MA.
24. Pols LCW, van Santen JPH, Abe, M., Kahn, D., Keller, E., 1998. The use of large text corpora for evaluating text-to-speech systems. In: Proc. of the First Int. Conf. on Language Resources and Evaluation, Granada, Spain.
25. Ralston, J.V., Pisoni, D.B., Mullennix, J.W., 1995. Perception and comprehension of speech. In: Syrdal, A.K., Bennet, R., Greenspan, S. (Eds.), Applied Speech Technology. CRC Press, Boca Raton, FL, pp. 233-288.
26. Roediger, H.L., 1997. The future of cognitive psychology? In: Solso, R.L. (Ed.), Mind and Brain Sciences in the 21st Century. MIT Press, Cambridge, MA, pp. 175-198.
27. Stern, S.E., Mullennix, J.W., Dyson, C.-L, Wilson, S.J., 1999. The persuasiveness of synthetic speech versus human speech. Human Factors 41, 588-595.
28. van Santen JPH, Pols LCW, Abe, M., Kahn, D., Keller, E., Vonwiller J., 1998. Report on the 3rd ESCA TTS workshop evaluation procedure. In: Proc. of the Third ESCA Workshop on Speech Synthesis, Jenolan Caves, Australia.
29. Venkatagiri, H.S., 2003. Segmental intelligibility of four currently used text-to-speech synthesis methods. J. Acoust. Soc. Am. 113, 2095-2104.
30. Viana, M.C., Oliveira, L.C., Mata, A.I., 2003. Prosodic phrasing: machine and human evaluation. Int. J. Speech Technol. 6, 83-94.
31. Voiers, W.D., 1983. Evaluating processed speech using the Diagnostic Rhyme Test. Speech Technol. (Jan/Feb), 30-39.
32. Wherry, R.J., 1938. Orders for the presentation of pairs in the method of paired comparison. J. Exp. Psychol. 23, 651-660.