Neural Networks for Statistical Recognition of Continuous Speech

Proceedings of the IEEE

Volumn 83, Issue 5, 1995, Pages 742-770

  Morgan, Nelson ^a   Bourlard, Hervè A ^b  

^a INTERNATIONAL COMPUTER SCIENCE INSTITUTE   (United States)

^b FACULTÉ POLYTECHNIQUE DE MONS   (Belgium)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTER SIMULATION; ESTIMATION; NEURAL NETWORKS; PATTERN RECOGNITION; PROBABILITY; STATISTICAL METHODS; TECHNOLOGY;

ARTIFICIAL NEURAL NETWORKS; AUTOMATIC SPEECH RECOGNITION; HIDDEN MARKOV MODELS; PHONETIC PROBABILITY ESTIMATION; STATISTICAL PATTERN RECOGNITION;

SPEECH RECOGNITION;

EID: 0029308753     PISSN: 00189219     EISSN: 15582256     Source Type: Journal    
DOI: 10.1109/5.381844     Document Type: Article

References (90)

1. J. Allen, “How do humans process and recognize speech?” IEEE Trans. Speech and Audio Process., special issue on robust speech recognition, vol. 2, pp. 567-578, Oct. 1994.
2. K. AsanovicA, J. Beck, J. Feldman, N. Morgan, and J. Wawrzynek, “A supercomputer for neural computation,” Proc. ICNN, vol. 7, pp. 4462-4465, 1994.
3. L. R. Bahl, P. F. Brown, P. V. de Souza, and R. L. Mercer, “Maximum mutual information estimation of hidden Markov model parameters,” in IEEE Proc. Int. Conf. on Acoust., Speech, and Signal Process., 1986, Tokyo, pp. 49-52.
4. L. Baum, “An inequality and associated maximization techniques in statistical estimation of probabilistic functions of Markov processes,” Inequalities, no. 3, pp. 1-8, 1972.
5. Y. Bengio, R. De Mori, G. Flammia, and R. Kompe, “Global optimization of a neural neural network-hidden Markov model hybrid,” IEEE Trans. Neural Networks, vol. 3, pp. 252-258, 1992.
6. H. Bourlard and C. J. Wellekens, “Links between Markov models and multilayer perceptrons,” IEEE Trans. Patt. Anal. and Mach. Intell., vol. 12, pp. 1167-1178, 1990.
7. H. Bourlard and N. Morgan, Connectionist Speech Recognition-A Hybrid Approach. Amsterdam: Kluwer, 1994.
8. H. Bourlard, Y. Konig, and N. Morgan, “REMAP: recursive estimation and maximization of a posterior probabilities — Application to transition-based connectionist speech recognition,” ICSI Tech. Rep. TR-94-064, 1994.
9. J. S. Bridle, “Probabilistic interpretation of feedforward classification network outputs, with relationships to statistical pattern recognition,” in Neurocomputing: Algorithms, Architectures and Applications, F. Fogelman Soulié and J. Hérault, Eds., NATO ASI Series, pp. 227-236, 1990.
10. J. S. Bridle, “Alpha-Nets: a recurrent neural network architecture with a hidden Markov model interpretation,” Speech Commun., vol. 9, pp. 83-92, 1990.
11. P. F. Brown, “The acoustic-modeling problem in automatic speech recognition,” Ph.D. dissertation, School of Computer Sci., Carnegie Mellon Univ., 1987.
12. Y.-L. Chow, “Maximum mutual information estimation of HMM parameters for continuous speech recognition using the N-best algorithm,” in IEEE Proc. Int. Conf. on Acoust., Speech, and Signal Process., Albuquerque, NM, 1990, pp. 701-704.
13. M. Cohen, “Phonological structures for speech recognition,” Ph.D. dissertation, Univ. Calif. Berkeley, 1989.
14. T. Cover and J. A. Thomas, Elements of Information Theory. New York: Wiley, 1991.
15. M. Cohen, H. Murveit, J. Bernstein, P. Price, and M. Weintraub, “The DECIPHER speech recognition system,” in IEEE Proc. Int. Conf. on Acoust., Speech, and Signal Process., Albuquerque, NM, 1990, pp. 77-80.
16. M. Cohen, H. Franco, N. Morgan, D. Rumelhart, and V. Abrash, “Context-dependent multiple distribution phonetic modeling,” in Advances in Neural Information Processing Systems 5, S. J. Hanson, J. D. Cowan, and C. L. Giles, Eds., pp. 649-657, 1993.
17. G. Cybenko, “Approximation by superpositions of a sigmoid function,” Math. Control, Signals and Syst., vol. 2, pp. 303-314, 1989.
18. A. de Boor, “Customs—A load balancing system,” Project Rep., Computer Science Div.,, Univ. Calif. Berkeley, CA, Nov. 1988.
19. A. Dempster, N. Laird, and D. Rubin, “Maximum likelihood from incomplete data via the EM algorithm,” J. Royal Stat. Soc., vol. 39, pp. 1-38, 1977.
20. L. Deng and D. Sun, “A statistical approach to automatic speech recognition using the atomic speech units constructed from overlapping articulatory features,” JASA, vol. 95, pp. 2702-2719, 1994.
21. R. O. Duda and P. E. Hart, Pattern Classification and Scene Analysis. New York: Wiley, 1973.
22. A. El-Jaroudi and J. Makhoul, “A new error criterion for posterior probability estimation with neural nets,” in IEEE Proc. Int. Joint Conf. on Neural Networks, San Diego, CA, 1990, pp. 111:185-192.
23. K. Fukunaga, Introduction to Statistical Pattern Recognition. New York: Academic, 1972.
24. S. Furui, “Speaker independent isolated word recognizer using dynamic features of speech spectrum,” IEEE Trans. Acoust., Speech, Signal Process., vol. ASSP-34, pp. 52-59, Jan. 1986.
25. J.-L. Gauvain, L.F. Lamel, G. Adda, and M. Adda-Decker, “The LIMSI continuous speech dictation system: evaluation on the ARPA Wall Street Journal Task,” in IEEE Proc. Int. Conf. on Acoust., Speech, and Signal Process., Adelaide, Australia, 1994, pp. I-557-560.
26. H. Gish, “A probabilistic approach to the understanding and training of neural network classifiers,” in IEEE Proc. Int. Conf. on Acoust., Speech and Signal Process., Albuquerque, NM, 1990, pp. 1361-1364.
27. J. Glass, “Finding acoustic regularities in speech applications to phonetic recognition,” Ph.D. dissertation, MIT, 1988.
28. R. Haeb-Umbach and H. Ney, “Linear discriminant analysis for improved large vocabulary continuous speech recognition,” in IEEE Proc. Int. Conf. on Acoust., Speech, and Signal Process., San Francisco, CA, 1992, pp. I-13-16.
29. J. Hampshire and A. Waibel, “Connectionist architectures for multi-speaker phoneme recognition,” in Advances in Neural Information Processing Systems 2, D. S. Touretzky, Ed. San Mateo, CA: Morgan Kaufmann, 1990.
30. M. M. Hochberg, G. D. Cook, S. J. Renals, A. J. Robinson, “Connectionist model combination for large vocabulary speech recognition,” in IEEE Proc. NNSP, Ermioni, Greece, Sept. 1994, pp 269-278.
31. X. D. Huang, Y. Ariki, and M. A. Jack, Hidden Markov Models for Speech Recognition. Edinburgh, UK: Edinburgh Univ. Press, 1990.
32. F. Jelinek, “Continuous speech recognition by statistical methods,” Proc. IEEE, vol. PROC-64, pp. 532-555, Apr. 1976.
33. M. Jordan, “Serial order: a parallel distributed processing approach,” in Advances in Connectionist Theory: Speech, J. L. Elman and D. E. Rumelhart, Eds. Hillside, NJ: Erlbaum, 1989.
34. M. I. Jordan and R. A. Jacobs, “Hierarchical mixtures of experts and the EM algorithm,” Neural Computation, vol. 6, pp. 181-214, 1994.
35. B. H. Juang and L. R. Rabiner, “Mixture autoregressive hidden markov models for speech signals,” IEEE Trans. Acoust., Speech, and Signal Process., vol. ASSP-33, pp. 1404-1413, June 1985.
36. D. Jurafsky, C. Wooters, G. Tajchman, J. Segal, A. Stolcke, and N. Morgan, “The Berkeley Restaurant Project,” in Proc. Int. Conf. on Spoken Language Process., Yokohama, Japan, 1994.
37. S. Katagiri, C.-H. Lee, and B.-H. Juang, “New discriminative training algorithms based on the generalized probabilistic descent method,” IEEE Proc. NNSP, B. H. Juang, S. Y. Kung, and C. A. Kamm, Eds., pp. 299-308, 1991.
38. A. Kehagias, “Optimal control for training: the missing link between hidden Markov models and connectionist networks,” Division of Applied Mathematics preprint, Brown Univ., 1989.
39. D. G. Kimber, M. A. Bush and G. Tajchman, “Speaker-independent vowel classification using hidden Markov models and LVQ2,” in IEEE Proc. Int. Conf. on Acoust., Speech, and Signal Process., Albuquerque, NM, 1990, pp. 497-500.
40. Y. Konig, N. Morgan, C. Wooters, V. Abrash, M. Cohen, and H. Franco, “Modeling consistency in a speaker independent continuous speech recognition system,” in Advances in Neural Information Processing Systems, S. J. Hanson, J. D. Cowan, and C. L. Giles, Eds., vol. 5, pp. 682-687, 1993.
41. F. Kubala, A. Anastasakos, J. Makhoul, L. Nguyen, R. Schwartz, and G. Zavaliagkos, “Comparative experiments on large vocabulary speech recognition,” in IEEE Proc. Int. Conf. on Acoust., Speech, and Signal Process., Adelaide, Australia, 1994, pp. I-561-564.
42. G. Kuhn and N. Herzberg, “Variations on training of recurrent networks,” in Proc. 24th Conf. Inform. Sci. and Syst., Princeton, NJ, Mar. 1990.
43. K. J. Lang, A. H. Waibel, and G. E. Hinton, “A time-delay neural network architecture for isolated word recognition,” Neural Networks, vol. 3, no. 1, pp. 23-43, 1990.
44. K. F. Lee, Large Vocabulary Speaker-Independent Continuous Speech Recognition: The SPHINX System. New York: Kluwer, 1989.
45. S. E. Levinson, “Structural methods in automatic speech processing,” Proc. IEEE, vol. 73, pp. 1625-1650, 1983.
46. L. A. Liporace, “Maximum likelihood estimation for multivariate observations of Markov sources,” IEEE Trans. Inform. Theory, vol. IT-28, pp. 729-734, Oct. 1982.
47. R. Lippmann and B. Gold, “Neural classifiers useful for speech recognition,” in IEEE Proc. 1st Int. Conf. on Neural Networks, San Diego, CA, vol. 4, 1987, pp. 417-422.
48. R. P. Lippmann, “Review of neural networks for speech recognition,” Neural Computation, vol. 1, no. 1, pp. 1-38, 1989.
49. D. M. Lubensky, A. O. Asadi, and J. M. Naik, “Connected digit recognition using connectionist probability estimators and mixture-gaussian densities,” in IEEE Proc. Int. Conf. on Spoken Language Process., Yokohama, Japan, 1994, pp. 295-298.
50. B. Merialdo, “Phonetic recognition using hidden Markov models and maximum mutual information for speech recognition,” in IEEE Proc. Int. Conf. on Acoust., Speech, and Signal Process., New York, 1988, pp. 111-114.
51. D. P. Morgan and C. L. Scofield, Neural Networks and Speech Processing. Amsterdam: Kluwer, 1991.
52. N. Morgan and H. Bourlard, “Generalization and parameter estimation in feedforward nets: Some experiments,” in Advances in Neural Information Processing Systems 2 D. S. Touretzky, Ed. San Mateo, CA: Morgan Kaufmann, 1990, pp. 630-637.
53. N. Morgan, J. Beck, P. Kohn, J. Bilmes, E. Allman, and J. Beer, “The ring array processor (RAP): a multiprocessing peripheral for connectionist applications,” J. Parallel and Distrib. Computing, special issue on neural networks, vol. 14, pp. 248-259, 1992.
54. N. Morgan, H. Bourlard, S. Renals, M. Cohen, and H. Franco, “Hybrid neural network/hidden Markov model systems for continuous speech recognition,” Int. J. Patt. Recog. and Artif. Intell., special issue on advances in pattern recognition systems using neural networks, I. Guyon and P. Wang, Eds., vol. 7, no. 4, 1993.
55. N. Morgan, H. Bourlard, S. Greenberg, and H. Hermansky, “Stochastic perceptual auditory-event-based models for speech recognition,” Int. Conf. on Spoken Language Process., pp. 1943-1946, Oct. 1994.
56. N. Morgan, H. Bourlard, S. Greenberg, and H. Hermansky, “Stochastic perceptual models of speech,” IEEE Proc. Int. Conf. on Acoust., Speech, and Signal Process., to be published.
57. A. Nadas, “A decision-theoretic formulation of a training problem in speech recognition and a comparison of training by unconditional versus conditional maximum likelihood,” IEEE Trans. Acoust., Speech, and Signal Process., vol. ASSP-31, pp. 814-817, 1983.
58. J. M. Naik and D. M. Lubensky, “A hybrid HMM-MLP speaker verification algorithm for telephone speech,” in IEEE Proc. Int. Conf. on Acoust., Speech, and Signal Process., Adelaide, Australia, 1994, pp. I-153-156.
59. H. Ney, “The use of a one-stage dynamic programming algorithm for connected word recognition,” IEEE Trans. Acoust., Speech, and Signal Process., vol. 32, pp. 263-271, 1984.
60. L. Niles, H. Silverman, G. Tajchman, and M. Bush, “How limited training data can allow a neural network classifier to outperform an ‘optimal’ statistical classifier,” in IEEE Proc. Int. Conf. on Acoust., Speech, and Signal Process., Glasgow, UK, 1989, pp. 17-20.
61. L. Niles and H. Silverman, “Combining hidden Markov models and neural network classifiers,” in IEEE Proc. Int. Conf. on Acoust., Speech, and Signal Process., Albuquerque, NM, 1990, pp. 417-420.
62. Y. Normandin, R. Lacouture, and R. Cardin, “MMIE training for large vocabulary continuous speech recognition,” in Proc. Int. Conf. on Spoken Language Process., Yokohama, Japan, 1994, pp. 1367-1370.
63. D. B. Paul, J. K. Baker, and J. M. Baker, “On the interaction between true source, training, and testing language models,” in IEEE Proc. Int. Conf. on Acoust., Speech, and Signal Process., Toronto, Canada, 1991, pp. 569-572.
64. D. Paul and B. Necioglu, “The Lincoln large-vocabulary stack-decoder HMM CSR,” in IEEE Proc. Int. Conf. on Acoust., Speech, and Signal Process., vol. 2, Minneapolis, MN, 1993, pp. 660-663.
65. S. Peeling and R. Moore, “Isolated digit recognition experiments using the multi-layer perceptron,” Speech Commun., vol. 7, pp. 403-409, 1988.
66. T. Poggio and F. Girosi, “Networks for approximation and learning,” Proc. IEEE, vol. 78, pp. 1481-1497, Sept. 1989.
67. A. B. Poritz, “Linear predictive hidden Markov models and the speech signal,” in IEEE Proc. Int. Conf. on Acoust., Speech, and Signal Process., Paris, 1982, pp. 1291-1294.
68. A. B. Poritz and A. L. Richter, “On hidden Markov models in isolated word recognition,” in IEEE Proc. Int. Conf. on Acoust., Speech, and Signal Process., Tokyo, Japan, 1986, pp. 14. 3.1-4.
69. L. R. Rabiner, “A tutorial on hidden Markov models and selected applications in speech recognition,” Proc. IEEE, vol. 77, pp. 257-285, Feb. 1989.
70. L. R. Rabiner and B. H. Juang, Fundamentals of Speech Recognition. Englewood Cliffs, NJ: Prentice-Hall, 1993.
71. S. Renals, M. Morgan, and H. Bourlard, “Probability estimation by feed-forward networks in continuous speech recognition,” in IEEE Proc. Workshop on Neural Networks for Signal Process., Princeton, NJ, B. H. Juang, S. Y. Kung and C. A. Kann, Eds., 1991, pp. 309-318.
72. S. Renals, N. Morgan, M. Cohen, and H. Franco, “Connectionist probability estimation in the decipher speech recognition system,” in IEEE Proc. Int. Conf. on Acoust., Speech, and Signal Process., San Francisco, CA, 1992, pp. 601-604.
73. S. Renals, N. Morgan, H. Bourlard, M. Cohen, and H. Franco, “Connectionist probability estimators in HMM speech recognition,” IEEE Trans. Speech and Audio Process., vol. 2, pp. 161-174, Jan. 1994.
74. M. D. Richard and R. P. Lippmann, “Neural network classifiers estimate Bayesian a posteriori probabilities,” Neural Computation, no. 3, pp. 461-483, 1991.
75. T. Robinson and F. Fallside, “Phoneme recognition from the TIMIT database using recurrent error propagation networks,” Tech. Rep. CUED/F-INFENG/TR42, Eng. Dept., Cambridge Univ., UK, 1990.
76. T. Robinson and F. Fallside, “A recurrent error propagation network speech recognition system,” Computer Speech and Language, no. 5, pp. 259-274, 1991.
77. T. Robinson, “An application of recurrent nets to phone probability estimation,” IEEE Trans. Neural Networks, vol. 5, pp. 298-305, 1994.
78. D. E. Rumelhart, G. E. Hinton, and R. J. Williams, “Learning internal representations by error propagation,” in Parallel Distributed Processing, D. E. Rumelhart and J. L. McClelland, Eds. Cambridge MA: MIT Press, 1986, vol. 1, pp. 318-362.
79. M. Schenkel, I. Guyon, and D. Henderson, “On-line cursive script recognition using neural networks and hidden Markov models,” in IEEE Proc. Int. Conf. on Acoust., Speech, and Signal Process., Adelaide, Australia, 1994, pp. II.637-640.
80. M. Schenkel, I. Guyon, and D. Henderson, “On-line cursive script recognition using time delay neural networks and hidden Markov models,” Special issue of Mach. Vision and Applications on Cursive Script Recognition, R. Plamondon, Ed., to be published.
81. R. Schwartz, oral presentation, Speech Res. Symp. 13, Johns Hopkins, MD, 1993.
82. A. Stolcke and S. Omohundro, “Hidden Markov model induction by Bayesian model merging,” in Advances in Neural Information Processing Systems, vol. 5, S. J. Hanson, J. D. Cowan, and C. L. Giles, Eds. San Mateo, CA: Morgan Kaufmann, 1993.
83. S. Viglione, “Applications of pattern recognition technology in adaptive learning and pattern recognition systems,” in Adaptive Learning and Pattern Recognition Systems, J. Mendel and K. Fu, Eds. New York, Academic, 1970, pp. 115-161.
84. A. Waibel, T. Hanazawa, G. Hinton, K. Shikano, and K. J. Lang, “Phoneme recognition: neural networks vs. hidden Markov models,” in IEEE Proc. Int. Conf. on Acoust., Speech, and Signal Process., New York, 1988, vol. 1, pp. 107-110.
85. R. Watrous and L. Shastri, “Learning phonetic features using connectionist networks: an experiment in speech recognition,” in Proc. 1st Int. Conf. on Neural Networks, San Diego, CA, 1987, vol. 2, pp. 619-627.
86. C. J. Wellekens, “Explicit time correlation in hidden Markov models for speech recognition,” in IEEE Proc. Int. Conf. on Acoust., Speech, and Signal Process., Dallas, TX, 1987, pp. 384-386.
87. P. J. Werbos, “Backpropagation through time: what it does and how to do it,” Proc. IEEE, vol. 78, pp. 1150-1160, 1990.
88. H. White, “Multilayer feedforward networks can learn arbitrary mappings: connectionist nonparametric regression with automatic and semi-automatic determination of network complexity,” Discussion Paper, Univ. Calif. San Diego, Dept. Economics, 1988.
89. P. C. Woodland, J. J. Odell, V. Valtchev, and S. J. Young, “Large vocabulary continuous speech recognition using HTK,” in IEEE Proc. Int. Conf. on Acoust, Speech, and Signal Process., Adelaide, Australia, 1994, pp. II-125-128.
90. C. Wooters, “Lexical modeling in a speaker-independent speech understanding system,” ICSI Tech. Rep. TR-93-068, also a Univ. Calif. Berkeley Ph.D. dissertation, 1993.