An Application of Recurrent Nets to Phone Probability Estimation

IEEE Transactions on Neural Networks

Volumn 5, Issue 2, 1994, Pages 298-305

  Robinson, Anthony J ^a  

^a UNIVERSITY OF CAMBRIDGE   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTATIONAL LINGUISTICS; COMPUTER ARCHITECTURE; CONTEXT FREE GRAMMARS; MATHEMATICAL MODELS; PARAMETER ESTIMATION; PROBABILITY; RANDOM PROCESSES; SPEECH ANALYSIS; SPEECH RECOGNITION; SPEECH TRANSMISSION;

CONTEXT INFORMATION; LARGE VOCABULARY; MARKOV MODELS; PHONE PROBABILITY ESTIMATION; PHONE RECOGNITION; RECURRENT NETWORKS; RESOURCE MANAGEMENT TASKS;

NEURAL NETWORKS;

EID: 0028392167     PISSN: 10459227     EISSN: 19410093     Source Type: Journal    
DOI: 10.1109/72.279192     Document Type: Article

References (55)

1. B. A. Pearlmutter, “Dynamic recurrent neural networks,” Tech. Rep. CMU-CS-90-196, School of Computer Science, Carnegie-Mellon University, Dec. 1990.
2. R. J. Williams and D. Zipser, “Gradient-based learning algorithms for recurrent connectionist networks,” Tech. Rep. NU-CCS-90-9, Northeastern University, Apr. 1990.
3. T. Robinson, “Practical network design and implementation,” in Proceedings of the Cambridge Neural Network Summer School, (Cambridge Programme for Industry, Cambridge University), Sept. 1992.
4. D. E. Rumelhart, G. E. Hinton, and R. J. Williams, “Learning internal representations by error propagation,” in Parallel Distributed Processing: Explorations in the Microstructure of Cognition. Vol. I: Foundations, D. E. Rumelhart and J. L. McClelland, Eds. Cambridge, MA: Bradford Books/MIT Press, ch. 8, 1986.
5. P. Ladefoged, A Course in Phonetics. New York: Harcourt Brace Jovanovich, second ed., 1982.
6. L. R. Rabiner, “A tutorial on hidden Markov models and selected applications in speech recognition,” Proc. IEEE, vol. 77, pp. 257–286, 1989.
7. L. R. Bahl, F. Jelinek, and R. L. Mercer, “A maximum likelihood approach to continuous speech recognition,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 5, pp. 179–190, Mar. 1983.
8. F. Jelinek and R. Mercer, “Interpolated estimation of Markov source parameters from sparse data,” Pattern Recognition in Practice, pp. 381–397, 1980.
9. K.-F. Lee, Automatic Speech Recognition: The Development of the SPHINX System. Boston: Kluwer Academic Publishers, 1989.
10. X. D. Huang, Y. Ariki, and M. A. Jack, Hidden Markov models for speech recognition. Edinburgh: Edinburgh University Press, 1990.
11. S. Furui, “Speaker independent isolated word recognition using dynamic features of speech spectrum,” in IEEE Transactions on Acoustics, Speech, and Signal Processing, pp. 52–59, 1986.
12. R. Haeb-Umbach and H. Ney, “Linear discriminant analysis for improved large vocabulary speech recognition,” in Proc. ICASSP, vol. I, pp. 13–16, 1992.
13. E. B. Baum and F. Wilczek, “Supervised learning of probability distributions by neural networks,” in Neural Information Processing Systems D. Z. Anderson, Ed. American Institute of Physics, 1988.
14. J. S. Bridle, “Probabilistic interpretation of feedforward classification network outputs, with relationships to statistical pattern recognition,” in Neuro-computing: Algorithms, Architectures and Applicatations F. Fougelman-Soulie and J. Hérault, Ed. Springer-Verlag, 1989, pp. 227–236.
15. H. Bourlard and C. J. Wellekens, “Links between Markov models and multilayer perceptrons,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 12, pp. 1167–1178, Dec. 1990.
16. H. Gish, “A probabilistic approach to the understanding and training of neural network classifiers,” in Proc. ICASSP, pp. 1361–1364, 1990.
17. M. D. Richard and R. P. Lippmann, “Neural network classifiers estimate Bayesian a posteriori probabilities,” Neural Computation, vol. 3, pp. 461–483, 1991.
18. H. Bourlard and N. Morgan, Continuous Speech Recognition: A Hybrid Approach. Kluwer Academic Publishers, 1993.
19. S. Renals, N. Morgan, H. Bourlard, M. Cohen, and H. Franco, “Connectionist probability estimators in HMM speech recognition,” IEEE Transactions on Speech and Audio Processing, vol. 2, Jan. 1994.
20. N. Morgan and H. Bourlard, “Continuous speech recognition using multilayer perceptrons with hidden Markov models,” in Proc. ICASSP, pp. 413–416, 1990.
21. S. Renals, N. Morgan, M. Cohen, and H. Franco, “Connectionist probability estimation in the Decipher speech recognition system,” in Proc. ICASSP, vol. I, pp. 601–604, 1992.
22. J. S. Bridle, “ALPHA-NETS: A recurrent ‘neural’ network architecture with a hidden Markov model interpretation,” Speech Communication, vol. 9, pp. 83–92, Feb. 1990.
23. J. S. Bridle and L. Dodd, “An Alphanet approach to optimising input transformations for continuous speech recognition,” in Proc. ICASSP. pp. 277–280, 1991.
24. L. T. Niles and H. F. Silverman, “Combining hidden Markov models and neural network classifiers,” in Proc. ICASSP, pp. 417–420, 1990.
25. S. J. Young, “Competitive training in hidden Markov models,” in Proc. ICASSP, pp. 681–684, 1990. Expanded in the technical report CUED/F-INFENG/TR.41, Cambridge University Engineering Department.
26. A. Waibel, T. Hanazawa, G. Hinton, K. Shikano, and K. J. Lang, “Phoneme recognition using time-delay neural networks,” IEEE Transactions on Acoustics, Speech, and Signal Processing, vol. 37, pp. 328–339, 1989.
27. A. J. Robinson and F. Fallside, “Static and dynamic error propagation networks with application to speech coding,” in Neural Information Processing Systems D. Z. Anderson, Ed. American Institute of Physics, 1988.
28. P. J. Werbos, “Backpropagation through time: What it does and how to do it,” Proc. IEEE, vol. 78, pp. 1550–1560, Oct. 1990.
29. G. Kuhn, “A first look at phonetic discrimination using a connectionist architecture with recurrent links,” SCIMP Working Paper No. 4/87, Communications Research Division, Institute for Defense Analyses, Princeton, NJ, 1987.
30. R. J. Williams and D. Zipser, “A learning algorithm for continually running fully recurrent neural networks,” ICS Report 8805, Institute for Cognitive Science, University of California, San Diego, Oct. 1988.
31. J. L. Elman, “Finding structure in time,” Tech. Rep. CRL-8801, Center for Research in Language, UCSD, Apr. 1988.
32. J. H. Schmidhuber, “Learning complex, extended sequences using the principle of history compression,” Neural Computation, vol. 4, pp. 234–242, 1992.
33. T. Robinson, “The state space and ‘ideal input’ representations of recurrent networks,” in Visual Representations of Speech Signals, M. Cooke, S. Beet, and M. Crawford, Eds. New York: John Wiley and Sons, pp. 327–334, 1993.
34. A. J. Robinson and F. Fallside, “A dynamic connectionist model for phoneme recognition,” in Neural Networks from Models to Applications: Proceedings of Neuro'88, Paris: I.D.S.E.T., 1989, pp. 541–550.
35. L. F. Lamel, R. H. Kasel, and S. Seneff, “Speech database development: Design and analysis of the acoustic-phonetic corpus,” in Proceedings of the DARPA Speech Recognition Workshop, pp. 26–32, Mar. 1987.
36. R. A. Jacobs, “Increased rates of convergence through learning rate adaptation,” Neural Networks, vol. 1, pp. 295–307, 1988.
37. W. Schiffmann, M. Joost, and R. Werner, “Optimization of the backpropagation algorithm for training multilayer perceptrons,” Tech. Rep., University of Koblenz, 1992.
38. T. Robinson, J. Holdsworth, R. Patterson, and F. Fallside, “A comparison of preprocessors for the Cambridge recurrent error propagation network speech recognition system,” in Proceedings of the International Conference on Spoken Language Processing, Kobe, Japan, Nov. 1990.
39. T. Robinson, “Several improvements to a recurrent error propagation network phone recognition system,” Tech. Rep. CUED/F-INFENG/TR.82, Cambridge University Engineering Department, Sept. 1991.
40. G. E. Hinton, “Connectionist learning procedures,” Tech. Rep. CMU-CS-87-115, Computer Science Department, Carnegie-Mellon University, June 1987.
41. S. Renals and N. Morgan, “Connectionist probability estimation in HMM speech recognition,” Tech. Rep. TR-92-081, International Computer Science Institute, 1992.
42. T. Robinson and F. Fallside, “A recurrent error propagation network speech recognition system,” Computer Speech and Language, vol. 5, pp. 259–274. July 1991.
43. K.-F. Lee and H.-W. Hon, “Speaker-independent phone recognition using hidden Markov models,” IEEE Transactions on Acoustics, Speech, and Signal Processing, vol. 37, pp. 1641–1648, 1989.
44. S. J. Young and P. C. Woodland, “The use of state tying in continous speech recognition,” in Proceedings of the European Conference on Speech Technology, 1993.
45. S. Kapadia, V. Valtchev, and S. J. Young, “MMI training for continuous phoneme recognition on the TIMIT database,” in Proc. ICASSP, 1993.
46. V. V. Digalakis, M. Ostendorf, and J. R. Rohlicek, “Fast algorithms for phone classification and recognition using segment-based models,” IEEE Transactions on Signal Processing, vol. 40, pp. 2885–2896, Dec. 1992.
47. A. Ljolje, “New developments in phone recognition using an ergodic hidden Markov model,” technical memorandum TM-11222-910829-12, AT&T Bell Laboratories, 1991.
48. P. Price, W. M. Fisher, J. Bernstein, and D. S. Pallett, “The DARPA 1000-word Resource Management database for continuous speech recognition,” in Proc. ICASSP. 1988, pp. 651–654.
49. M. H. Cohen, “Phonological structures for speech recognition,” Ph.D. thesis, Computer Science Division, University of California at Berkeley, Apr. 1989.
50. T. Robinson, “Recurrent nets for phone probability estimation,” in Proceedings of the ARPA Continuous Speech Recognition Workshop, Stanford, Sept. 1992.
51. B. L. Yoon and J. D. Prange, Eds., Proceedings of the ARPA Continuous Speech Recognition Workshop, Sept. 1992.
52. P. C. Woodland and S. J. Young, “The HTK tied-state continuous speech recognizer,” in Proceedings of the European Conference on Speech Technology, pp. 2207–2210, 1993.
53. S. J. Nowlan and G. E. Hinton, “Soft weight-sharing,” in Advances in Neural Information Processing Systems 4, San Mateo, CA: Morgan Kaufmann, 1991.
54. H. Bourlard, N. Morgan, C. Wooters, and S. Renals, “CDNN: A context dependent neural network for continuous speech recognition,” in Proc. ICASSP, vol. II, pp. 349–352, 1992.
55. M. Cohen, H. Franco, N. Morgan, D. Rumelhart, and V. Abrash, “Context-dependent multiple distribution phonetic modelling with MLPs,” in Advances in Neural Information Processing Systems 5, C. L. Giles, S. J. Hanson, and J. D. Cowen, Eds. San Mateo, CA: Morgan Kaufmann, 1993.