Hidden neural networks

Neural Computation

Volumn 11, Issue 2, 1999, Pages 541-563

  Krogh, Anders ^a   Riis, Søren Kamaric ^a  

^a TECHNICAL UNIVERSITY OF DENMARK   (Denmark)

Author keywords

[No Author keywords available]

Indexed keywords

ARTICLE; ARTIFICIAL NEURAL NETWORK; AUTOMATED PATTERN RECOGNITION; DATA BASE; HUMAN; PROBABILITY; SPEECH PERCEPTION; STATISTICAL MODEL;

DATABASES; HUMANS; LIKELIHOOD FUNCTIONS; MARKOV CHAINS; MODELS, STATISTICAL; NEURAL NETWORKS (COMPUTER); PATTERN RECOGNITION, AUTOMATED; SPEECH PERCEPTION;

EID: 0033556867     PISSN: 08997667     EISSN: None     Source Type: Journal    
DOI: 10.1162/089976699300016764     Document Type: Review

References (46)

1. Ackley, D. H., Hinton, G. E., & Sejnowski, T. J. (1985). A learning algorithm for Boltzmann machines. Cognitive Science, 9, 147-169.
2. Bahl, L. R., Brown, P. F., de Souza, P. V., & Mercer, R. L. (1986). Maximum mutual information estimation of hidden Markov model parameters for speech recognition. In Proceedings of ICASSP'86 (pp. 49-52).
3. Baldi, P., & Chauvin, Y. (1994). Smooth on-line learning algorithms for hidden Markov models. Neural Computation, 6(2), 307-318.
4. Baldi, P., & Chauvin, Y. (1996). Hybrid modeling, HMM/NN architectures, and protein applications. Neural Computation, 8, 1541-1565.
5. Bengio, S., & Bengio, Y. (1996). An EM algorithm for asynchronous input/output hidden Markov models. In Proceedings of the ICONIP'96.
6. Bengio, Y., De Mori, R., Flammia, G., & Kompe, R. (1992). Global optimization of a neural network-hidden Markov model hybrid. IEEE Transactions on Neural Networks, 3(2), 252-259.
7. Bengio, Y., & Frasconi, P. (1996). Input/output HMMs for sequence processing. IEEE Transactions on Neural Networks, 7(5), 1231-1249.
8. Bengio, Y., LeCun, Y., Nohl, C., & Burges, C. (1995). Lerec: A NN/HMM hybrid for on-line handwritting recognition. Neural Computation, 7(5).
9. Bourlard, H., Konig, Y., & Morgan, N. (1994). REMAP: Recursive estimation and maximization of a posteriori probabilities (Tech. Rep. TR-94-064). Berkeley, CA: International Computer Science Institute.
10. Boutilier, C., Friedman, N., Goldszmidt, M., & Koller, D. (1996). Context-specific independence in Bayesian networks. In E. Horvitz & F. V. Jensen (Eds.), Proceedings of the Twelfth Conference on Uncertainty in Artificial Intelligence (pp. 115-123). San Francisco: Morgan Kaufmann.
11. Bridle, J. S. (1990). Alphanets: A recurrent "neural" network architecture with a hidden Markov model interpretation. Speech Communication, 9, 83-92.
12. Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977). Maximum likelihood from incomplete data via the EM algorithm. Royal Statistical Society B, 39, 1-38.
13. Durbin, R. M., Eddy, S. R., Krogh, A., & Mitchison, G. (1998). Biological sequence analysis. Cambridge: Cambridge University Press.
14. Eddy, S. R. (1996). Hidden Markov models. Current Opinion in Structural Biology, 6, 361-365.
15. Garofolo, J. S., Lamel, L. F., Fisher, W. M., Fiscus, J. G., Pallet, D. S., & Dahlgren, N. L. (1993). DARPA TIMIT Acoustic-Phonetic Continuous Speech Corpus CD-ROM. Gaithersburg, MD: National Institute of Standards.
16. Gopalakrishnan, P. S., Kanevsky, D., Nádas, A., & Nahamoo, D. (1991). An inequality for rational functions with applications to some statistical estimation problems. IEEE Transactions on Information Theory, 37(1), 107-113.
17. Helmbold, D. P., Schapire, R. E., Singer, Y., & Warmuth, M. K. (1997). A comparison of new and old algorithms for a mixture estimation problem. Machine Learning, 27(1), 97-119.
18. Hennebert, J., Ris, C., Bourlard, H., Renals, S., & Morgan, N. (1997). Estimation of global posteriors and forward-backward training of hybrid HMM/ANN systems. In Proceedings of EUROSPEECH'97.
19. Hertz, J. A., Krogh, A., & Palmer, R. (1991). Introduction to the theory of neural computation. Redwood City, CA: Addison-Wesley.
20. Johansen, F. T. (1994). Global optimisation of HMM input transformations. In Proceedings of ICSLP'94 (Vol. 1, pp. 239-242).
21. Johansen, F. T., & Johnsen, M. H. (1994). Non-linear input transformations for discriminative HMMs. In Proceedings of ICASSP'94 (Vol. 1, pp. 225-228).
22. Juang, B. H., & Rabiner, L. R. (1991). Hidden Markov models for speech recognition. Technometrics, 33(3), 251-272.
23. Kivinen, J., & Warmuth, M. K. (1997). Exponentiated gradient versus gradient descent for linear predictors. Information and Computation, 132(1), 1-63.
24. Kohonen, T., Barna, G., & Chrisley, R. (1988). Statistical pattern recognition with neural networks: Benchmarking studies. In Proceedings of ICNN'88 (Vol. 1, pp. 61-68).
25. Konig, Y., Bourlard, H., & Morgan, N. (1996). REMAP: Recursive estimation and maximization of a posteriori probabilities - Application to transition-based connectionist speech recognition. In D. S. Touretzky, M. C. Mozer, & M. E. Hasselmo (Eds.), Advances in neural information processing systems, 8 (pp. 388-394). Cambridge, MA: MIT Press.
26. Krogh, A. (1994). Hidden Markov models for labeled sequences. In Proceedings of the 12th IAPR ICPR'94 (pp. 140-144).
27. Krogh, A., Brown, M., Mian, I. S., Sjölander, K., & Haussler, D. (1994). Hidden Markov models in computational biology: Applications to protein modeling. Journal of Molecular Biology, 235, 1501-1531.
28. Lauritzen, S. L. (1996). Graphical models. New York: Oxford University Press.
29. Le Cerf, P., Ma, W., & Compernolle, D. V. (1994). Multilayer perceptrons as labelers for hidden Markov models. IEEE Transactions on Speech and Audio Processing, 2(1), 185-193.
30. Lee, K.-F. (1990). Context-dependent phonetic hidden Markov models for speaker-independent continuous speech recognition. IEEE Transactions on Acoustics, Speech and Signal Processing, 38(4), 599-609.
31. McDermott, E., & Katagiri, S. (1991). LVQ-based shift-tolerant phoneme recognition. IEEE Transactions on Acoustics, Speech and Signal Processing, 39, 1398-1411.
32. Morgan, N., Bourlard, H., Greenberg, S., & Hermansky, H. (1994). Stochastic perceptual auditory-event-based models for speech recognition. In Proceedings of Interternational Conference on Spoken Language Processing (pp. 1943-1946).
33. Nádas, A. (1983). A decision-theoretic formulation of a training problem in speech recognition and a comparison of training by unconditional versus conditional maximum likelihood. IEEE Transactions on Acoustics, Speech and Signal Processing, 31(4), 814-817.
34. Nádas, A., Nahamoo, D., & Picheny, M. A. (1988). On a model-robust training method for speech recognition. IEEE Transactions on Acoustics, Speech and Signal Processing, 36(9), 814-817.
35. Rabiner, L. R. (1989). A tutorial on hidden Markov models and selected applications in speech recognition. Proceedings of IEEE, 77(2), 257-286.
36. Renals, S., Morgan, N., Bourlard, H., Cohen, M., & Franco, H. (1994). Connectionist probability estimators in HMM speech recognition. IEEE Transactions on Speech and Audio Processing, 2(1), 161-174.
37. Riis, S. K. (1998a). Hidden Markov models and neural networks for speech recognition. Doctoral dissertation, IMM-PHD-1998-46, Technical University of Denmark.
38. Riis, S. K. (1998b). Hidden neural networks: Application to speech recognition. In In Proceedings of ICASSP'98 (Vol. 2, pp. 1117-1121).
39. Riis, S. K., & Krogh, A. (1997). Hidden neural networks: A framework for HMM/NN hybrids. In Proceedings of ICASSP'97 (pp. 3233-3236).
40. Robinson, A. J. (1994). An application of recurrent nets to phone probability estimation. IEEE Transactions on Neural Networks, 5, 298-305.
41. Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986). Learning representations by back-propagating errors. Nature, 323, 533-536.
42. Saul, L. K., & Jordan, M. I. (1995). Boltzman chains and hidden Markov models. In G. Tesauro, D. Touretzky, & T. Leen (Eds.), Advances in neural information processing systems, 7 (pp. 435-442). San Mateo, CA: Morgan Kaufmann.
43. Schwarz, R., & Chow, Y.-L. (1990). The N-best algorithm: An efficient and exact procedure for finding the N most likely hypotheses. In Proceedings of ICASSP'90 (pp. 81-84).
44. Senior, A., & Robinson, T. (1996). Forward-backward retraining of recurrent neural networks. In D. Touretzky, M. Mozer, & M. Hasselmo (Eds.), Advances in neural information processing dystems, 8 (pp. 743-749). San Mateo, CA: Morgan Kaufmann.
45. Smyth, P., Heckerman, D., & Jordan, M. I. (1997). Probabilistic independence networks for hidden Markov probability models. Neural Computation, 9, 227-269.
46. Valtchev, V., Kapadia, S., & Young, S. (1993). Recurrent input transformations for hidden Markov models. In Proceedings of ICASSP'93 (pp. 287-290).