Toward the border between neural and Markovian paradigms

IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics

Volumn 28, Issue 2, 1998, Pages 146-159

  Wiliñski, Piotr ^b   Solaiman, Basel ^a,b   Hillion, Alain ^a,b   Czarnecki, Witold ^c  

^a IEEE

^b ECOLE DES MINES DE NANTES   (France)

^c WARSAW UNIVERSITY OF TECHNOLOGY   (Poland)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 0001512406     PISSN: 10834419     EISSN: None     Source Type: Journal    
DOI: 10.1109/3477.662756     Document Type: Article

References (67)

1. J. S. Bridle, "Alpha-nets: A recurrent 'neural' network architecture with hidden Markov model interpretation," Speech Commun., vol. 9, pp. 83-92, Feb. 1990.
2. L. T. Niles and H. F. Silverman, "Combining hidden Markov models and neural network classifiers," in Proc. IEEE ICASSP '90, Albuquerque, NM, vol. 1, pp. 417-420.
3. H. Bourlard and C. J. Wellekens, "Speech pattern discrimination and multilayer perceptrons," Comput. Speech Lang. 3, pp. 1-19, 1989.
4. R. P. Lippmann, "An introduction to computing with neural nets," IEEE ASSP Mag., pp. 4-22, Apr. 1987.
5. B. Widrowand and M. A. Lehr, "30 years of adaptative neural networks: Perceptron, Madeline, and backpropagation," Proc. IEEE, vol. 78, pp. 1415-1442, Sept. 1990.
6. S-I. Amari, "Mathematical foundations of neurocomputing," Proc. IEEE, vol. 78, pp. 1443-1463, Sept. 1990.
7. L. R. Rabiner, "A tutorial on hidden Markov models and selected applications in speech recognition," Proc. IEEE, vol. 77, Feb. 1989.
8. L. E. Baum and T. Petrie, "Statistical inference for probabilistic functions of finite state Markov chains," Ann. Math. Stat., vol. 37, pp. 1554-1563, 1966.
9. A. J. Viterbi, "Error bounds for convolutional codes and on asymptotically optimal decoding algorithm," IEEE Trans. Inform. Theory, vol. IT-13, pp. 260-269, Apr. 1967.
10. A. P. Dempster, L. A. Laird, and D. B. Rubin, "Maximum likelihood from incomplete data via the EM algorithm," J. Roy. Stat. Soc., vol. 39, no. 1, pp. 1-38, 1977.
11. D. S. Geman "Gibbs fields, simulated annealing, and Bayesian reconstruction of images," IEEE Trans. Pattern Anal. Machine Intell., vol. PAMI-6, pp. 721-741, 1984.
12. N. Metropolis, A. W. Rosenblath, M. N. Rosenblath, A. H. Teller, and E. Teller, "Equation of state calculations by fast computing machines," J. Chem. Phys., vol. 21, pp. 1087-1091, 1953.
13. R. Dubes and A. Jain, "Random field models in image analysis," J. Appl. Stat., pp. 131-164, 1989.
14. A. H. Waibel, T. Hanazawa, G. Hinton, K. Shikano, and K. Lang, "Phoneme recognition using time-delay neural networks," IEEE Trans. Acoust., Speech, Signal Processing, vol. 37, no. 3, pp. 328-339, 1988.
15. J. L. Elman, "Finding structure in time," CRL_TR_8801, Univ. Calif., San Diego, 1988.
16. H. Sakoe, R. Isotani, K. Yoshida, K. Iso, and T. Watanabe, "Speaker-independent word recognition using dynamic programming neural networks," in Proc. IEEE ICASSP '89, New York, pp. 107-110.
17. K. Iso and T. Watanabe, "Speaker independent word recognition using a neural prediction model," in Proc. IEEE ICASSP '90, Albuquerque, NM, pp. 661-664.
18. J. Tebelskis and A. Waibel, "Large vocabulary recognition using linked predictive neural networks," in Proc. IEEE ICASSP '90, Albuquerque, NM, pp. 437-440.
19. T. Poggio and F. Girosi, "Networks for approximation and learning," Proc. IEEE, vol. 78, pp. 1481-1497, Sept. 1990.
20. H. Bourlard and C. J. Wellekens, "Links between Markov models and multilayer perceptrons," in Advances in Neural Information Processing Systems 1, D. S. Touretzky, Ed. Los Altos, CA: Morgan Kaufmann, 1988, pp. 502-510.
21. K. Torkolla, J. Kangas, P. Utela, S. Kaski, M. Kokkonen, and T. Koonen, "Status report of the finish phonetic typewriter project," in Artificial Neural Networks, T. Kohonen, K. Makisara, O. Simula, and Y. Kangas, Eds. Amsterdam, The Netherlands: Elsevier, 1991.
22. G. J. Clary and J. H. L. Hansen, "Feature enhancement for multi-layer perceptron and semi-continous hidden Markov model based classifiers using neural networks," in Proc. SPIE Neural Stochast. Meth. Image Signal Process., 1992, vol. 1766, pp. 529-540.
23. Y. M. Cheng, D. O'Shaughnessy, V. Gupta, P. Kenny, M. Lennig, P. Mermelstein, and S. Parthasarthy, "Hybrid segmental-LVQ/HMM for large vocabulary speech recognition," in Proc. IEEE ICASSP '92, pp. 593-597.
24. I. L. Kim and H. Y. Yang, "Efficient labeling based on Markov random field and error backpropagation network," Pattern Recognit., vol. 26, pp. 1695-1707, Nov. 1993.
25. J-N. Hwang, E. T. Y. Chen, and A. F. Lippmann, "Probabilistic image modeling via neural networks," in Proc. IEEE ICASSP '93, vol. 1, pp 585-588.
26. J.-N. Hwang, E. T. Y. Chen, and A. F. Lippmann, "Gibbs sampling via neural network probability estimation," in 26th Asilomar Conf. Signals, Systems, Computers, Monterey, CA, 1992.
27. S. Y. Kung, Digital Neural Networks. Englewood Cliffs, NJ: Prentice-Hall, 1993.
28. E. Levin, "Word recognition using hidden neural architecture," in Proc. IEEE ICASSP '90, Albuquerque, NM, pp. 433-436.
29. M. M. Hochberg, L. T. Niles, J. T. Foote, and H. F. Silverman, "Hidden Markov models/neural network training techniques for connected speech recognition," in Proc. IEEE ICASSP '91, Toronto, Ont., Canada, pp. 109-112.
30. S. J. Young, "Competitive training: A connexionist approach to the discriminative training of hidden Markov models," Proc. Inst. Elect. Eng., vol. 138, pp. 61-68, 1991.
31. F. J. Pineda, "Generalization of back-propoagation to recurrent neural networks," Phys. Rev. Lett., vol. 59, no. 19, pp. 2229-2232, 1987.
32. P. J. Werbos, "Backpropagation through time: What it does and how to do it," Proc. IEEE, vol. 78, Oct. 1990.
33. R. L. Watrous and L. Shastri, "Learning phonetic features using connectionist networks: An experiment in speech recognition," 1st Int. Conf. Neural Networks, San Diego, CA, 1987, pp. IV-381-IV-388.
34. A. Waibel, H. Sawai, and K. Shikano, "Modularity and scaling in large phonemic neural networks," IEEE Trans. Acoust., Speech, Signal Processing, vol. 37, Dec. 1989.
35. W. Ma and D. Van Comperolle, "TDNN labeling for a HMM recognizer," in Proc. IEEE ICASSP '90, Albuquerque, NM, pp. 421-424.
36. C. Dugast and L. Devillers, "Incorporating acoustic-phonetic knowledge in hybrid TDNN/HMM frameworks," in Proc. IEEE ICASSP '92, vol. 1, pp. 421-424.
37. E. Tsuboka, Y. Takada, and H. Wakita, "Neural predictive model," in Proc. IEEE ICASSP '91, Toronto, Ont., Canada, pp. 373-377.
38. B. Petek and J. Tebelskis, "Context-dependent hidden control neural networks architecture for continuous speech recognition," in Proc. IEEE ICASSP '92, vol. 1, pp. 397-400.
39. E. Gelenbe, "Random neural networks with negative and positive signals and product form solution," Neural Comput., vol. 1, no. 4, 1989.
40. N. Morgan and H. Bourlard, "Continous speech recognition using multilayer perceptrons with hidden Markov models," in Proc. IEEE ICASSP '90, Albuquerque, NM, vol. 2, pp. 413-416.
41. J. Makhoul, "Pattern recognition properties of neural networks," IEEE-SP Workshop Neural Networks Signal Processing, Princeton, NJ, 1991, pp. 173-186.
42. Y. Bengio, R. De Mori, G. Flammia, and R. Kompe, "Global optimization of a neural network-hidden Markov model hybrid," IEEE Trans. Neural Networks, vol. 3, Mar. 1992.
43. G. Rigoll, "Unsupervised information theory-based training algorithms for multilayer neural networks," in Proc. IEEE ICASSP '92, vol. 1, pp. 393-396.
44. Y. Bengio, R. Cardin, R. De Mori, and Y. Normandin, "A hybrid coder for hidden Markov models using a recurrent neural network," in Proc. IEEE ICASSP '90, Albuquerque, NM, pp. 537-540.
45. P. LeCerf and D. Van Compernolle, "Using parallel MLP's as labelers for multiple codebook HMM's," in Proc. IEEE ICASSP '93, vol. 1, pp. 561-564.
46. H. Iwamida, S. Katargi, and E. McDermott, "Speaker-independent large vocabulary word recognition using LVQ/HMM hybrid algorithm," in Proc. IEEE ICASSP '91, Toronto, Ont., Canada, pp. 553-556.
47. O. Schmidbauer and J. Tebelskis, "An LVQ based reference model for speaker-adaptive speech recognition," in Proc. IEEE ICASSP '92, vol. 1, pp. 441-444.
48. D. G. Kumber, M. A. Bush, and G. N. Tajchman, "Speaker independent vowel classification using hidden Markov models and LVQ2," Proc. IEEE ICASSP '90, Albuquerque, NM, pp. 497-500.
49. J. Guo and H. C. Lui, "A multilayer perceptron postprocessor to hidden markov modeling for speech recognition," in Proc. IEEE ICASSP '93, vol. 2, pp. 263-266.
50. R. P. Lippmann and E. Singer, "Hybrid neural-network/HMM approaches to wordspotting," in Proc. IEEE ICASSP '93, vol. 1, pp. 565-568.
51. S. Austin, G. Zavaliagkos, J. Makhoul, and R. Schwartz, "Speech recognition using segmental neural nets," in Proc. IEEE ICASSP '92, vol. 1, pp. 625-628.
52. B. Benmiloud, A. Peng, and W. Pieczynski, "Estimation conditionnelle itérative dans les chaînes de Markov cachées et segmentation statistique non supérvisée," in 14ème coloque GRETSI, Juan-les-pins, Sept. 13-16, 1993 (in French).
53. J-L. Chen and A. Kundu, "Automatic unsupervised texture segmentation using hidden Markov model," in Proc. IEEE ICASSP '93, vol. 5, pp. 21-24.
54. Y. He and A. Kundu, "2-D shape classification using hidden Markov model," IEEE Trans. Pattern Anal. Machine Intell., vol. 13, Nov. 1991.
55. P. Y. Mundkur, S. Kapoor, and U. B. Desai, "Boltzmann machines for depth recovery using a MRF model," in Int. Joint Conf. Neural Neworks, New York, June 7-11, 1992.
56. R. Azencott, "Synchronous Boltzmann machines and Gibbs fields: Learning algorithms," Neurocomput., F. Fogelman Soulie and J. Herault, Eds. Berlin, Germany: Springer-Verlag, 1990.
57. M. T. Figueiredo and J. M. N. Leitao, "Image restoration using neural networks," in Proc. IEEE ICASSP '92, vol. 2, pp. 409-412.
58. R. Durbin and D. E. Rumelhart, "Product units - A computationally powerful and biological plausible extension to backpropagation networks," Neural Comp., vol. 1, pp. 133-142, 1989.
59. E. Levin, N. Tishby, and S. A. Solla, "A statistical Approach to Learning and generalization in layered neural networks" Proc. IEEE, vol. 78, pp. 1568-1574, Oct. 1990.
60. Y. Ephraim, A. Dembo, and R. L. Rabiner, "A minimum discrimination information approach for hidden Markov modeling," in IEEE ICASSP '87, Dallas, TX, pp. 25-28.
61. H. Gish, "A probabilistic approach to the understanding and training of neural network classifiers," Proc. IEEE ICASSP '90, Abuquerque, NM, pp. 1361-1364.
62. J. J. Hopfield, "Neural networks and physical systems with emergent collective computational abilities," in Proc. Nat. Acad. Sci., Apr. 1982, vol. 79, pp. 2553-2558.
63. C. Peterson and E. Harlman, "Explorations of the mean field theory learning algorithm," Neural Networks, pp. 475-494, 1989.
64. S. J. Young, "Competitive training in hidden Markov models," in Proc. IEEE ICASSP '90, Albuquerque, NM, pp. 681-684.
65. R. P. Lippmann and B. Gold, "Neural classifiers useful for speech recognition," in IEEE 1st Int. Conf. Neural Networks, June 1987, vol. 4, pp. 417-425.
66. M. Franzini, K.-F. Lee, and A. Waibel, "Connexionist Viterbi training: A new hybrid method for continous speech recognition," in Proc. IEEE ICASSP '90, Albuquerque, NM, pp. 425-428.
67. M. A. Franzini, M. J. Witbrock, and K.-F. Lee, "A connectionist approach to continous speech recognition," in Proc. IEEE ICASSP '89, New York, pp. 425-428.