Noisy hidden Markov models for speech recognition

Proceedings of the International Joint Conference on Neural Networks

Volumn , Issue , 2013, Pages

  Audhkhasi, Kartik ^a   Osoba, Osonde ^a   Kosko, Bart ^a  

^a University of Southern California   (United States)

Author keywords

Expectation Maximization algorithm; Hidden Markov model; noise injection; noisy EM algorithm; speech recognition; stochastic resonance

Indexed keywords

EM ALGORITHMS; EXPECTATION MAXIMIZATION; EXPECTATION-MAXIMIZATION ALGORITHMS; GAUSSIAN MIXTURE MODEL; HIDDEN MARKOV MODELS (HMMS); MAXIMUM-LIKELIHOOD ESTIMATE; NOISE INJECTION; STOCHASTIC RESONANCES;

HIDDEN MARKOV MODELS; IMAGE SEGMENTATION; MAGNETIC RESONANCE; MAXIMUM PRINCIPLE; NEURAL NETWORKS; SPEECH RECOGNITION;

ALGORITHMS;

EID: 84893549944     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/IJCNN.2013.6707088     Document Type: Conference Paper

References (52)

1. O. Osoba, S. Mitaim, and B. Kosko, "Noise Benefits in the ExpectationMaximization Algorithm: NEM theorems and Models, " in The International Joint Conference on Neural Networks (IJCNN). IEEE, 2011, pp. 3178-3183.
2. O. Osoba, S. Mitaim, and B. Kosko, "The Noisy Expectation Maximization Algorithm, " in review, 2012.
3. L.E. Baum, T. Petrie, G. Soules, and N. Weiss, "A maximization technique occurring in the statistical analysis of probabilistic functions of Markov chains, " The Annals of Mathematical Statistics, pp. 164-171, 1970.
4. B. Kosko, Noise, Viking, 2006.
5. A. Patel and B. Kosko, "Levy Noise Benefits in Neural Signal Detection, " in Acoustics, Speech and Signal Processing, 2007. ICASSP 2007. IEEE International Conference on, 2007, vol. 3, pp. III1413-III1416.
6. A. Patel and B. Kosko, "Stochastic Resonance in Continuous and Spiking Neurons with Levy Noise, " IEEE Transactions on Neural Networks, vol. 19, no. 12, pp. 1993-2008, December 2008.
7. M. Wilde and B. Kosko, "Quantum forbiddeninterval theorems for stochastic resonance, " Journal of Physical A: Mathematical Theory, vol. 42, no. 46, 2009.
8. A. Patel and B. Kosko, "Errorprobability noise benefits in threshold neural signal detection, " Neural Networks, vol. 22, no. 5, pp. 697-706, 2009.
9. A. Patel and B. Kosko, "Optimal MeanSquare Noise Benefits in QuantizerArray Linear Estimation, " IEEE Signal Processing Letters, vol. 17, no. 12, pp. 1005-1009, Dec. 2010.
10. A. Patel and B. Kosko, "Noise Benefits in QuantizerArray Correlation Detection and Watermark Decoding, " IEEE Transactions on Signal Processing, vol. 59, no. 2, pp. 488-505, Feb. 2011.
11. B. Franzke and B. Kosko, "Noise Can Speed Convergence in Markov Chains, " Physical Review E, vol. 84, no. 4, pp. 041112, 2011.
12. AR Bulsara, RD Boss, and EW Jacobs, "Noise effects in an electronic model of a single neuron, " Biological cybernetics, vol. 61, no. 3, pp. 211-222, 1989.
13. L. Gammaitoni, P. Hanggi, P. Jung, and F. Marchesoni, "Stochastic resonance, " Reviews of Modern Physics, vol. 70, no. 1, pp. 223, 1998.
14. R. Benzi, A. Sutera, and A. Vulpiani, "The mechanism of stochastic resonance, " Journal of Physics A: mathematical and general, vol. 14, no. 11, pp. L453, 1999.
15. R. K. Adair, R. D. Astumian, and J. C. Weaver, "Detection of Weak Electric Fields by Sharks, Rays and Skates, " Chaos: Focus Issue on the Constructive Role of Noise in Fluctuation Driven Transport and Stochastic Resonance, vol. 8, no. 3, pp. 576-587, 1998.
16. T. R. Albert, A. R. Bulsara, G. Schmera, and M. Inchiosa, "An Evaluation of the Stochastic Resonance Phenomenon as a Potential Tool for Signal Processing, " in Conference Record of the Twenty Seventh Asilomar Conference on Signals, Systems & Computers, November 1993, vol. 1, pp. 583-587.
17. V. S. Anishchenko and T. Kapitaniak, "Chaotic Resonance: Birth of DoubleDouble Scroll Attractor, " in AIP Conference Proceedings 375: Chaotic, Fractal, and Nonlinear Signal Processing, 1995, R. A. Katz, Ed., 1996, pp. 420-428.
18. F. Apostolico, L. Gammaitoni, Marchesoni, and S. Santucci, "Resonant Trapping: A Failure Mechanism in Switch Transitions, " Physical Review E, vol. 55, no. 1, pp. 36-39, January 1997.
19. A. S. Asdi and A. H. Tewfik, "Detection of Weak Signals Using Adaptive Stochastic Resonance, " in Proceedings of the 1995 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP95), May 1995, vol. 2, pp. 1332-1335.
20. R. Benzi, G. Parisi, A. Sutera, and A. Vulpiani, "A Theory of Stochastic Resonance in Climatic Change, " SIAM Journal on Applied Mathematics, vol. 43, no. 3, pp. 565-578, June 1983.
21. R. Benzi, A. Sutera, and A. Vulpiani, "The Mechanism of Stochastic Resonance, " Journal of Physics A: Mathematical and General, vol. 14, pp. L453-L457, 1981.
22. A. Krogh, M. Brown, I.S. Mian, K. Sjolander, and D. Haussler, "Hidden Markov models in computational biology: Applications to protein modeling, " Journal of molecular biology, vol. 235, no. 5, pp. 1501-1531, 1994.
23. S.R. Eddy et al., "Multiple alignment using hidden Markov models, " in Proc. ISMB, 1995, vol. 3, pp. 114-120.
24. L.R. Rabiner, "A tutorial on hidden Markov models and selected applications in speech recognition, " Proceedings of the IEEE, vol. 77, no. 2, pp. 257-286, 1989.
25. S.E. Levinson, "Continuously variable duration hidden Markov models for automatic speech recognition, " Computer Speech And Language, vol. 1, no. 1, pp. 29-45, 1986.
26. J.G. Wilpon, L.R. Rabiner, C.H. Lee, and ER Goldman, "Automatic recognition of keywords in unconstrained speech using hidden Markov models, " IEEE Transactions on Acoustics, Speech and Signal Processing, vol. 38, no. 11, pp. 1870-1878, 1990.
27. S.R. Eddy, "Profile hidden Markov models., " Bioinformatics, vol. 14, no. 9, pp. 755-763, 1998.
28. K. Karplus, C. Barrett, and R. Hughey, "Hidden Markov models for detecting remote protein homologies., " Bioinformatics, vol. 14, no. 10, pp. 846-856, 1998.
29. J. Yamato, J. Ohya, and K. Ishii, "Recognizing human action in timesequential images using hidden Markov model, " in Proc. CVPR. IEEE, 1992, pp. 379-385.
30. M. Brand, N. Oliver, and A. Pentland, "Coupled hidden Markov models for complex action recognition, " in Proc. CVPR. IEEE, 1997, pp. 994-999.
31. M.S. Crouse, R.D. Nowak, and R.G. Baraniuk, "Waveletbased statistical signal processing using hidden Markov models, " IEEE Transactions on Signal Processing, vol. 46, no. 4, pp. 886-902, 1998.
32. R.J. Elliott, L. Aggoun, and J.B. Moore, Hidden Markov models: Estimation and Control, vol. 29, Springer, 1994.
33. S. Young, G. Evermann, D. Kershaw, G. Moore, J. Odell, D. Ollason, V. Valtchev, and P.Woodland, "The HTK book, " Cambridge University Engineering Department, vol. 3, 2002.
34. W. Walker, P. Lamere, P. Kwok, B. Raj, R. Singh, E. Gouvea, P. Wolf, and J. Woelfel, "Sphinx4: A flexible open source framework for speech recognition, " 2004.
35. B. Pellom and K. Hacioglu, "Recent improvements in the CU SONIC ASR system for noisy speech: The SPINE task, " in Proc. ICASSP. IEEE, 2003, vol. 1, pp. 14.
36. D. Rybach, C. Gollan, G. Heigold, B. Hoffmeister, J. Loof, R. Schluter, and H. Ney, "The RWTH Aachen University open source speech recognition system, " in Proc. Interspeech, 2009, pp. 2111-2114.
37. D. Povey, A. Ghoshal, G. Boulianne, L. Burget, O. Glembek, N. Goel, M. Hannemann, P. Motlicek, Y. Qian, P. Schwarz, et al., "The Kaldi speech recognition toolkit, " in Proc. ASRU, 2011.
38. H. Soltau, G. Saon, and B. Kingsbury, "The IBM Attila speech recognition toolkit, " in Proc. SLT. IEEE, 2010, pp. 97-102.
39. Y. Chow, M. Dunham, O. Kimball, M. Krasner, G. Kubala, J. Makhoul, P. Price, S. Roucos, and R. Schwartz, "BYBLOS: The BBN continuous speech recognition system, " in Proc. ICASSP. IEEE, 1987, vol. 12, pp. 89-92.
40. V. Goffin, C. Allauzen, E. Bocchieri, D. HakkaniTur, A. Ljolje, S. Parthasarathy, M. Rahim, G. Riccardi, and M. Saraclar, "The AT&T Watson speech recognizer, " in Proc. ICASSP, 2005, pp. 1033-1036.
41. J. S. Bridle, "AlphaNets: A recurrent neural network architecture with a hidden Markov model interpretation, " Speech Communication, vol. 9, no. 1, pp. 83-92, 1990.
42. Geoffrey Hinton, Li Deng, Dong Yu, George Dahl, AbdelRahman Mohamed, Navdeep Jaitly, Andrew Senior, Vincent Vanhoucke, Patrick Nguyen, Tara Sainath, et al., "Deep neural networks for acoustic modeling in speech recognition, " IEEE Signal Processing Magazine, 2012.
43. T.N. Sainath, B. Kingsbury, B. Ramabhadran, P. Fousek, P. Novak, and A.R. Mohamed, "Making deep belief networks effective for large vocabulary continuous speech recognition, " in Proc. ASRU. IEEE, 2011, pp. 30-35.
44. A. Mohamed, G. Dahl, and G. Hinton, "Deep belief networks for phone recognition, " in Proc. NIPS Workshop on Deep Learning for Speech Recognition and Related Applications, 2009.
45. A. Mohamed, G.E. Dahl, and G. Hinton, "Acoustic modeling using deep belief networks, " IEEE Transactions on Audio, Speech, and Language Processing, vol. 20, no. 1, pp. 14-22, 2012.
46. A.R. Mohamed, T.N. Sainath, G. Dahl, B. Ramabhadran, G.E. Hinton, and M.A. Picheny, "Deep belief networks using discriminative features for phone recognition, " in Acoustics, Speech and Signal Processing (ICASSP), 2011 IEEE International Conference on. IEEE, 2011, pp. 5060-5063.
47. L. Rabiner and B.H. Juang, "Fundamentals of speech recognition, " 1993.
48. S. Boyd and L. Vandenberghe, Convex optimization, Cambridge university press, 2004.
49. H. Teicher, "On the mixture of distributions, " The Annals of Mathematical Statistics, pp. 55-73, 1960.
50. H. Teicher, "Identifiability of finite mixtures, " The Annals of Mathematical Statistics, vol. 34, no. 4, pp. 1265-1269, 1963.
51. J.S. Garofolo, TIMIT: Acousticphonetic Continuous Speech Corpus, Linguistic Data Consortium, 1993.
52. A.K. Halberstadt and J.R. Glass, "Heterogeneous acoustic measurements for phonetic classification, " in Proc. Eurospeech, 1997, vol. 97, pp. 401-404.