Real-time robust automatic speech recognition using compact support vector machines

IEEE Transactions on Audio, Speech and Language Processing

Volumn 20, Issue 4, 2012, Pages 1347-1361

  Solera Ureña, Rubén ^a   García Moral, Ana Isabel ^b   Peláez Moreno, Carmen ^a   Martínez Ramon, Manel ^a   Díaz De María, Fernando ^a  

^a UNIVERSIDAD CARLOS III DE MADRID   (Spain)

^b Fonetic Solutions S L   (Spain)

Author keywords

Additive noise; artificial neural network (ANN) hidden Markov model (HMM); compact support vector machine (SVM); hybrid automatic speech recognition (ASR); machine learning; real time ASR; robust ASR; SVM HMM

Indexed keywords

ARTIFICIAL NEURAL NETWORK; AUTOMATIC SPEECH RECOGNITION; COMPACT SUPPORT; MACHINE-LEARNING; REAL-TIME ASR; ROBUST ASR; SVM/HMM;

ADDITIVE NOISE; HIDDEN MARKOV MODELS; NEURAL NETWORKS; SPEECH PROCESSING; SPEECH RECOGNITION;

SUPPORT VECTOR MACHINES;

EID: 84857488250     PISSN: 15587916     EISSN: None     Source Type: Journal    
DOI: 10.1109/TASL.2011.2178597     Document Type: Article

References (83)

1. H. Bourlard and N. Morgan, Connectionist Speech Recognition: A Hybrid Approach. Norwell, MA: Kluwer, 1993.
2. N. Morgan and H. Bourlard, "Continuous speech recognition: An introduction to the hybrid HMM/connectionist approach," IEEE Signal Process. Mag., vol. 12, no. 3, pp. 24-42, May 1995.
3. E. Trentin and M. Gori, "A survey of hybrid ANN/HMM models for automatic speech recognition," Neurocomputing, vol. 37, no. 1-4, pp. 91-126, Apr. 2001.
4. P. Pujol, S. Pol, C. Nadeu, A. Hagen, and H. Bourlard, "Comparison and combinationoffeaturesinahybrid HMM/MLPandaHMM/GMM speech recognition system," IEEE Trans. Speech Audio Process., vol. 13, no. 1, pp. 14-22, Jan. 2005. (Pubitemid 40049936)
5. A. I. García-Moral, R. Solera-Ureña, C. Peláez- Moreno, and F. Díaz-de-María, "Data balancing for efficient training of hybrid ANN/HMM automatic speech recognition systems," IEEE Trans. Audio, Speech, Lang. Process., vol. 19, no. 3, pp. 468-481, Mar. 2011.
6. B. Schölkopf and A. J. Smola, Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond. Cambridge, MA: MIT Press, 2001.
7. H. Shimodaira, K. Noma, M. Nakai, and S. Sagayama, "Support vector machine with dynamic time-alignment kernel for speech recognition," in Proc. 7th Eur. Conf. Speech Commun. Technol. (Eurospeech 2001), Aalborg, Denmark, Sep. 2001, pp. 1841-1844.
8. N. D. Smith and M. Niranjan, "Data-dependent kernels in SVM classification of speech patterns," in Proc. 6th Int. Conf. Spoken Language Process. (ICSLP 2000), Beijing, China, Oct. 2000, vol. 1, pp. 297-300.
9. A. Ganapathiraju, J. E. Hamaker, and J. Picone, "Hybrid SVM/HMM architectures for speech recognition," in Proc. 6th Int. Conf. Spoken Language Process. (ICSLP 2000), Beijing, China, Oct. 2000, vol. 4, pp. 504-507.
10. S. E. Krüger, M. Schafföner, M. Katz, E. Andelic, and A. Wendemuth, "Speech recognition with support vector machines in a hybrid system," in Proc. 9th Eur. Conf. Speech Commun. Technol. (Interspeech 2005), Lisbon, Portugal, Sep. 2005, pp. 993-996. (Pubitemid 43908231)
11. J. Padrell-Sendra, D. Martín-Iglesias, and F. Díaz-de- María, "Support vector machines for continuous speech recognition," in Proc. 14th Eur. Signal Process. Conf. (EUSIPCO 2006), Florence, Italy, Sep. 2006.
12. D. Bolaños-Alonso, "Advances in the application of support vector machines as probabilistic estimators for continuous automatic speech recognition," Ph.D. dissertation, Univ. Autónoma de Madrid, Escuela Politécnica Superior, Madrid, Spain, 2008.
13. M. J. F. Gales and F. Flego, "Discriminative classifiers with adaptive kernels for noise robust speech recognition," Comput. Speech Lang., vol. 24, no. 4, pp. 648-662, Oct. 2010.
14. F. Pérez-Cruz, A. Navia-Vázquez, J. L. Rojo-Álvarez, and A. Artés-Rodríguez, "A new training algorithm for support vector machines," in Proc. 5th Bayona Workshop Emerging Technol. in Telecommun., Baiona, Spain, 1999, pp. 116-120.
15. F. Pérez-Cruz, "Máquina de vectores soporte adaptativa y compacta," Ph.D. dissertation, Univ. Politécnica de Madrid, Madrid, Spain, 2000.
16. A. Navia-Vázquez, F. Pérez-Cruz, A. Artés- Rodríguez, and A. R. Figueiras-Vidal, "Weighted least squares training of support vector classifiers leading to compact and adaptive schemes," IEEE Trans. Neural Netw., vol. 12, no. 5, pp. 1047-1059, Sep. 2001. (Pubitemid 32981747)
17. Y. Engel, S. Mannor, and R. Meir, "Sparse online greedy support vector regression," in Proc. Mach. Learn.: ECML'02, Berlin/Heidelberg, Germany, 2002, vol. 2430/2002, Lecture Notes in Computer Science (LNCS), pp. 84-96.
18. Y. Engel, S. Mannor, and R. Meir, "The Kernel recursive least-squares algorithm," IEEE Trans. Signal Process., vol. 52, no. 8, pp. 2275-2285, Aug. 2004.
19. S. Haykin, Neural Networks: A Comprehensive Foundation. Upper Saddle River, NJ: Prentice-Hall, 1998.
20. H. Bourlard and N. Morgan, "Hybrid HMM/ANN systems for speech recognition: Overview and new research directions," in Adaptive Processing of Sequences and Data Structures. Berlin/Heidelberg, Germany: Springer-Verlag, 1998, vol. 1387/1998, Lecture Notes in Computer Science (LNCS), pp. 389-417. (Pubitemid 128056038)
21. J. P. Neto, C. Martins, and L. B. Almeida, "An incremental speaker-adaptation technique for hybrid HMM-MLP recognizer," in Proc. Fourth Int. Conf. Spoken Lang. Process. (ICSLP 1996), Philadelphia, PA, Oct. 1996, vol. 3, pp. 1293-1296.
22. J. P. Neto, C. Martins, and L. B. Almeida, "Speaker-adaptation in a hybrid HMM-MLP recognizer," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP 1996), Washington, DC, May 1996, vol. 6,pp. 3382-3385.
23. R. Gemello, F. Mana, S. Scanzio, P. Laface, and R. De Mori, "Linear hidden transformations for adaptation of hybrid ANN/HMM models," Speech Commun., vol. 49, no. 10-11, pp. 827-835, Oct. 2007. (Pubitemid 47277472)
24. S. Thomas, S. Ganapathy, and H. Hermansky, "Spectro-temporal features for automatic speech recognition using linear prediction in spectral domain," in Proc. 16th Eur. Signal Process. Conf. (EUSIPCO 2008), Lausanne, Switzerland, Aug. 2008.
25. S. Ganapathy, S. Thomas, and H. Hermansky, "Modulation frequency features for phoneme recognition in noisy speech," J. Acoust. Soc. Amer., vol. 125, no. 1, pp. EL8-EL12, Jan. 2009.
26. J. Frankel and S. King, "A hybrid ANN/DBN approach to articulatory feature recognition," in Proc. 9th Int. Conf. Speech Commun. Technol. (Interspeech 2005), Lisbon, Portugal, Sep. 2005, pp. 3045-3048.
27. J. Frankel, M. Wester, and S. King, "Articulatory feature recognition using dynamic Bayesian networks," Comput. Speech Lang., vol. 21, no. 4, pp. 620-640, Oct. 2007. (Pubitemid 46880190)
28. S. King, J. Frankel, K. Livescu, E. McDermott, K. Richmond, and M. Wester, "Speech production knowledge in automatic speech recognition," J. Acoust. Soc. Amer., vol. 121, no. 2, pp. 723-742, Feb. 2007. (Pubitemid 46192674)
29. N. Morgan, B. Y. Chen, Q. Zhu, and A. Stolcke, "Scaling up: Learning large-scale recognition methods from small-scale recognition tasks," in Proc. Special Workshop (SWIM), Maui, HI, Jan. 2004.
30. S. Y. Zhao, S. Ravuri, and N. Morgan, "Multi-stream to many-stream: Using spectro-temporal features for ASR," in Proc. 10th Annu. Conf. Int. Speech Commun. Assoc. (Interspeech 2009), Brighton, U.K., Sep. 2009, pp. 2951-2954.
31. A. Abad and J. Neto, "Incorporating acoustical modelling of phone transitions in an hybrid ANN/HMM speech recognizer," in Proc. 9th Annu. Conf. Int. Speech Commun. Assoc. (Interspeech '08), Brisbane, Australia, Sep. 2008, pp. 2394-2397.
32. A. Abad, T. Pellegrini, I. Trancoso, and J. Neto, "Context dependent modelling approaches for hybrid speech recognizers," in Proc. 11th Annu. Conf. Int. Speech Commun. Assoc. (Interspeech 2010), Chiba, Japan, Sep. 2010, pp. 2950-2953.
33. R. Solera-Ureña, D. Martín-Iglesias, A. Gallardo- Antolín, C. Peláez-Moreno, and F. Díaz-de-María, "Robust ASR using support vector machines," Speech Commun., vol. 49, no. 4, pp. 253-267, Apr. 2007. (Pubitemid 46517709)
34. R. Solera-Ureña, J. Padrell-Sendra, D. Martín-Iglesias, A. Gal-lardo-Antolín, C. Peláez-Moreno, and F. Díaz-de- María, "SVMs for automatic speech recognition: A survey," in Progress in Nonlinear Speech Processing. Berlin/Heidelberg, Germany: Springer-Verlag, 2007, vol. 4391/2007, Lecture Notes in Computer Science (LNCS), pp. 190-216. (Pubitemid 351256466)
35. A. Ech-Cherif, M. Kohili, A. Benyettou, and M. Benyettou, "La-grangian support vector machines for phoneme classification," in Proc. 9th Int. Conf. Neural Inf. Process. (ICONIP 2002), Singapore, Nov. 2002, vol. 5, pp. 2507-2511.
36. S. V. Gangashetty, C. Chandra-Sekhar, and B. Yegnanarayana, "Combining evidence from multiple classifiers for recognition of consonant-vowel units of speech in multiple languages," in Proc. Int. Conf. Intell. Sens. Inf. Process., Chennai, India, Dec. 2005, pp. 387-391.
37. D. Martín-Iglesias, J. Bernal-Chaves, C. Peláez-Moreno, A. Gallardo-Antolín, and F. Díaz-de-María, "A speech recognizer based on multi-class SVMs with HMM-guided segmentation," in Nonlinear Analyses and Algorithms for Speech Processing. Berlin/Heidelberg, Germany: Springer-Verlag, 2005, vol. 3817/2005, Lecture Notes in Computer Science (LNCS), pp. 256-266.
38. P. Clarkson and P. J. Moreno, "On the use of support vector machines for phonetic classification," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP 1999), Phoenix, AZ, Mar. 1999, vol. 2, pp. 585-588.
39. N. Thubthong and B. Kijsirikul, "Support vector machines for Thai phoneme recognition," Int. J. Uncertainty, Fuzziness, Knowl.-Based Syst., vol. 9, no. 6, pp. 803-813, 2001. (Pubitemid 33682991)
40. C. Chandra-Sekhar, W. F. Lee, K. Takeda, and F. Itakura, "Acoustic modeling of subword units using support vector machines," in Proc. Workshop Spoken Lang. Process. (WSLP 2003), Mumbai, India, Jan. 2003, pp. 79-86.
41. A. Ganapathiraju, "Support vector machines for speech recognition," Ph.D. dissertation, Mississipi State Univ., Mississippi State, MS, 2002.
42. A. Ganapathiraju, J. E. Hamaker, and J. Picone, "Applications of support vector machines to speech recognition," IEEE Trans. Signal Process., vol. 52, no. 8, pp. 2348-2355, Aug. 2004.
43. S. Fine, G. Saon, and R. A. Gopinath, "Digit recognition in noisy environments via a sequential GMM/SVM system," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP'02), Orlando, FL, May 2002, vol. 1, pp. 49-52.
44. M. Schafföner, S. E. Krüger, E. Andelic, M. Katz, and A. Wendemuth, "Limited training data robust speech recognition using kernel-based acoustic models," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP 2006), Toulouse, France, May 2006, vol. 1, pp. 1137-1140.
45. D. Bolaños and W. Ward, "Implicit state-tying for support vector machines based speech recognition," in Proc. 9th Annu. Conf. Int. Speech Commun. Assoc. (Interspeech '08), Brisbane, Australia, Sep. 2008, pp. 924-927.
46. B. E. Boser, I. M. Guyon, and V. N. Vapnik, "A training algorithm for optimal margin classifiers," in Proc. 5th Annu. Workshop Comput. Learn. Theory, Pittsburgh, PA, Jul. 1992, pp. 144-152.
47. V. N. Vapnik, Statistical Learning Theory. New York: Wiley-Inter-science, 1998.
48. J. Nocedal and S. J. Wright, Numerical Optimization. New York: Springer, 1999.
49. M. A. Aizerman, E. M. Braverman, and L. I. Rozonoer, "Theoretical foundations of the potential function method in pattern recognition learning," Autom. Remote Control, vol. 25, pp. 821-837, 1964.
50. J. Weston and C. Watkins, "Multi-Class Support Vector Machines," Tech. Rep. Univ. of London, Dept. of Comput. Sci., Royal Holloway, Egham, U.K., 1998.
51. K. Crammer and Y. Singer, "On the algorithmic implementation of multiclass kernel-based vector machines," J. Mach. Learn. Res., vol. 2, no. 5, pp. 265-292, 2001.
52. Y. Lee, Y. Lin, and G. Wahba, "Multicategory support vector machines: Theory and application to the classification of microarray data and satellite radiance data," J. Amer. Statist. Assoc., vol. 99, no. 465, pp. 67-81, Mar. 2004. (Pubitemid 38545195)
53. J. C. Platt, N. Cristianini, and J. Shawe-Taylor, "Large margin DAGs for multiclass classification," in Advances in Neural Information Processing Systems, S.A.Solla, T. K. Leen, and K.-R. Müller, Eds. Cambridge, MA: MIT Press, 2000, vol. 12, pp. 547-553.
54. T. G. Dietterich and G. Bakiri, "Solving multiclass learning problems via error-correcting output codes," J. Artif. Intell. Res., vol. 2, no. 1, pp. 263-286, 1995.
55. E. L. Allwein, R. E. Schapire, and Y. Singer, "Reducing multiclass to binary: A unifying approach for margin classifiers," J. Mach. Learn. Res., vol. 1, pp. 113-141, 2000. (Pubitemid 33738778)
56. C. W. Hsu and C.-J. Lin, "A comparison of methods for multiclass support vector machines," IEEE Trans. Neural Netw., vol. 13, no. 2, pp. 415-425, Mar. 2002. (Pubitemid 34475042)
57. C.-C. Chang and C.-J. Lin, "LIBSVM: A library for support vector machines," 2001, [Online]. Available: http://www.csie.ntu.edu.tw/~cjlin/ libsvm
58. J. C. Platt, "Probabilities for SV machines," in Advances in Large Margin Classifiers. Cambridge, MA: MIT Press, 2000, pp. 61-74.
59. T.-F. Wu, C.-J. Lin, and R. C. Weng, "Probability estimates for multi-class classification by pairwise coupling," J. Mach. Learn. Res., vol. 5, pp. 975-1005, 2004.
60. V. García, J. Sánchez, and R. Mollineda, "An empirical study of the behavior of classifiers on imbalanced and overlapped data sets," in Progress in Pattern Recognition, Image Analysis and Applications. Berlin/Heidelberg, Germany: Springer-Verlag, 2008, vol. 4756/2008, Lecture Notes in Computer Science (LNCS), pp. 397-406.
61. G. M. Weiss, B. Zadrozny, and M. Saar-Tsechansky, "Guest editorial: Special issue on utility-based data mining," Data Mining Knowl. Disc., vol. 17, no. 2, pp. 129-135, Oct. 2008.
62. X.-Y. Liu, J. Wu, and Z.-H. Zhou, "Exploratory under-sampling for class-imbalance learning," IEEE Trans. Syst., Man, Cybern. B: Cy-bern., vol. 39, no. 2, pp. 539-550, Apr. 2009.
63. L. Tóth and A. Kocsor, "Training HMM/ANN hybrid speech recognizers by probabilistic sampling," in Proc. Artif. Neural Netw.: Biol. Inspirations (ICANN 2005), Berlin/Heidelberg, Germany, 2005, vol. 3696/2005, Lecture Notes in Computer Science (LNCS), pp. 597-603.
64. S. Scanzio, P. Laface, R. Gemello, and F. Mana, "Speeding-up neural network training using sentence and frame selection," in Proc. 8th Annu. Conf. Int. Speech Commun. Assoc. (Interspeech '07), Antwerp, Belgium, Aug. 2007, pp. 1725-1728.
65. D. Albesano, R. Gemello, and F. Mana, "Hybrid HMM-NN for speech recognition and prior class probabilities," in Proc. 9th Int. Conf. Neural Inf. Process. (ICONIP '02), Singapore, Nov. 2002, vol. 5, pp. 2391-2395.
66. A. Hagen, "Robust speech recognition based on multi-stream processing," Ph.D. dissertation, École Polytechnique Fédérale de Lausanne, Lausanne, Switzerland, 2001.
67. F. Pérez-Cruz, A. Navia-Vázquez, P. L. Alarcón- Diana, and A. Artés-Rodríguez, "SVC-based equalizer for burst TDMA transmissions," Signal Process., vol. 81, no. 8, pp. 1681-1693, Aug. 2001. (Pubitemid 32694055)
68. F. Pérez-Cruz, C. Bousoño-Calzón, and A. Artés-Rodríguez, "Convergence of the IRWLS procedure tothe support vector machine solution," Neural Comput., vol. 17, no. 1, pp. 7-18, Jan. 2005. (Pubitemid 40305880)
69. H. W. Kuhn and A. W. Tucker, "Nonlinear programming," in Proc. 2nd Berkeley Symp. Math. Statist. Probab., Berkeley, CA, Jul.-Aug. 1950, pp. 481-492.
70. C. J. C. Burges, "A tutorial on support vector machines for pattern recognition," Data Mining Knowl. Disc., vol. 2, pp. 121-167, 1998. (Pubitemid 128695475)
71. A. N. Tikhonov and V. Y. Arsenin, Solution of Ill-Posed Problems. Washington, DC: W. H. Winston, 1977.
72. G. Kimeldorf and G. Wahba, "Some results on Tchebycheffian spline function," J. Math. Anal. Applicat., vol. 33, no. 1, pp. 82-95, Jan. 1971.
73. C. J. C. Burges and B. Schölkopf, "Improving the accuracy and speed of support vector machines," in Advances in Neural Information Processing Systems, M. C. Mozer, M. I. Jordan, and T. Petsche, Eds. Cambridge, MA: MIT Press, 1997, vol. 9, pp. 375-381.
74. S. E. Borys, "An SVM Front End Landmark Speech Recognition System," M.Sc., University of Illinois at Urbana-Champaign, Graduate College, Urbana, IL, USA, 2008.
75. F. Orabona, J. Keshet, and B. Caputo, "Bounded kernel-based online learning," J. Mach. Learn. Res., vol. 10, pp. 2643-2666, 2009.
76. A. Moreno, "SpeechDat Spanish database for fixed telephone network," Tech. Rep. Univ. Politecnica de Catalunya, Barcelona, Spain, 1997.
77. A. Varga and H. J. M. Steeneken, "Assessment for automatic speech recognition: II. NOISEX-92: A database and an experiment to study the effect of additive noise on speech recognition systems," Speech Commun., vol. 12, no. 3, pp. 247-251, 1993.
78. S. Young, G. Evermann, M. Gales, T. Hain, and D. Kershaw, HTK-Hidden Markov Model Toolkit (ver. 3.4). Cambridge, U.K.: Cambridge Univ., 2006.
79. F. T. Johansen, N. Warakagoda, B. Lindberg, G. Lehtinen, Z. Kacic, A. Zgank, K. Elenius, and G. Salvi, in COST 249 MCM, Tech. Annex, 1999.
80. B. Kotnik, Z. Kacic, and B. Horvat, "Development and integration of the LDA-toolkit into the COST249 SpeechDat (II) SIG reference recognizer," in Proc. 4th Int. Conf. Lang. Resources Eval. (LREC '04), Lisbon, Portugal, May 2004.
81. Y. Altun, I. Tsochantaridis, and T. Hofmann, "Hidden Markov support vector machines," in Proc. 20th Int. Conf. Mach. Learn. (ICML '03), Washington, DC, Aug. 2003, pp. 3-10.
82. B. Taskar, C. Guestrin, andD. Koller, "Max-margin Markov networks," in Advances in Neural Information Processing Systems, S. Thrun, L. Saul, and B. Schölkopf, Eds. Cambridge, MA: MIT Press, 2004, vol. 16.
83. F. Pérez-Cruz, Z. Ghahramani, and M. Pontil, "Kernel conditional graphical models," in Predicting Structured Data, F. Bakir, T. Hofmann, B. Schölkopf, A. J. Smola, B. Taskar, and S. V. N. Vish-wanathan, Eds. Cambridge, MA: MIT Press, 2007, pp. 265-282.