Component-based discriminative classification for hidden Markov models

Pattern Recognition

Volumn 42, Issue 11, 2009, Pages 2637-2648

  Bicego, Manuele ^a,b   Pȩkalska, Elzbieta ^c   Tax, David M J ^d   Duin, Robert P W ^d  

^a UNIVERSITY OF VERONA   (Italy)

^b UNIVERSITY OF SASSARI   (Italy)

^c UNIVERSITY OF MANCHESTER   (United Kingdom)

^d DELFT UNIVERSITY OF TECHNOLOGY   (Netherlands)

Author keywords

Dimensionality reduction; Discriminative classification; Generative embeddings; Hidden Markov models; Hybrid models

Indexed keywords

COMPONENT BASED; DIMENSIONALITY REDUCTION; DISCRIMINATIVE CLASSIFICATION; DISCRIMINATIVE CLASSIFIERS; FISHER KERNELS; GENERATIVE EMBEDDINGS; HYBRID MODELS; MAXIMUM A POSTERIORS; PRIOR KNOWLEDGE; SEQUENCE MODELING; SPECIFIC COMPONENT; TEST SEQUENCE; TRAINING DATA; TRAINING FEATURES; VECTOR SPACES;

CLASSIFIERS; COMPUTATIONAL GRAMMARS; HYBRID SENSORS; LEARNING SYSTEMS; OBJECT RECOGNITION;

HIDDEN MARKOV MODELS;

EID: 67649388040     PISSN: 00313203     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.patcog.2009.03.023     Document Type: Article

References (64)

1. Rabiner L. A tutorial on hidden Markov models and selected applications in speech recognition. Proceedings of the IEEE 77 2 (1989) 257-286
2. Jelinek F. Statistical Methods for Speech Recognition (1998), MIT Press, Cambridge, MA
3. E.V.F. Casacuberta, J.M. Vilar, Architectures for speech-to-speech translation using finite-state models, in: Workshop on Speech-to-Speech Translation: Algorithms and Systems, ACL, Philadelphia, 2002, pp. 39-44.
4. Molina A., and Pla F. Shallow parsing using specialized HMMs. Journal on Machine Learning Research 2 (2002) 595-613
5. H. Bunke, T. Caelli, Hidden Markov Models Applications in Computer Vision, Series in Machine Perception and Artificial Intelligence, vol. 45, World Scientific, Singapore, 2001.
6. He Y., and Kundu A. 2-D shape classification using hidden Markov model. IEEE Transactions on Pattern Analysis and Machine Intelligence 13 11 (1991) 1172-1184
7. Bicego M., and Murino V. Investigating hidden Markov models' capabilities in 2D shape classification. IEEE Transactions on Pattern Analysis and Machine Intelligence 26 2 (2004) 281-286
8. M. Bicego, U. Castellani, V. Murino, Using hidden Markov models and wavelets for face recognition, in: International Conference on Image Analysis and Processing, 2003, pp. 52-56.
9. S. Eickeler, A. Kosmala, G. Rigoll, Hidden Markov model based continuous online gesture recognition, in: International Conference on Pattern Recognition, vol. 2, 1998, pp. 1206-1208.
10. R. Nowak, Multiscale hidden Markov models for Bayesian image analysis, in: B. Vidakovic, P. Muller (Eds.), Bayesian Inference in Wavelet Based Models, Lecture Notes in Statistics, vol. 141, Springer, Berlin, 1999.
11. Krogh A., Brown M., Mian I., Sjolander K., and Haussler D. Hidden Markov models in computational biology: applications to protein modeling. Journal of Molecular Biology 235 (1994) 1501-1531
12. Durbin R., Eddy S., Krogh A., and Mitchison G. Biological Sequence Analysis: Probabilistic Models of Proteins and Nucleic Acids (1998), Cambridge University Press, Cambridge
13. Gough J., Karplus K., Hughey R., and Chothia C. Assignment of homology to genome sequences using a library of hidden Markov models that represent all proteins of known structure. Journal of Molecular Biology 313 (2001) 903-919
14. Baum L., and Egon J. An inequality with applications to statistical estimation for probabilistic functions of a Markov process and to a model for ecology. Bulletin of the American Meteorology Society 73 (1967) 360-363
15. Baum L. An inequality and associated maximization technique in statistical estimation for probabilistic functions of Markov processes. Inequality 3 (1970) 1-8
16. Duda R., Hart P., and Stork D. Pattern Classification. second ed. (2001), Wiley, New York
17. Fine S., Singer Y., and Tishby N. The hierarchical hidden Markov model: analysis and applications. Machine Learning 32 (1998) 41-62
18. Bengio Y., Lauzon V.-P., and Ducharme R. Experiments on the application of IOHMMs to model financial returns series. IEEE Transactions on Neural Networks 12 1 (2001) 113-123
19. Ghahramani Z., and Jordan M. Factorial hidden Markov models. Machine Learning 29 (1997) 245-273
20. M. Brand, N. Oliver, S. Pentland, Coupled hidden Markov models for complex action recognition, in: IEEE International Conference on Computer Vision and Pattern Recognition, 1997.
21. L. Bahl, P. Brown, P. de Souza, R. Mercer, Maximum mutual information estimation of hidden Markov model parameters for speech recognition, in: IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP 86), vol. 11, 1986, pp. 49-52.
22. Z. Kaiser, B. Horvat, Z. Kacic, A novel loss function for the overall risk criterion based discriminative training of HMM models, in: International Conference on Spoken Language Processing, vol. 2, Beijing, China, 2000, pp. 887-890.
23. K. Na, B. Jeon, D. Chang, S. Chae, S. Ann, Discriminative training of hidden Markov models using overall risk criterion and reduced gradient method, in: European Conference on Speech Communication and Technology, Madrid, Spain, 1995, pp. 97-100.
24. Kaiser Z., Horvat B., and Kacic Z. Overall risk criterion estimation of hidden Markov model parameters. Speech Communication 38 3-4 (2002) 383-398
25. Woodland P.C., and Povey D. Large scale discriminative training of hidden Markov models for speech recognition. Computer Speech and Language 16 (2002) 25-47
26. M. Gales, Discriminative models for speech recognition, in: Information Theory and Applications Workshop, 2007.
27. J. Lafferty, A. McCallum, F. Pereira, Conditional random fields: probabilistic models for segmenting and labelling sequence data, in: International Conference on Machine Learning, 2001, pp. 591-598.
28. A. Gunawardana, M. Mahajan, A. Acero, J. Platt, Hidden conditional random fields for phone classification, in: Interspeech, Lisbon, Portugal, 2005, pp. 1117-1120.
29. Y. Altun, I. Tsochantaridis, T. Hofmann, Hidden Markov support vector machines, in: International Conference on Machine Learning, 2003, pp. 3-10.
30. M. Collins, Discriminative training methods for hidden Markov models: theory and experiments with perceptron algorithm, in: Conference on Empirical Methods in Natural Language Processing, 2002.
31. S. Chakrabartty, G. Cauwenberghs, Forward decoding kernel machines: a hybrid HMM/SVM approach to sequence recognition, in: International Conference on Pattern Recognition: SVM Workshop, 2002.
32. Wolpert D. Stacked generalization. Neural Networks 5 2 (1992) 241-260
33. Ting K., and Witten I.H. Issues in stacked generalization. Journal of Artificial Intelligence Research 10 (1999) 271-289
34. R. Duin, The combining classifier: to train or not to train? in: International Conference on Pattern Recognition, vol. II, Canada, 2002, pp. 765-770.
35. Bicego M., Murino V., and Figueiredo M. Similarity-based classification of sequences using hidden Markov models. Pattern Recognition 37 12 (2004) 2281-2291
36. M. Bicego, E. Pȩkalska, R. Duin, Group-induced vector spaces, in: M. Haindl, J. Kittler, F. Roli (Eds.), Multiple Classifier Systems, Lecture Notes in Computer Science, vol. 4472, Springer, Berlin, 2007, pp. 190-199.
37. Lai C., Tax D., Duin R., Pȩkalska E., and Paclík P. A study on combining image representations for image classification and retrieval. International Journal of Pattern Recognition and Artificial Intelligence 18 5 (2004) 867-890
38. T. Jaakkola, D. Haussler, Exploiting generative models in discriminative classifiers, in: Advances in Neural Information Processing Systems, 1999, pp. 487-493.
39. Lodhi H., Saunders C., Shawe-Taylor J., Cristianini N., and Watkins C. Text classification using string kernels. Journal of Machine Learning Research 2 (2002) 419-444
40. Cortes C., Haffner P., and Mohri M. Rational kernels: theory and algorithms. Journal of Machine Learning Research 5 (2004) 1035-1062
41. N. Smith, M. Gales, Speech recognition using svms, in: Advances in Neural Information Processing Systems, 2002, pp. 1197-1204.
42. L. Chen, H. Man, Combination of Fisher Scores and appearance based features for face recognition, in: ACM SIGMM Multimedia Biometrics Methods and Applications Workshop, 2003, pp. 74-81.
43. L. Chen, H. Man, Discriminant analysis of stochastic models and its application to face recognition, in: International Workshop on Analysis and Modeling of Faces and Gestures, 2003, pp. 5-10.
44. M. Layton, M. Gales, Augmented statistical models: exploiting generative models in discriminative classifiers, in: Advances in Neural Information Processing Systems, 2005.
45. N. Smith, Using augmented statistical models and score spaces for classification, Ph.D. Thesis, Engineering Department, Cambridge University, 2003.
46. R. Duin, P. Juszczak, P. Paclík, E. Pȩkalska, D. De Ridder, D. Tax, Prtools4, a matlab toolbox for pattern recognition, Delft University of Technology, 2004 〈http://www.prtools.org〉.
47. Fukanaga K. Introduction to Statistical Pattern Recognition. second ed. (1990), Academic Press, San Diego
48. Hastie T., Tibshirani R., and Friedman J. The Elements of Statistical Learning (2001), Springer, Berlin
49. Everitt B.S. The Cambridge Dictionary of Statistics (2003)
50. Schölkopf B., and Smola A. Learning with Kernels (2002), MIT Press, Cambridge, MA
51. Bennett K. Combining support vector and mathematical programming methods for induction. In: Schölkopf B., Burges C., and Smola A. (Eds). Advances in Kernel Methods-SV Learning (1999), MIT Press, Cambridge, MA 307-326
52. Bellman R. Adaptive Control Processes: A Guided Tour (1961), Princeton University Press, New York
53. Vapnik V. Statistical Learning Theory (1998), Wiley, New York
54. Bhattacharyya C., Grate L., Rizki A., Radisky D., Molina F., Jordan M., Bissel M., and Mian I. Simultaneous classification and relevant feature identification in high-dimensional spaces: application to molecular profiling data. Signal Processing 83 (2003) 729-743
55. Chen L., Man H., and Nefian A.V. Face recognition based on multi-class mapping of Fisher Scores. Pattern Recognition 38 (2005) 799-811
56. Tsuda K., Kin T., and Asai K. Marginalised kernels for biological sequences. Bioinformatics 18 (2002) 268-275
57. Layton M., and Gales M. Acoustic modelling using continuous rational kernels. Journal of VLSI Signal Processing 48 (2007) 67-82
58. Amari S. Natural gradient works efficiently in learning. Neural Computation 10 (1998) 251-276
59. N. Arica, F. Yarman-Vural, A shape descriptor based on circular hidden Markov model, in: International Conference on Pattern Recognition, vol. 1, 2000, pp. 924-927.
60. G. Andreu, A. Crespo, J. Valiente, Selecting the toroidal self-organizing feature maps (TSOFM) best organized to object recognition, in: International Conference on Neural Networks, vol. 2, 1997, pp. 1341-1346.
61. Mollineda R., Vidal E., and Casacuberta F. Cyclic sequence alignments: approximate versus optimal techniques. International Journal of Pattern Recognition and Artificial Intelligence 16 3 (2002) 291-299
62. M. Kadous, Temporal classification: extending the classification paradigm to multivariate time series, Ph.D. Thesis, School of Computer Science and Engineering, University of New South Wales, 2002.
63. M. Kadous, Learning comprehensible descriptions of multivariate time series, in: International Conference on Machine Learning, 1999, pp. 454-463.
64. Kudo M., Toyama J., and Shimbo M. Multidimensional curve classification using passing-through regions. Pattern Recognition Letters 20 11-13 (1999) 1103-1111