Exploitation of Unlabeled Sequences in Hidden Markov Models

IEEE Transactions on Pattern Analysis and Machine Intelligence

Volumn 25, Issue 12, 2003, Pages 1570-1581

  Inoue, Masashi ^a   Ueda, Naonori ^b  

^a NARA INSTITUTE OF SCIENCE AND TECHNOLOGY   (Japan)

^b NTT Communication Science Laboratories   (Japan)

Author keywords

Extended Baum Welch algorithm; Hidden Markov models; Sequential data; Unlabeled data

Indexed keywords

ALGORITHMS; CLASSIFICATION (OF INFORMATION); COMPUTATIONAL METHODS; DATA REDUCTION; ERRORS; LEARNING SYSTEMS; MAXIMUM LIKELIHOOD ESTIMATION; PARAMETER ESTIMATION; PROBLEM SOLVING; VECTORS;

DATA LABELING; SEQUENTIAL DATA;

MARKOV PROCESSES;

EID: 0347380864     PISSN: 01628828     EISSN: None     Source Type: Journal    
DOI: 10.1109/TPAMI.2003.1251150     Document Type: Article

References (29)

1. S.J. Raudys and A.K. Jain, "Small Sample Size Effects in Statistical Pattern Recognition: Recommendations for Practitioners," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 13, no. 3, pp. 252-264, Mar. 1991.
2. S. Ganesalingam and G.J. McLachlan, "Some Efficiency Results for the Estimation of the Mixing Proportion in a Mixture of Two Normal Distributions," Biometrics, vol. 37, pp. 22-33, Mar. 1981.
3. G.J. McLachlan, Discriminant Analysis and Statistical Pattern Recognition. John Wiley and Sons, 1992.
4. B.M. Shahshahani and D.A. Landgrebe, "The Effect of Unlabeled Samples in Reducing the Small Sample Size Problem and Mitigating the Hughes Phenomenon," IEEE Trans. Geoscience and Remote Sensing, vol. 32, no. 5, pp. 1087-1095, Sept. 1994.
5. D.J. Miller and H.S. Uyar, "A Mixture of Experts Classifier with Learning Based on Both Labelled and Unlabelled Data," Advances in Neural Information Processing Systems, M.C. Mozer, M.I. Jordan, T. Petsche, eds., vol. 9, pp. 571-577, 1997.
6. K. Nigam, A.K. McCallum, S. Thrun, and T. Mitchell, "Text Classification from Labeled and Unlabeled Documents Using EM," Machine Learning, vol. 39, nos. 2/3, pp. 103-134, May 2000.
7. T. Kemp and A. Waibel, "Unsupervised Training of a Speech Recognizer: Recent Experiments," Proc. Eurospeech, vol. 6, pp. 2725-2728, 1999.
8. L. Lamel, J.L. Gauvain, and G. Adda, "Lightly Supervised and Unsupervised Acoustic Model Training," Computer Speech and Language, vol. 16, no. 1, pp. 115-129, Jan. 2002.
9. L.R. Bahl, F. Jelinek, and R. Mercer, "A Maximum Likelihood Approach to Continuous Speech Recognition," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 5, pp. 179-190, Mar. 1983.
10. T.E. Starner and A. Pentland, "Visual Recognition of American Sign Language Using Hidden Markov Models," Proc. Int'l Work-shop Automatic Face and Gesture Recognition, pp. 189-194, 1995.
11. A. Krogh, M. Brown, I.S. Mian, K. Sjölander, and D. Haussler, "Hidden Markov Models in Computational Biology Applications to Protein Modeling," J. Molecular Biology, vol. 235, no. 5, pp. 1501-1531, Feb. 1994.
12. D. Cutting, J. Kupiec, J. Pedersen, and P. Sibun, "A Practical Part-of-Speech Tagger," Proc. Third Conf. Applied Natural Language Processing, pp. 133-140, 1992.
13. D.M. Bikel, R. Schwartz, and R.M. Weischedel, "An Algorithm that Learns What's in a Name," Machine Learning, vol. 34, nos. 1-3, pp. 211-231, Feb. 1999.
14. B. Merialdo, "Tagging English Text with a Probabilistic Model, " Computational Linguistics, vol. 20, no. 2 pp. 155-171, June 1994.
15. D. Elworthy, "Does Baum-Welch Reestimation Help Taggers?" Proc. Fourth Conf. Applied Natural Language Processing, pp. 53-58, 1994.
16. K. Seymore, A. McCallum, and R. Rosenfeld, "Learning Hidden Markov Model Structure for Information Extraction," Proc. AAAI Workshop Machine Learning for Information Extraction, pp. 37-42, 1999.
17. J.R. Bellegarda and D. Nahamoo, "Tied Mixture Continuous Parameter Modeling for Speech Recognition," IEEE Trans. Acoustics, Speech, and Signal Processing, vol. 38, no. 12, pp. 2033-2045, Dec. 1990.
18. X.D. Huang, "Phoneme Classification Using Semicontinuous Hidden Markov Models," IEEE Trans. Signal Processing, vol. 40, no. 5, pp. 1062-1067, May 1992.
19. A.P. Dempster, N.M. Laird, and D.B. Rubin, "Maximum Likelihood from Incomplete Data via the EM Algorithm," J. Royal Statistical Soc. 5, vol. 39, no. 1, pp. 1-38, 1977.
20. M. Inoue and N. Ueda, "HMMs for Both Labeled and Unlabeled Time Series Data," Proc. 11th IEEE Workshop Neural Networks for Signal Processing, pp. 93-102, 2001.
21. G. Aversano, A. Esposito, A. Esposito, and M. Marinaro, "A New Text-Independent Method for Phoneme Segmentation," Proc. 44th IEEE Midwest Symp. Circuits and Systems, vol. 2, pp. 516-519, 2001.
22. L.E. Baum, T. Petrie, G. Soules, and N. Weiss, "A Maximization Technique Occurring in the Statistical Analysis of Probabilistic Function of Markov Chains," The Annals of Math. Statistics, vol. 41, no. 1, pp. 164-171, 1970.
23. X.D. Huang, Y. Ariki, and M.A. Jack, Hidden Markov Models for Speech Recognition. Edinburgh: Edinburgh Univ. Press, 1990.
24. R.O. Duda, P.E. Hart, and D.G. Stork, Pattern Classification, second ed. John Wiley and Sons, 2001.
25. J.S. Garofolo, L.F. Lamel, W.M. Fisher, J.G. Fiscus, D.S. Pallett, and N.L. Dahlgren, DARPA TIMIT Acoustic-Phonetic Continuous Speech Corpus CD-ROM. Gaithersburg, Md.: Nat'l Inst. of Standards and Technology, 1993.
26. K.F. Lee and H.W. Hon, "Speaker-Independent Phone Recognition Using Hidden Markov Models," IEEE Trans. Acoustics, Speech, and Signal Processing, vol. 37, no. 11, pp. 1641-1648, Nov. 1989.
27. E. McDermott and S. Katagiri, "String-Level MCE for Continuous Phoneme Recognition," Proc. Eurospeech, vol. 1, pp. 123-126, 1997.
28. A. Corduneanu and T. Jaakkola, "Continuation Methods for Mixing Heterogeneous Sources," Uncertainty in Artificial Intelligence: Proc. 18th Conf., pp. 111-118, 2002.
29. A. Blum and T. Mitchell, "Combining Labeled and Unlabeled Data with Co-Training," Proc. Workshop Computational Learning Theory, pp. 92-100, 1998.