A regularized maximum figure-of-merit (rMFoM) approach to supervised and semi-supervised learning

IEEE Transactions on Audio, Speech and Language Processing

Volumn 19, Issue 5, 2011, Pages 1316-1327

  Ma, Chengyuan ^a   Lee, Chin Hui ^b  

^a NUANCE COMMUNICATIONS   (United States)

^b GEORGIA INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

Deterministic annealing; discriminative training; maximum figure of merit; regularization; semi supervised learning; supervised learning; text categorization; trust region method

Indexed keywords

DETERMINISTIC ANNEALING; DISCRIMINATIVE TRAINING; MAXIMUM FIGURE-OF-MERIT; REGULARIZATION; SEMI-SUPERVISED LEARNING; TEXT CATEGORIZATION; TRUST REGION METHOD;

LEARNING ALGORITHMS; TEXT PROCESSING;

SUPERVISED LEARNING;

EID: 79956286980     PISSN: 15587916     EISSN: None     Source Type: Journal    
DOI: 10.1109/TASL.2010.2090146     Document Type: Article

References (42)

1. R. O. Duda, P. E. Hart, and D. G. Stork, Pattern Classification, 2nd ed. New York: Wiley-Interscience, 2000.
2. C. M. Bishop, Pattern Recognition and Machine Learning. New York: Springer, 2006.
3. T. Hastie, R. Tibshirani, and J. Friedman, The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd ed. New York: Springer, 2009.
4. C.-J. Lin, R. C.Weng, and S. S. Keerthi, "Trust region Newton method for logistic regression," J. Mach. Learn. Res., vol. 9, pp. 627-650, 2008.
5. A. L. Berger, S. D. Pietra, and V. J. D. Pietra, "A maximum entropy approach to natural language processing," Comput. Linguist., vol. 22, no. 1, pp. 39-71, 1996.
6. L. R. Rabiner, "A tutorial on hidden Markov models and selected applications in speech recognition," Proc. IEEE, vol. 77, no. 2, pp. 257-286, Feb. 1989.
7. C.-H. Lee and Q. Huo, "On adaptive decision rules and decision parameter adaptation for automatic speech recognition," Proc. IEEE, vol. 88, no. 8, pp. 1241-1269, Aug. 2000.
8. G. Casella and R. L. Berger, Statistical Inference, 2nd ed. Pacific Grove, CA: Duxbury, 2001.
9. C. J. Burges, "A tutorial on support vector machines for pattern recognition," Data Mining Knowl. Discov., vol. 2, pp. 121-167, 1998. (Pubitemid 128695475)
10. L. Rabiner and B.-H. Juang, Fundamentals of Speech Recognition. Englewood Cliffs, NJ: Prentice-Hall, 1993.
11. C. D. Manning, P. Raghavan, and H. Schütze, Introduction to Information Retrieval. Cambridge, U.K.: Cambridge Univ. Press, 2008.
12. C. Ma, B. Byun, I. Kim, and C.-H. Lee, "A detection-based approach to broadcast news video story segmentation," in Proc. ICASSP, 2009, pp. 1957-1960.
13. C. Ma and C.-H. Lee, "An efficient gradient computation approach to discriminative fusion optimization in semantic concept detection," in Proc. ICPR, 2008.
14. A. Martin, G. Doddington, T. Kamm, M. Ordowski, and M. Przybocki, "The DET curve in assessment of detection task performance," in Proc. EuroSpeech, 1997, pp. 1895-1898.
15. B.-H. Juang and S. Katagiri, "Discriminative learning for minimum error classification," IEEE Trans. Signal Process., vol. 40, no. 12, pp. 3043-3054, Dec. 1992. (Pubitemid 23603018)
16. K.-A. Toh and H.-L. Eng, "Between classification-error approximation and weighted least-squares learning," IEEE Trans. Pattern Anal. Mach. Intell., vol. 30, no. 4, pp. 658-669, Apr. 2008. (Pubitemid 351559719)
17. T. Joachims, "A support vector method for multivariate performance measures," in Proc. ICML, 2005.
18. B.-H. Juang, W. Chou, and C.-H. Lee, "Minimum classification error rate methods for speech recognition," IEEE Trans. Speech Audio Process., vol. 5, no. 3, pp. 257-265, May 1997. (Pubitemid 127745998)
19. R. Schlüter, W. Macherey, B. Müller, and H. Ney, "Comparison of discriminative training criteria and optimization methods for speech recognition," Speech Commun., vol. 34, pp. 287-310, 2001. (Pubitemid 32284868)
20. C. Ma and E. Chang, "Comparison of discriminative training methods for speaker verification," in Proc. ICASSP, 2003, pp. 192-195.
21. S. Gao, W. Wu, C.-H. Lee, and T.-S. Chua, "A maximal figure-of-merit learning approach to text categorization," in Proc. ACM SIGIR, 2003, pp. 174-181.
22. S. Gao, C.-H. Lee, and J. H. Lim, "An ensemble classifier learning approach to ROC optimization," in Proc. ICPR, 2006, pp. 679-682. (Pubitemid 46532290)
23. T. Fawcett, "ROC Graphs: Notes and Practical Considerations for Data Mining Researchers," Tech. Rep. HP Labs, 2003.
24. S. Gao, W.Wu, C.-H. Lee, and T.-S. Chua, "A maximal figure-of-merit (MFoM)-learning approach to robust classifier design for text categorization," ACM Trans. Inf. Syst., vol. 24, no. 2, pp. 190-218, 2006. (Pubitemid 44180737)
25. A. N. Tikhonov and V. A. Arsenin, Solution of Ill-Posed Problems. Washington, DC: Winston & Sons, 1977.
26. M. Jansche, "Maximum expected F-measure training of logistic regression model," in Proc. EMNLP, 2005.
27. O. Chapelle, B. Schölkopf, and A. Zien, Semi-Supervised Learning. Cambridge, MA: MIT Press, 2006.
28. K. Nigam, A. Mccallum, S. Thrun, and T. Mitchell, "Text classification from labeled and unlabeled documents using EM," Mach. Learn., pp. 103-134, 1999.
29. S. Abney, "Understanding the Yarowsky algorithm," Comput. Linguist., vol. 30, no. 3, pp. 365-395, 2004.
30. X. Zhu, Semi-Supervised Learning Literature Survey Univ. of Wisconsin- Madison, Comput. Sci., 2005, Tech. Rep. 1530.
31. T. Joachims, "Transductive inference for text classification using support vector machines," in Proc. ICML, 1999, pp. 200-209.
32. J. Nocedal and S. J. Wright, Numerical Optimization, 2nd ed. New York: Springer, 2006.
33. D. Povey and P. Woodland, "Minimum phone error and I-smoothing for improved discriminative training," in Proc. ICASSP, 2002, pp. 321-324.
34. J. Rokui, "Rapid discriminative learning based on misclassification measure," Syst. Comput. Japan, vol. 37, pp. 58-68, 2006.
35. R. Tibshirani, "Regression shrinkage and selection via the lasso," J. R. Statist. Soc., Ser. B, vol. 58, pp. 267-288, 1994.
36. V. Sindhwani and S. S. Keerthi, "Large scale semi-supervised linear SVMs," in Proc. ACM SIGIR, 2006, pp. 477-484.
37. K. Koh, S.-J. Kim, and S. Boyd, "An interior-point method for largescale l1-regularized logistic regression," J. Mach. Learn. Res., vol. 8, pp. 1519-1555, 2007. (Pubitemid 47210273)
38. S. Gao, W. Wu, C.-H. Lee, and T.-S. Chua, "A MFoM learning approach to robust multiclass multi-label text categorization," in Proc. ICML, 2004.
39. A. P. Dempster, N. M. Laird, and D. B. Rubin, "Maximum likelihood from incomplete data via the EM algorithm," J. R. Statist. Soc., vol. 39, pp. 1-38, 1977.
40. K. Rose, "Deterministic annealing for clustering, compression, classification, regression, and related optimization problems," Proc. IEEE, pp. 2210-2239, 1998. (Pubitemid 128720301)
41. S. Katagiri, B.-H. Juang, and C.-H. Lee, "Pattern recognition using a family of design algorithm based upon the generalized probabilistic descent method," IEEE Trans. Speech Audio Process., vol. 86, pp. 2345-2373, 1998.
42. A. R. Conn, N. I. M. Gould, and P. L. Toint, Trust-Region Methods. Philadelphia, PA: SIAM, 2000.