A rate distortion approach for semi-supervised conditional random fields

Advances in Neural Information Processing Systems 22 - Proceedings of the 2009 Conference

Volumn , Issue , 2009, Pages 2008-2016

  Wang, Yang ^a   Haffari, Gholamreza ^a   Wang, Shaojun ^b   Mori, Greg ^a  

^a SIMON FRASER UNIVERSITY   (Canada)

^b WRIGHT STATE UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ELECTRIC DISTORTION; INFORMATION THEORY; LEARNING ALGORITHMS; RANDOM PROCESSES; SIGNAL DISTORTION; SUPERVISED LEARNING;

CONDITIONAL ENTROPY; CONDITIONAL LIKELIHOOD; INFORMATION-THEORETIC APPROACH; LABELED DATA; NOVEL INFORMATION; RANDOM FIELDS; RATE DISTORTIONS; REGULARISATION; SEMI-SUPERVISED; SEMI-SUPERVISED LEARNING;

ENTROPY;

EID: 84863351869     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (32)

1. S. Abney. Semi-Supervised Learning for Computational Linguistics. Chapman & Hall/CRC, 2007.
2. Y. Altun, D. McAllester and M. Belkin. Maximum margin semi-supervised learning for structured variables. NIPS 18:33-40, 2005.
3. S. Arimoto. An algorithm for computing the capacity of arbitrary discrete memoryless channels. IEEE Transactions on Information Theory, 18:1814-1820, 1972.
4. L. Bahl, P. Brown, P. de Souza and R. Mercer. Maximum mutual information estimation of hidden Markov model parameters for speech recognition. ICASSP, 11:49-52, 1986.
5. T. Berger and J. Gibson. Lossy source coding. IEEE Transactions on Information Theory, 44(6):2693-2723, 1998.
6. R. Blahut. Computation of channel capacity and rate-distortion functions. IEEE Transactions on Information Theory, 18:460-473, 1972.
7. R. Blahut. Principles and Practice of Information Theory, Addison-Wesley, 1987.
8. S. Boyd and L. Vandenberghe. Convex Optimization, Cambridge University Press, 2004.
9. U. Brefeld and T. Scheffer. Semi-supervised learning for structured output variables. ICML, 145-152, 2006.
10. O. Chapelle, B. Scholköpf and A. Zien. Semi-Supervised Learning, MIT Press, 2006.
11. A. Corduneanu and T. Jaakkola. On information regularization. UAI, 151-158, 2003.
12. A. Corduneanu and T. Jaakkola. Distributed information regularization on graphs. NIPS, 17:297-304, 2004.
13. A. Corduneanu and T. Jaakkola. Data dependent regularization. In Semi-Supervised Learning, O. Chapelle, B. Scholköpf and A. Zien, (Editors), 163-182, MIT Press, 2006.
14. T. Cover and J. Thomas. Elements of Information Theory, Wiley, 1991.
15. Y. Grandvalet and Y. Bengio. Semi-supervised learning by entropy minimization. NIPS, 17:529-536, 2004.
16. F. Jiao, S. Wang, C. Lee, R. Greiner and D. Schuurmans. Semi-supervised conditional random fields for improved sequence segmentation and labeling. COLING/ACL, 209-216, 2006.
17. M. Jordan, Z. Ghahramani, T. Jaakkola and L. Saul. Introduction to variational methods for graphical models. Machine Learning, 37:183-233, 1999.
18. D. Jurafsky and J. Martin. Speech and Language Processing, 2nd Edition, Prentice Hall, 2008.
19. J. Lafferty, A. McCallum and F. Pereira. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. ICML, 282-289, 2001.
20. C. Lee, S. Wang, F. Jiao, D. Schuurmans and R. Greiner. Learning to model spatial dependency: Semi-supervised discriminative random fields. NIPS, 19:793-800, 2006.
21. G. Mann and A. McCallum. Efficient computation of entropy gradient for semi-supervised conditional random fields. NAACL/HLT, 109-112, 2007.
22. G. Mann and A. McCallum. Generalized expectation criteria for semi-supervised learning of conditional random fields. ACL, 870-878, 2008.
23. Y. Normandin. Maximum mutual information estimation of hidden Markov models. In Automatic Speech and Speaker Recognition: Advanced Topics, C. Lee, F. Soong and K. Paliwal (Editors), 57-81, Springer, 1996.
24. N. Oliver and A. Garg. MMIHMM: maximum mutual information hidden Markov models. ICML, 466-473, 2002.
25. K. Rose. Deterministic annealing for clustering, compression, classification, regression, and related optimization problems. Proceedings of the IEEE, 80:2210-2239, 1998.
26. N. Slonim, G. Atwal, G. Tkacik and W. Bialek. Information based clustering. Proceedings of National Academy of Science (PNAS), 102:18297-18302, 2005.
27. S. Still and W. Bialek. How many clusters? An information theoretic perspective. Neural Computation, 16:2483-2506, 2004.
28. M. Szummer and T. Jaakkola. Information regularization with partially labeled data. NIPS, 1025-1032, 2002.
29. B. Taskar, C. Guestrain and D. Koller. Max-margin Markov networks. NIPS, 16:25-32, 2003.
30. N. Tishby, F. Pereira, and W. Bialek. The information bottleneck method. The 37th Annual Allerton Conference on Communication, Control, and Computing, 368-377, 1999.
31. L. Zheng, S. Wang, Y. Liu and C. Lee. Information theoretic regularization for semi-supervised boosting. KDD, 1017-1026, 2009.
32. X. Zhu. Semi-supervised learning literature survey. Computer Sciences TR 1530, University of Wisconsin Madison, 2007.