Averaged stochastic gradient descent with feedback: An accurate, robust, and fast training method

Proceedings - IEEE International Conference on Data Mining, ICDM

Volumn , Issue , 2010, Pages 1067-1072

  Sun, Xu ^a   Kashima, Hisashi ^a   Matsuzaki, Takuya ^a   Ueda, Naonori ^b  

^a UNIVERSITY OF TOKYO   (Japan)

^b NTT Corporation   (Japan)

Author keywords

[No Author keywords available]

Indexed keywords

CONDITIONAL MODELS; CONVERGENCE PROPERTIES; GENERAL PURPOSE; LARGE DATASETS; STOCHASTIC GRADIENT; STOCHASTIC GRADIENT DESCENT; TRAINING METHODS; TRAINING SPEED; WEIGHT VECTOR;

DATA MINING; TIME VARYING NETWORKS;

STOCHASTIC SYSTEMS;

EID: 79951744105     PISSN: 15504786     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICDM.2010.26     Document Type: Conference Paper

References (16)

1. S. Petrov and D. Klein, "Discriminative log-linear grammars with latent variables," in Advances in Neural Information Processing Systems 20 (NIPS), 2008, pp. 1153-1160.
2. S. B. Wang, A. Quattoni, L.-P. Morency, D. Demirdjian, and T. Darrell, "Hidden conditional random fields for gesture recognition," in Proceedings of CVPR'06. IEEE Computer Society, 2006, pp. 1521-1527. (Pubitemid 44931500)
3. X. Sun, N. Okazaki, and J. Tsujii, "Robust approach to abbreviating terms: A discriminative latent variable model with global information," in Proceedings of the ACL'09, Suntec, Singapore, August 2009, pp. 905-913.
4. X. Sun, T. Matsuzaki, D. Okanohara, and J. Tsujii, "Latent variable perceptron algorithm for structured classification," in Proceedings of the 21st International Joint Conference on Artificial Intelligence (IJCAI 2009), 2009, pp. 1236-1242.
5. S. Petrov, "Products of random latent variable grammars," in Proceedings of NAACL'10, 2010, to appear.
6. J. Nocedal and S. J. Wright, "Numerical optimization," Springer, 1999.
7. Y. Tsuruoka, J. Tsujii, and S. Ananiadou, "Stochastic gradient descent training for l1-regularized log-linear models with cumulative penalty," in Proceedings of ACL'09, Suntec, Singapore, August 2009, pp. 477-485.
8. S. Shalev-Shwartz, Y. Singer, and N. Srebro, "Pegasos: Primal estimated sub-gradient solver for svm," in Proceedings of ICML'07, 2007.
9. L. Bottou, "Online algorithms and stochastic approximations," Online Learning and Neural Networks. Saad, David. Cambridge University Press, 1998.
10. L.-P. Morency, A. Quattoni, and T. Darrell, "Latent-dynamic discriminative models for continuous gesture recognition," in Proceedings of CVPR'07, 2007, pp. 1-8.
11. M. Collins, A. Globerson, T. Koo, X. Carreras, and P. L. Bartlett, "Exponentiated gradient algorithms for conditional random fields and max-margin markov networks," J. Mach. Learn. Res. (JMLR), vol. 9, pp. 1775-1822, 2008.
12. M. Collins, "Discriminative training methods for hidden markov models: Theory and experiments with perceptron algorithms," in Proceedings of EMNLP'02, 2002, pp. 1-8.
13. L. Bottou, "Stochastic learning," Advanced Lectures on Machine Learning, pp. 146-168, 2004. (Pubitemid 39741631)
14. J.-D. Kim, T. Ohta, Y. Tsuruoka, and Y. Tateisi, "Introduction to the bio-entity recognition task at JNLPBA," in Proceedings of BioNLP'04, 2004, pp. 70-75.
15. T. Huynh, M. Fritz, and B. Schiele, "Discovery of activity patterns using topic models," in Proceedings of the 10th international conference on Ubiquitous computing. ACM, 2008, pp. 10-19.
16. Y. Hattori, M. Takemori, S. Inoue, G. Hirakawa, and O. Sudo, "Operation and baseline assessment of large scale activity gathering system by mobile device," in Proceedings of DICOMO'10, 2010.