Efficient multiple hyperparameter learning for log-linear models

Advances in Neural Information Processing Systems 20 - Proceedings of the 2007 Conference

Volumn , Issue , 2008, Pages

  Do, Chuong B ^a   Foo, Chuan Sheng ^a   Ng, Andrew Y ^a  

^a Stanford University   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE; BIOINFORMATICS; REGRESSION ANALYSIS;

HYPER-PARAMETER; INPUT FEATURES; LOGLINEAR MODEL; MODELING COMPLEXITY; NEURAL NETWORK AND SUPPORT VECTOR MACHINES; PREDICTION MODELLING; REGULARISATION; REGULARIZER; SEQUENCE LABELING; STRUCTURED PREDICTION;

FORECASTING;

EID: 85161996864     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (39)

1. L. Andersen, J. Larsen, L. Hansen, and M. Hintz-Madsen. Adaptive regularization of neural classifiers. In NNSP, 1997.
2. D. Anguita, S. Ridella, F. Rivieccio, and R. Zunino. Hyperparameter design criteria for support vector classifiers. Neurocomputing, 55:109-134, 2003. (Pubitemid 37336672)
3. Y. Bengio. Gradient-based optimization of hyperparameters. Neural Computation, 12:1889-1900, 2000.
4. O. Chapelle, V. Vapnik, O. Bousquet, and S.Mukherjee. Choosing multiple parameters for support vector machines. Machine Learning, 46(1-3):131-159, 2002. (Pubitemid 34129966)
5. D. Chen and M. Hagan. Optimal use of regularization and cross-validation in neural network modeling. In IJCNN, 1999.
6. C. B. Do, S. S. Gross, and S. Batzoglou. CONTRAlign: discriminative training for protein sequence alignment. In RECOMB, pages 160-174, 2006. (Pubitemid 44019154)
7. C. B. Do, D. A. Woods, and S. Batzoglou. CONTRAfold: RNA secondary structure prediction without physics-based models. Bioinformatics, 22(14):e90-e98, 2006. (Pubitemid 44288277)
8. K. Duan, S. S. Keerthi, and A.N. Poo. Evaluation of simple performance measures for tuning SVM hyperparameters. Neurocomputing, 51(4):41-59, 2003. (Pubitemid 36367224)
9. R. Eigenmann and J. A. Nossek. Gradient based adaptive regularization. In NNSP, pages 87-94, 1999. (Pubitemid 30510363)
10. T. Glasmachers and C. Igel. Gradient-based adaptation of general Gaussian kernels. Neural Comp., 17(10):2099-2105, 2005. (Pubitemid 41186994)
11. A. Globerson, T. Y. Koo, X. Carreras, and M. Collins. Exponentiated gradient algorithms for log-linear structured prediction. In ICML, pages 305-312, 2007.
12. C. Goutte and J. Larsen. Adaptive regularization of neural networks using conjugate gradient. In ICASSP, 1998.
13. S. Griffiths-Jones, S. Moxon, M. Marshall, A. Khanna, S. R. Eddy, and A. Bateman. Rfam: annotating non-coding RNAs in complete genomes. Nucleic Acids Res, 33:D121-D124, 2005. (Pubitemid 40207843)
14. A. Kapoor, Y. Qi, H. Ahn, and R. W. Picard. Hyperparameter and kernel learning for graph based semisupervised classification. In NIPS, pages 627-634, 2006.
15. S. S. Keerthi. Efficient tuning of SVM hyperparameters using radius/margin bound and iterative algorithms. IEEE Transaction on Neural Networks, 13(5):1225-1229, 2002.
16. S. S. Keerthi, V. Sindhwani, and O. Chapelle. An efficient method for gradient-based adaptation of hyperparameters in SVM models. In NIPS, 2007.
17. K. Kobayashi, D. Kitakoshi, and R. Nakano. Yet faster method to optimize SVR hyperparameters based on minimizing cross-validation error. In IJCNN, volume 2, pages 871-876, 2005. (Pubitemid 44055566)
18. K. Kobayashi and R. Nakano. Faster optimization of SVR hyperparameters based on minimizing crossvalidation error. In IEEE Conference on Cybernetics and Intelligent Systems, 2004.
19. J. Lafferty, A. McCallum, and F. Pereira. Conditional random fields: probabilistic models for segmenting and labeling sequence data. In ICML 18, pages 282-289, 2001.
20. J. Larsen, L. K. Hansen, C. Svarer, and M. Ohlsson. Design and regularization of neural networks: the optimal use of a validation set. In NNSP, 1996.
21. J. Larsen, C. Svarer, L. N. Andersen, and L. K. Hansen. Adaptive regularization in neural network modeling. In Neural Networks: Tricks of the Trade, pages 113-132, 1996.
22. D. J. C. MacKay. Bayesian interpolation. Neural Computation, 4(3):415-447, 1992.
23. D. J. C. MacKay and R. Takeuchi. Interpolation models with multiple hyperparameters. Statistics and Computing, 8:15-23, 1998. (Pubitemid 128405007)
24. J. R. R. A. Martins, P. Sturdza, and J. J. Alonso. The complex-step derivative approximation. ACM Trans. Math. Softw., 29(3):245-262, 2003.
25. T. P. Minka. Expectation propagation for approximate Bayesian inference. In UAI, volume 17, pages 362-369, 2001.
26. I. Murray and Z. Ghahramani. Bayesian learning in undirected graphical models: approximate MCMC algorithms. In UAI, pages 392-399, 2004.
27. R. M. Neal. Bayesian Learning for Neural Networks. Springer, 1996.
28. A. Y. Ng. Preventing over fitting of cross-validation data. In ICML, pages 245-253, 1997.
29. A. Y. Ng. Feature selection, L1 vs. L2 regularization, and rotational invariance. In ICML, 2004.
30. J. Nocedal and S. J. Wright. Numerical Optimization. Springer, 1999.
31. B. A. Pearlmutter. Fast exact multiplication by the Hessian. Neural Comp, 6(1):147-160, 1994.
32. Y. Qi, M. Szummer, and T. P. Minka. Bayesian conditional random fields. In AISTATS, 2005.
33. M. Seeger. Cross-validation optimization for large scale hierarchical classification kernel methods. In NIPS, 2007.
34. F. Sha and F. Pereira. Shallow parsing with conditional random fields. In NAACL, pages 134-141, 2003.
35. S. Sundararajan and S. S. Keerthi. Predictive approaches for choosing hyperparameters in Gaussian processes. Neural Comp., 13(5):1103-1118, 2001. (Pubitemid 33595024)
36. S. V. N. Vishwanathan, N. N. Schraudolph, M. W. Schmidt, and K. P. Murphy. Accelerated training of conditional random fields with stochastic gradient methods. In ICML, pages 969-976, 2006.
37. M. Wellings and S. Parise. Bayesian random fields: the Bethe-Laplace approximation. In ICML, 2006.
38. C. K. I. Williams and D. Barber. Bayesian classification with Gaussian processes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20(12):1342-1351, 1998. (Pubitemid 128741378)
39. X. Zhang and W. S. Lee. Hyperparameter learning for graph based semi-supervised learning algorithms. In NIPS, 2007.