An interior-point stochastic approximation method and an L1-regularized delta rule

Advances in Neural Information Processing Systems 21 - Proceedings of the 2008 Conference

Volumn , Issue , 2009, Pages 233-240

  Carbonetto, Peter ^a   Schmidt, Mark ^a   De Freitas, Nando ^a  

^a UNIVERSITY OF BRITISH COLUMBIA   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

APPROXIMATION THEORY; ARTIFICIAL INTELLIGENCE; LEARNING SYSTEMS; LINEAR PROGRAMMING; NONLINEAR PROGRAMMING;

APPROXIMATION METHODS; GENERAL CONSTRAINTS; INTERIOR POINT; INTERIOR-POINT METHOD; LEARNING PROBLEM; MACHINE-LEARNING; ONLINE LEARNING; PROXIMATION; STOCHASTIC APPROXIMATION METHODS; STOCHASTICS;

STOCHASTIC SYSTEMS;

EID: 78649405622     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (27)

1. L. Bottou and O. Bousquet, The tradeoffs of large scale learning, in Advances in Neural Infor-mation Processing Systems, vol. 20, 1998.
2. S. Boyd and L. Vandenberghe, Convex optimization, Cambridge University Press, 2004.
3. S. Chen, D. Donoho, and M. Saunders, Atomic decomposition by basis pursuit, SIAM Journal on Scientific Computing, 20 (1999), pp. 33-61. (Pubitemid 128689501)
4. G. V. Cormack and T. R. Lynam, Spam corpus creation for TREC, in Proc. 2nd CEAS, 2005.
5. -, Online supervised spam filter evaluation, ACM Trans. Information Systems, 25 (2007).
6. A. V. Fiacco and G. P. McCormick, Nonlinear programming: sequential unconstrained mini-mization techniques, John Wiley and Sons, 1968.
7. A. Forsgren, P. E. Gill, and M. H. Wright, Interior methods for nonlinear optimization, SIAM Review, 44 (2002), pp. 525-597.
8. A. Genkin, D. D. Lewis, and D. Madigan, Large-scale Bayesian logistic regression for text categorization, Technometrics, 49 (2007), pp. 291-304. (Pubitemid 47292487)
9. T. Hastie, R. Tibshirani, and J. Friedman, The elements of statistical learning, Springer, 2001.
10. S.-J. Kim, K. Koh, M. Lustig, S. Boyd, and D. Gorinevsky, An interior-point method for large-scale L1-regularized least squares, IEEE J. Selected Topics in Signal Processing, 1 (2007).
11. H. J. Kushner and D. S. Clark, Stochastic approximation methods for constrained and uncon-strained systems, Springer-Verlag, 1978.
12. T. M. Mitchell, Machine Learning, McGraw-Hill, 1997.
13. J. Nocedal and S. J. Wright, Numerical Optimization, Springer, 2nd ed., 2006.
14. B. T. Poljak, Nonlinear programming methods in the presence of noise, Mathematical Pro-gramming, 14 (1978), pp. 87-97. (Pubitemid 8591418)
15. H. Robbins and S. Monro, A stochastic approximation method, Annals Math. Stats., 22 (1951).
16. B. C. Russell, A. Torralba, K. P. Murphy, and W. T. Freeman, LabelMe: a database and web-based tool for image annotation, Intl. Journal of Computer Vision, 77 (2008), pp. 157-173.
17. D. Saad, ed., On-line learning in neural networks, Cambridge University Press, 1998.
18. M. Schmidt, G. Fung, and R. Rosales, Fast optimization methods for L1 regularization, in Proceedings of the 18th European Conference on Machine Learning, 2007, pp. 286-297.
19. S. Shalev-Shwartz, Y. Singer, and N. Srebro, Pegasos: primal estimated sub-gradient solver for SVM, in Proceedings of the 24th Intl. Conference on Machine learning, 2007, pp. 807-814.
20. J. C. Spall, Adaptive stochastic approximation by the simultaneous perturbation method, IEEE Transactions on Automatic Control, 45 (2000), pp. 1839-1853. (Pubitemid 32067073)
21. J. C. Spall and J. A. Cristion, Model-free control of nonlinear stochastic systems with discrete-time measurements, IEEE Transactions on Automatic Control, 43 (1998), pp. 1148-1210.
22. R. Tibshirani, Regression shrinkage and selection via the Lasso, Journal of the Royal Statistical Society, 58 (1996), pp. 267-288.
23. L. N. Trefethen and D. Bau, Numerical linear algebra, SIAM, 1997.
24. I. Wang and J. C. Spall, Stochastic optimization with inequality constraints using simultaneous perturbations and penalty functions, in Proc. 42nd IEEE Conf. Decision and Control, 2003.
25. M. H.Wright, Some properties of the Hessian of the logarithmic barrier function, Mathematical Programming, 67 (1994), pp. 265-295.
26. -, Ill-conditioning and computational error in interior methods for nonlinear programming, SIAM Journal on Optimization, 9 (1998), pp. 84-111.
27. A. Zheng, Statistical software debugging, PhD thesis, University of California, Berkeley, 2005.