On the use of the spectral projected gradient method for support vector machines

Computational and Applied Mathematics

Volumn 28, Issue 3, 2009, Pages 327-364

  Cores, Debora ^a   Escalante, René ^a   González Lima, María ^a   Jiménez, Oswaldo ^a  

^a UNIVERSIDAD SIMÓN BOLÍVAR   (Venezuela)

Author keywords

Quadratic programming; Spectral projected gradient method; Support vector machines

Indexed keywords

CONSTRAINED OPTIMIZATION; GRADIENT METHODS; SUPPORT VECTOR MACHINES;

ALTERNATING PROJECTION ALGORITHM; COMPUTATIONAL COSTS; GRADIENT ALGORITHM; MATHEMATICAL EQUATIONS; OPTIMALITY CONDITIONS; OPTIMIZATION PROBLEMS; PROJECTED GRADIENT; SPECTRAL PROJECTED GRADIENT METHOD; SUPPORT VECTORS MACHINE; SVM-OPTIMIZATION;

QUADRATIC PROGRAMMING;

EID: 77950826983     PISSN: 22383603     EISSN: 18070302     Source Type: Journal    
DOI: 10.1590/S1807-03022009000300005     Document Type: Article

References (41)

1. Neural Information Processing Systems (NIPS) Conference: Feature Selection Challenge, 2003. http://www.nipsfsc.ecs.soton.ac.uk/datasets/.
2. IEEE World Congress on Computational Intelligence: Performance Prediction Challenge, 2006. http://www.modelselect.inf.ethz.ch/.
3. E. Birgin, J.M. Martínez and M. Raydan, Inexact spectral projected gradient methods on convex sets. IMA Journal of Numerical Analysis, 23 (2003), 539-559.
4. E.G. Birgin and M. Raydan, Robust stopping criteria for Dykstra's algorithm. SIAM Journal on Scientific Computing, 26 (2005), 1405-1414.
5. E.G. Birgin, J.M. Martínez and M. Raydan, Nonmonotone spectral projected gradient methods on convex sets. SIAM Journal on Optimization, 10 (2000), 1196-1211.
6. V. Blanz, B. Schölkopf, H. Bülthoffand, C. Burges, V.N. Vapnik and T. Vetter, Comparison of view-based object recognition algorithms using realistic 3d models. In: J.C. Vorbrüggen, C. von der Malsburg, W. von Seelen and B. Sendhoff, editors, Artificial Neural Networks, ICIANN'96, pages 251-256, Berlin 1996. Springer Lecture Notes in Computer Science, 1112 (1996).
7. th ACM Workshop on Computational Learning Theory, pages 144-152, Pittsburgh, PA, July 1992.
8. J.P. Boyle and R.L. Dykstra, A method for finding projections onto the intersections of convex sets in Hilbert spaces. Lecture Notes in Statistics, 37 (1986), 28-47.
9. C.J.C. Burges, A tutorial on support vector machines for pattern recognition. In: U. Fayyad, editor, Data Mining on Knowledge Discovery, pages 121-167. Kluwer Academic Publishers, Netherlands (1998).
10. C.J.C. Burges and B. Schölkopf, Improving the accuracy and speed of support vector machines. In: M. Jordan M. Mozer and T. Petsche, editors, Advances in Neural Information Processing Systems 9, pages 375-381. MIT Press, Cambridge, MA (1997).
11. W. Cheney and A. Goldstein, Proximity maps for convex sets. Proc. Amer. Math. Soc., 10 (1959), 448-450.
12. C. Cortes and V. Vapnik, Support vector networks. Machine Learning, 20 (1995), 1-25.
13. N. Cristianini and J.S. Taylor, An Introduction to Support Vector Machines and Other Kernel-Based Learning Methods. Cambridge University Press, United Kingdom (2000).
14. Y.H. Dai and R. Fletcher, New algorithms for singly linearly constrained quadratic programs subject to lower and upper bounds. Math. Program., 106 (2006), 403-421.
15. J.E. Dennis Jr. and R.B. Schnabel, Numerical Methods for Unconstrained Optimization and Nonlinear Equations. Prentice-Hall, Englewood Cliffs, NJ (1983).
16. R.L. Dykstra, An algorithm for restricted least-squares regression. J. Amer. Stat. Assoc., 78 (1983), 837-842.
17. R. Escalante and M. Raydan, Dykstra's algorithm for a constrained least-squares matrix problem. Num. Linear Algebra Appl., 3(6) (1996), 459-471.
18. R. Escalante and M. Raydan, Dykstra's algorithm for constrained least-squares rectangular matrix problems. Computers Math. Applic., 35(6) (1998), 73-79.
19. M. Ferris and T. Munson, Interior-point methods for massive support vector machines. SIAM Journal on Optimization, 13(3) (2003), 783-804.
20. J.C. Gilbert and J. Nocedal, Global convergence properties of conjugate gradient methods for optimization. SIAM J. Optim., 2, 21-42.
21. R.P. Gorman and T.J. Sejnowski, Analysis of hidden units in a layered network trained to classify sonar targets. Neural Networks, 1 (1988), 75-89.
22. S.P. Han, A successive projection method. Math. Program, 40 (1988), 1-14.
23. H. Lin and C. Lin, A study on sigmoid kerrnels for svm and the training of non-psd kernels by smo-type methods. Technical report, Department of Computer Science and Information Engineering, National Taiwan University, Taipei Taiwan (2003).
24. T. Joachims, Text categorization with support vector machines: Learning with many relevant features. In: C. Nédellec and C. Rouveirol, editors, Proceedings of the 10th European Conference on Machine Learning (ECML-98), pages 137-142. Springer, April 1998.
25. J.J. Júdice and F.M. Pires, Solution of large-scale separable strictly convex quadratic programs on the simplex. Linear Algebra and its applications, (1989), 215-220.
26. J. Nocedal and S.J. Wright, Numerical Optimization. Springer, New York (1999).
27. E.E. Osuna, R. Freund and F. Girosi, Support Vector Machines: Training and Applications. Technical Report A.I. Memo No. 1602, C.B.C.L. Paper No. 144, Massachusetts Institute of Technology, Cambridge, MA, USA (1997).
28. E.E. Osuna, R. Freund and F. Girosi, Training support vector vector machines: An application to face detection. In: IEEE Conference on Computer Vision and Pattern Recognition, (1997), 130-136.
29. P.M. Pardalos and N. Kovoor, An algorithm for a singly constrained class of quadratic programs subject to upper and lower bounds. Mathematical Programming, 46.
30. M. Raydan, The Barzilai and Borwein gradient method for the large scale unconstrained minimization problem. SIAM J. Opt., 7 (1997), 26-33.
31. B.D. Riply, Neural networks and related methods for classification (with discussion). J. Royal Statistical Soc. Series B, 56 (1994), 409-456.
32. M. Schmidt, Identifying speaker with support vector networks. In: Interface'96 Proceedings. Sydney, Australia (1996).
33. B. Schölkopf, C. Burges and V.N. Vapnik, Extracting support data for a given task. In: U.M. Fayyad and R. Uthurusamy, editors, Proceedings First International Conference on Knowledge Discovery and Data Mining. AAAI Press, Menlo Park, CA (1995).
34. B. Schölkopf, C. Burges and V.N. Vapnik, Incorporating invariances in support vector learning machines. In: J.C. Vorbrüuggen C. von der Malsburg, W. von Seelen and B. Sendhoff, editors, Artificial Neural Networks, ICIANN'96, pages 47-52, Berlin 1996. Springer Lecture Notes in Computer Science, 1112 (1996).
35. T. Serafini, G. Zanghirati and L. Zanni, Gradient projection methods for quadratic programs and applications in training support vector machines. Optimization Methods and Software, 20 (2003), 353-378.
36. D.F. Shanno and K.H. Phua, Remark on algorithm 500: Minimization of unconstrained multivariate functions. ACM Trans. Math. Software, 6 (1980), 618-622.
37. K. Sung and T. Poggio, Example-based learning for view-based human face detection. Technical Report A.I. Memo No. 1521, C.B.C.L. Paper No. 112, Massachusetts Institute of Technology, Cambridge, MA, USA (1994).
38. V. Vapnik, The Nature of Statistical Learning Theory. Springer-Verlag, New York (1995).
39. J. von Neumann, Functional operators, Vol. II. The geometry of orthogonal spaces. Annals of Math. Studies, 22 (1950). Princeton University Press.
40. J. Weston, A. Gammerman, M.O. Stitson, V. Vapnik, V. Vork and C. Watkins, Density estimation using support vector machines. Technical Report CSD-TR-97-23, Royal Holloway College (1997).
41. L. Zanni, T. Serafini and G. Zanghirati, Parallel software for training large scale support vector machines on multiprocessor systems. Journal of Machine Learning Research, 7 (2006), 1467-1492.