Fast and efficient strategies for model selection of Gaussian support vector machine

IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics

Volumn 39, Issue 5, 2009, Pages 1292-1307

  Xu, Zongbei ^a   Dai, Mingwei ^a   Meng, Deyu ^a  

^a XI'AN JIAOTONG UNIVERSITY   (China)

Author keywords

Approximation methods; Data mining; Distance measurement; Kernel; Kernel methods; Optimized production technology; Pattern classification; Support vector machine (SVM); Support vector machines; Training; Visualization

Indexed keywords

APPROXIMATION METHODS; KERNEL; KERNEL METHODS; OPTIMIZED PRODUCTION TECHNOLOGY; PATTERN CLASSIFICATION; SUPPORT VECTOR MACHINE (SVM); TRAINING;

APPROXIMATION THEORY; DATA MINING; DATA VISUALIZATION; DISTANCE MEASUREMENT; GEARS; HEURISTIC ALGORITHMS; IMAGE RETRIEVAL; INFORMATION MANAGEMENT; MINING; MODEL STRUCTURES; MULTILAYER NEURAL NETWORKS; OPTIMIZATION; SEQUENTIAL SWITCHING; SET THEORY; VECTORS; VISUALIZATION;

SUPPORT VECTOR MACHINES;

EID: 67649403178     PISSN: 10834419     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSMCB.2009.2015672     Document Type: Article

References (51)

1. V. N. Vapnik, Estimation of Dependences Based on Empirical Data. Moscow, Russia: Nauka, 1979.
2. B. Schökopf, C. Burges, and A. Smola, Eds., "Introduction to support vector learning," in Advances in Kernel Methods-Support Vector Learning. Cambridge, MA: MIT Press, 1999, pp. 1-15.
3. V. N. Vapnik, Statistical Learning Theory. New York: Wiley, 1998.
4. C. J. C. Burges, "A tutorial on support vectormachines for pattern recognition," Data Mining Knowl. Discov., vol. 2, no. 2, pp. 121-167, Jun. 1998.
5. N. Cristianini and J. Shawe-Taylor, An Introduction to Support Vector Machines. Cambridge, U.K.: Cambridge Univ. Press, 2000.
6. C. Cortes and V. N. Vapnik, "Support vector networks," Mach. Learn. vol. 20, no. 3, pp. 273-297, Sep. 1995.
7. V. N. Vapnik, The Nature of Statistical Learning Theory. New York: Springer-Verlag, 1993.
8. V. N. Vapnik, "An overview of statistical learning theory," IEEE Trans. Neural Netw., vol. 10, no. 5, pp. 988-999, Sep. 1999.
9. M. Schmidt, "Identifying speakers with support vector networks," in Proc. 28th Symp. Interface, Sydney, Australia, 1996.
10. E. Osuna, R. Freund, and F. Girosi, "Support vector machines: Training and applications," MIT, Cambridge, MA, Tech. Rep. AIM-1602, 1997.
11. E. Osuna, R. Freund, and F. Girosi, "Training support vector machines: An application to face detection," in Proc. IEEE Comput. Soc. Conf. CVPR, 1997, pp. 130-136.
12. H. Drucker, D. Wu, and V. N. Vapnik, "Support vector machines for spam categorization," IEEE Trans. Neural Netw., vol. 10, no. 5, pp. 1048-1054, Sep. 1999.
13. K. Duan, S. Sathiya Keerthi, and A. N. Poo, "Evaluation of simple performance measures for tuning SVM hyperparameters," Neurocomputing vol. 51, pp. 41-59, Apr. 2003.
14. M. Stone, "Cross-validatory choice and assessment of statistical predictions," J. R. Stat. Soc., B, vol. 36, no. 1, pp. 111-147, 1974.
15. A. Elisseeff and M. Pontil, "Leave-one-out error and stability of learning algorithms with applications," in Learning Theory and Practice. Washington, DC: IOS Press, 2002.
16. T. Joachims, "The maximum-margin approach to learning text classifiers: Method, theory and algorithms," Ph.D. dissertation, Dept. Comput. Sci., Univ. Dortmund, Dortmund, Germany, 2000.
17. V. N. Vapnik and O. Chapelle, "Bounds on error expectation for support vector machine," in Advances in Large Margin ClassiBers. Cambridge, MA: MIT Press, 1999.
18. G. Wahba, Y. Lin, and H. Zhang, "GACV for support vector machines," in Advances in Large Margin ClassiBers. Cambridge, MA: MIT Press, 1999.
19. O. Chapelle, V. N. Vapnik, O. Bousquet, and S. Mukherjee, "Choosing multiple kernel parameters for support vector machines," Mach. Learn., vol. 46, no. 1-3, pp. 131-159, Jan. 2002.
20. S. Sathiya Keerthi, "Efficient tuning of SVM hyperparameters using radius/margin bound and iterative algorithms," IEEE Trans. Neural Netw., vol. 13, no. 5, pp. 1225-1229, Sep. 2002.
21. K. M. Chung, W. C. Kao, C. L. Sun, L. L. Wang, and C. J. Lin, "Radius margin bounds for support vector machines with the RBF kernel," Neural Comput., vol. 15, no. 11, pp. 2643-2681, Nov. 2003.
22. W. J. Wang, Z. B. Xu, W. Z. Lu, and X. Y. Zhang, "Determination of the spread parameter in the Gaussian kernel for classification and regression," Neurocomputing, vol. 55, no. 3, pp. 643-663, Oct. 2003.
23. N. E. Ayat, M. Cheriet, and C. Y. Suen, "Optimization of the SVM kernels using an empirical error minimization scheme," in Pattern Recognition with Support Vector Machines, vol. 2388, S. W. Lee and A. Verri, Eds. Berlin, Germany: Springer-Verlag, 2002, pp. 354-369.
24. N. Cristianini, C. Campbell, and J. Shawe-Taylor, "Dynamically adapting kernels in support vector machines," in Advances in Neural Information Processing Systems, vol. 11, M. S. Kearns, S. A. Solla, and D. A. Cohn, Eds. Cambridge, MA: MIT Press, 1999, pp. 204-210.
25. O. Chapelle and V.N. Vapnik, "Model selection for support vector machines," in Proc. 12th Conf. Adv. Neural Inf. Process. Syst., S. Solla, T. Leen, and K.-R. Müller, Eds. Cambridge, MA: MIT Press, 2000, vol. 12, pp. 230-236.
26. C. L. Huang and J. F. Dun, "A distributed PSO-SVM hybrid system with feature selection and parameter optimization," Appl. Soft Comput. vol. 8, no. 4, pp. 1381-1391, Sep. 2008.
27. C. Y. Sun and D. C. Gong, "Support vector machines with PSO algorithm for short-term load forecasting," in Proc. IEEE ICNSC, 2006, pp. 676-680.
28. X. Li, S. D. Yang, and J. X. Qi, "A new support vector machine optimized by improved particle swarm optimization and its application," J. Central South Univ. Technol., vol. 13, no. 5, pp. 568-572, Oct. 2006.
29. S. W. Lin, Z. J. Lee, S. C. Chen, and T. Y. Tseng, "Parameter determination of support vector machine and feature selection using simulated annealing approach," Appl. Soft Comput., vol. 8, no. 4, pp. 1505-1512, Sep. 2008.
30. S. Liu, C. Y. Jia, and H. Ma, "A new weighted support vector machine with GA-based parameter selection," in Proc. Int. Conf. Mach. Learn. Cybern., 2005, vol. 7, pp. 4351-4355.
31. C. L. Huang and C. J. Wang, "A GA-based feature selection and parameters optimization for support vector machines," Expert Syst. Appl., vol. 31, no. 2, pp. 231-240, 2006.
32. W. C. Hong, "Determining parameters of support vector machines by genetic algorithms - Applications to reliability prediction," Int. J. Oper. Res., vol. 2, no. 1, pp. 1-7, 2005.
33. M. Debruyne, M. Hubert, and J. Suykens, "Model selection for kernel regression using the influence function," J. Mach. Learn. Res., vol. 9, pp. 2377-2400, Oct. 2008.
34. M. Debruyne, "Robustness of censored depth quantiles, PCA and kernel based regression, with new tools for model selection," Ph.D. dissertation, Katholieke Universiteit Leuven, Leuven, Belgium, 2007. 91 p.
35. A. Smola, "Learning with kernels," Ph.D. dissertation, GMD First, Berlin, Germany, 1998.
36. S. Sathiya Keerthi and C.J. Lin, "Asymptotic behaviors of support vector machines with Gaussian kernel," Neural Comput., vol. 15, no. 7, pp. 1667-1689, Jul. 2003.
37. Y. Leung, J. S. Zhang, and Z. B. Xu, "Clustering by scale-space filtering," IEEE Trans. Pattern Anal. Mach. Intell., vol. 22, no. 12, pp. 1396-1410, Dec. 2000.
38. J. C. Platt, "Sequential minimal optimization: A fast algorithm for training support vector machines," Microsoft Res., Redmond, WA, Tech. Rep. MSR-TR-98-14, 1998.
39. J. C. Platt, "Fast training of support vector machines using sequential minimal optimization," in Advances in Kernel Methods: Support Vector Learning. Cambridge, MA: MIT Press, 1999.
40. T. Joachims, "Making large-scale SVM learning practical," in Advances in Kernel Methods-Support Vector Learning, B. Schökopf, C. Burges, and A. Smola, Eds. Cambridge, MA: MIT Press, 1999, ch. 11.
41. G. Rätsch, Benchmark Data Sets, 1999. [Online]. Available: http://ida.first.fhg.de/projects/bench/benchmarks.htm
42. G. Räatsch, T. Onoda, and K. R. Muller, "Soft margins for AdaBoost," Mach. Learn., vol. 42, no. 3, pp. 287-320, Mar. 2001.
43. S. Mika, G. Räatsch, J. Weston, B. Scholkopf, and K. R. Muller, "Fisher discriminant analysis with kernels," in Proc. Neural Netw. Signal Process. IX, H. Hu, J. Larsen, E. Wilson, and S. Douglas, Eds., 1999, pp. 41-48.
44. C. C. Chang and C. J. Lin, "IJCNN 2001 challenge: Generalization ability and text decoding," in Proc. IEEE IJCNN, 2001, pp. 1031-1036.
45. T. Hastie, S. Rosset, R. Tibishirani, and J. Zhu, "The entire regularization path for the support vector machine," J. Mach. Learn. Res., vol. 5, pp. 1391-1415, Dec. 2004.
46. 1 regularization path algorithm for generalized linear models," J. R. Stat. Soc. B, vol. 69, no. 4, pp. 659-677, Sep. 2007.
47. G. Kunapuli, K. Bennett, J. Hu, and J. S. Pang, "Classification model selection via bilevel programming," Optim. Methods Softw., vol. 23, no. 4, pp. 475-489, Aug. 2008.
48. K. Bennett, J. G. Hu, X. Y. Ji, G. Kunapuli, and J. S. Pang, "Model selection via bilevel optimization," in Proc. Int. Joint Conf. Neural Netw., 2006, pp. 1922-1929.
49. G. Kunapuli, K. Bennett, J. Hu, and J. S. Pang, "Bilevel model selection for support vector machines," in Proc. CRM, Lecture Notes 2008, vol. 45, pp. 129-158.
50. C. H. Goulden, Methods of Statistical Analysis, 2nd ed. New York: Wiley, 1956, pp. 50-55.
51. M. C. Ferris and T. S. Munson, "Interior-point methods for massive support vector machines," SIAM J. Optim., vol. 13, no. 3, pp. 783-804, 2002.