Framelet Kernels with applications to support vector regression and regularization networks

IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics

Volumn 40, Issue 4, 2010, Pages 1128-1144

  Zhang, Wei Feng ^a   Dai, Dao Qing ^a   Yan, Hong ^b,c  

^a SUN YAT SEN UNIVERSITY   (China)

^b CITY UNIVERSITY OF HONG KONG   (Hong Kong)

^c UNIVERSITY OF SYDNEY   (Australia)

Author keywords

Framelet; kernel; multiresolution analysis (MRA); regularization networks (RNs); support vector regression (SVR)

Indexed keywords

APPROXIMATE FUNCTION; APPROXIMATION PROPERTIES; FEATURE SPACE; FRAMELETS; GAUSSIAN KERNELS; HARD PROBLEMS; INPUT SPACE; KERNEL FUNCTION; KERNEL METHODS; LEARNING MACHINES; MULTI-SCALE STRUCTURES; OVERFITTING; REGRESSION PROBLEM; REGULARIZATION NETWORKS; SAMPLE DATA; SPARSE DATA; SUPPORT VECTOR REGRESSIONS; TIGHT WAVELET FRAMES;

MULTIRESOLUTION ANALYSIS; VECTORS;

REGRESSION ANALYSIS;

ALGORITHM; ARTICLE; ARTIFICIAL NEURAL NETWORK; COMPUTER SIMULATION; REGRESSION ANALYSIS; SIGNAL PROCESSING; STATISTICAL ANALYSIS; STATISTICAL MODEL;

ALGORITHMS; COMPUTER SIMULATION; DATA INTERPRETATION, STATISTICAL; MODELS, STATISTICAL; NEURAL NETWORKS (COMPUTER); REGRESSION ANALYSIS; SIGNAL PROCESSING, COMPUTER-ASSISTED;

EID: 77954762258     PISSN: 10834419     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSMCB.2009.2034993     Document Type: Article

References (44)

1. J. Shawe-Taylor and N. Cristianini, Kernel Methods for Pattern Analysis. Cambridge, U.K.: Cambridge Univ. Press, 2004.
2. A. Smola and B. Schölkopf, "A tutorial on support vector regression," Stat. Comput., vol.14, no.3, pp. 199-222, Aug. 2004.
3. Y. J. Lee and S. Y. Huang, "Reduced support vector machines: A statistical theory," IEEE Trans. Neural Netw., vol.18, no.1, pp. 1-13, Jan. 2007.
4. J. J. Pan, J. T. Kwok, Q. Yang, and Y. Q. Chen, "Multidimensional vector regression for accurate and low-cost location estimation in pervasive computing," IEEE Trans. Knowl. Data Eng., vol.18, no.9, pp. 1181-1193, Sep. 2006.
5. H. Takeda, S. Farsiu, and P. Milanfar, "Kernel regression for image processing and reconstruction," IEEE Trans. Image Process., vol.16, no.2, pp. 349-366, Feb. 2007.
6. Y. Tan and J. Wang, "A support vector machine with a hybrid kernel and minimal Vapnik-Chervonenkis dimension," IEEE Trans. Knowl. Data Eng., vol.16, no.4, pp. 385-395, Apr. 2004.
7. V. N. Vapnik, The Nature of Statistical Learning Theory. New York: Springer-Verlag, 1995.
8. V. N. Vapnik, Statistical Learning Theory. New York: Wiley, 1998.
9. T. Poggio and F. Girosi, "Networks for approximation and learning," Proc. IEEE, vol.78, no.9, pp. 1481-1497, Sep. 1990.
10. F. Girosi, M. Jones, and T. Poggio, "Regularization theory and neural networks architectures," Neural Comput., vol.7, no.2, pp. 219-269, Mar. 1995.
11. T. Evgeniou, M. Pontil, and T. Poggio, "Regularization networks and support vector machines," Adv. Comput. Math., vol.13, no.1, pp. 1-50, Apr. 2000.
12. A. J. Smola, B. Schölkopf, and K. R. Müller, "The connection between regularization operators and support vector kernels," Neural Netw., vol.11, no.4, pp. 637-649, Jun. 1998.
13. G. Ben-Artzi, H. Hel-Or, and Y. Hel-Or, "The gray-code filter kernels," IEEE Trans. Pattern Anal. Mach. Intell., vol.29, no.3, pp. 382-393, Mar. 2007.
14. F. Camastra and A. Verri, "A novel kernel method for clustering," IEEE Trans. Pattern Anal. Mach. Intell., vol.27, no.5, pp. 801-805, May 2005.
15. S. Chen, X. X. Wang, X. Hong, and C. J. Harris, "Kernel classifier construction using orthogonal forward selection and boosting with fisher ratio class separability measure," IEEE Trans. Neural Netw., vol.17, no.6, pp. 1652-1656, Nov. 2006. (Pubitemid 44824278)
16. S. K. Zhou and R. Chellappa, "From sample similarity to ensemble similarity: Probabilistic distance measures in reproducing kernel Hilbert space," IEEE Trans. Pattern Anal. Mach. Intell., vol.28, no.6, pp. 917-929, Jun. 2006.
17. W. S. Chen, P. C. Yuen, J. Huang, and D. Q. Dai, "Kernel machinebased one-parameter regularized Fisher discriminant method for face recognition," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol.35, no.4, pp. 659-669, Aug. 2005. (Pubitemid 41655573)
18. O. L. Mangasarian and E. W. Wild, "Nonlinear knowledge in kernel approximation," IEEE Trans. Neural Netw., vol.18, no.1, pp. 300-306, Jan. 2007.
19. D. N. Zheng, J. X. Wang, and Y. N. Zhao, "Non-flat function estimation with a multi-scale support vector regression," Neurocomputing, vol.70, no.1-3, pp. 420-429, Dec. 2006.
20. L. V. Ling, "Finding numerical derivatives for unstructured and noisy data by multiscale kernels," SIAM J. Numer. Anal., vol.44, no.4, pp. 1780-1800, 2006.
21. R. Opfer, "Multiscale kernels," Adv. Comput. Math., vol.25, no.4, pp. 357-380, Nov. 2006.
22. U. Amato, A. Antoniadis, and M. Pensky, "Wavelet kernel penalized estimation for non-equispaced design regression," Stat. Comput., vol.16, no.1, pp. 37-55, Mar. 2006.
23. A. Antoniadis, E. Paparoditis, and T. Sapatinas, "A functional waveletkernel approach for time series prediction," J. R. Stat. Soc., Ser. B Stat. Methodol., vol.68, no.5, pp. 837-857, Mar. 2006.
24. A. Rakotomamonjy, X. Mary, and S. Canu, "Non-parametric regression with wavelet kernels," Appl. Stoch. Models Bus. Ind., vol.21, no.2, pp. 153-163, Mar. 2005.
25. L. Zhang, W. Zhou, and L. Jiao, "Wavelet support vector machine," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol.34, no.1, pp. 34-39, Feb. 2004.
26. J. B. Gao, C. J. Harris, and S. R. Gunn, "On a class of support vector kernels based on frames in function Hilbert spaces," Neural Comput., vol.13, no.9, pp. 1975-1994, Sep. 2001.
27. A. Rakotomamonjy and S. Canu, "Frames, reproducing kernels, regularization and learning," J. Mach. Learn. Res., vol.6, pp. 1485-1515, Dec. 2005.
28. R. Opfer, "Tight frame expansions of multiscale reproducing kernels in Sobolev spaces," Appl. Comput. Harmon. Anal., vol.20, no.3, pp. 357-374, 2006.
29. R. Gribonval and M. Nielsen, "On approximation with spline generated framelets," Constr. Approx., vol.20, no.2, pp. 207-232, 2004.
30. O. Amrani, A. Z. Averbuch, and V. A. Zheludev, Symmetric Interplatory Framelets and Their Erasure Recovery Properties, vol.4105, Lecture Notes in Computer Science. Berlin, Germany: Springer-Verlag, 2006, pp. 643-649.
31. L. X. Shen, M. Papadakis, I. A. Kakadiaris, I. Konstantinidis, D. Kouri, and D. Hoffman, "Image denoising using a tight frame," IEEE Trans. Image Process., vol.15, no.5, pp. 1254-1263, May 2006.
32. I. Daubechies, B. Han, A. Ron, and Z. Shen, "Framelets: MRA-based constructions of wavelet frames," Appl. Comput. Harmon. Anal., vol.124, pp. 44-88, 2003.
33. W. F. Zhang, D. Q. Dai, and H. Yan, On a New Class of Framelet Kernels for Support Vector Regression and Regularization Networks, vol.4426, Lecture Notes in Artificial Intelligence. Berlin, Germany: Springer-Verlag, 2007, pp. 355-366.
34. O. Christensen, An Introduction to Frames and Riesz Bases. Boston, MA: Birkhäuser, 2003, ser. Applied and Numerical Harmonic Analysis.
35. K. R. Müller, S. Mika, G. Rätsch, K. Tsuda, and B. Schölkopf, "An introduction to kernel-based learning algorithms," IEEE Trans. Neural Netw., vol.12, no.2, pp. 181-201, Mar. 2001.
36. B. Schökopf, S. Mika, C. Burges, P. Knirsch, K. R. Müller, G. Räsch, and A. J. Smola, "Input space versus feature space in kernel-based methods," IEEE Trans. Neural Netw., vol.10, no.5, pp. 1000-1017, Sep. 1999.
37. A. Ron and Z. Shen, "Construction of compactly supported affine frames in L2(R)," Math. Comput., vol.67, pp. 191-207, 1998.
38. I. Daubechies, Ten Lectures onWavelets. Philadelphia, PA: SIAM, 1992.
39. C. C. Chang and C. J. Lin. (2001). LIBSVM: A Library for Support Vector Machines. [Online]. Available: http://www.csie.ntu.edu.tw/~cjlin/ libsvm
40. O. Chapelle, V. Vapnik, O. Bousquet, and S. Mukherjee, "Choosing multiple parameters for support vector machines," Mach. Learn., vol.46, no.1-3, pp. 131-159, Jan. 2002. (Pubitemid 34129966)
41. F. Cuker and S. Smale, "Best choices for regularization parameters in learning theory: On the bias-variance problem," Found. Comput. Math., vol.2, no.4, pp. 413-428, 2002.
42. T. Hastie, S. Rosset, R. Tibshirani, and J. Zhu, "The entire regularization path for the support vector machine," J. Mach. Learn. Res., vol.5, pp. 1391-1415, Dec. 2004.
43. G. Wang, D. Y. Yeung, and F. H. Lochovsky, "A kernel path algorithm for support vector machines," in Proc. 24th ICML, 2007, pp. 951-958.
44. A. Antoniadis, J. Bigot, and T. Sapatinas, "Wavelet estimators in nonparametric regression: A comparative simulation study," J. Stat. Softw., vol.6, no.6, pp. 1-83, Jun. 2001.