Limited Stochastic Meta-Descent for Kernel-Based Online Learning

Neural Computation

Volumn 21, Issue 9, 2009, Pages 2667-2686

  He, Wenwu ^a  

^a FUJIAN UNIVERSITY OF TECHNOLOGY   (China)

Author keywords

[No Author keywords available]

Indexed keywords

E-LEARNING;

ADAPTIVE LEARNING RATES; DATA SET; KERNEL ALGORITHMS; ONLINE LEARNING; PERFORMANCE; STEP SIZE; STEP-SIZE ADAPTATIONS; STOCHASTIC META DESCENTS; SUPPORT VECTOR LEARNING;

STOCHASTIC SYSTEMS;

ALGORITHM; ARTICLE; ARTIFICIAL NEURAL NETWORK; AUTOMATED PATTERN RECOGNITION; COMPUTER SIMULATION; DECISION SUPPORT SYSTEM; HUMAN; LEARNING; ONLINE SYSTEM; PHYSIOLOGY;

ALGORITHMS; COMPUTER SIMULATION; DECISION SUPPORT TECHNIQUES; HUMANS; LEARNING; NEURAL NETWORKS (COMPUTER); ONLINE SYSTEMS; PATTERN RECOGNITION, AUTOMATED;

EID: 70349251856     PISSN: 08997667     EISSN: 1530888X     Source Type: Journal    
DOI: 10.1162/neco.2009.07-08-809     Document Type: Article

References (24)

1. Bray, M., Koller-Meier, E., Müller, P., Schraudolph, N. N., & Van Gool, L. (2005). Stochastic optimization for high-dimensional tracking in dense range maps. Vision, Image and Signal Processing, 152(4), 501–512.
2. Bray, M., Koller-Meier, E., Schraudolph, N. N., & Van Gool, L. (2007). Fast stochastic optimization for articulated structure tracking. Image and Vision Computing, 25(3), 352–364.
3. Cesa-Bianchi, N., Long, P., & Warmuth, M. (1996). Worst-case quadratic loss bounds for on-line prediction of linear functions by gradient descent. IEEE Transactions on Neural Networks, 7(2), 604–619.
4. Cheng, L., Vishwanathan, S. V. N., Schuurmans, D., Wang, S., & Caelli, T. (2006). Implicit online learning with kernels. In B. Schölkopf, J. Platt, & T. Hoffman (Eds.), Advances in neural information processing systems, 19 (pp. 256–249). Cambridge, MA: MIT Press.
5. Günter, S., Schraudolph, N. N., & Vishwanathan, S. V. N. (2007). Fast iterative kernel principal component analysis. Journal of Machine Learning Research, 8, 1893– 1918.
6. He, W., & Jiang, H. (2008). Implicit update vs. explicit update. In IEEE International Joint Conference on Neural Networks-IJCNN 2008 (pp. 3441–3447). Piscataway, NJ: IEEE Press.
7. Jacobs, R. A. (1988). Increased rates of convergence through learning rate adaptation. Neural Networks, 1, 295–307.
8. Kim, K. I., Franz, M. O., & Schölkopf, B. (2005). Iterative kernel principal component analysis for image modeling. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(9), 1351–1366.
9. Kivinen, J., Smola, A. J., & Williamson, R. C. (2002). Online learning with kernels. In T. G. Dietterich, S. Becker, & Z. Ghahramani (Eds.), Advances in neural information processing systems, 14 (pp. 785–792). Cambridge, MA: MIT Press.
10. Kivinen, J., Smola, A. J., & Williamson, R. C. (2004). Online learning with kernels. IEEE Transactions on Signal Processing, 52(8), 2165–2176.
11. Kivinen, J., & Warmuth, M. K. (1997). Exponentiated gradient versus gradient descent for linear predictors. Information and Computation, 132(1), 1–64.
12. Schölkopf, B., Smola, A. J., Williamson, R. C., & Bartlett, P. L. (2000). New support vector algorithms. Neural Computation, 12(5), 1207–1245.
13. Schraudolph, N. N. (1999). Local gain adaptation in stochastic gradient descent. In Proc. Intl. Conf. Artificial Neural Networks (pp. 569–574). London: IEE.
14. Schraudolph, N. N. (2002). Fast curvature matrix-vector products for second-order gradient descent. Neural Computation, 14(7), 1723–1738.
15. Schraudolph, N. N., & Giannakopoulos, X. (2000). Online independent component analysis with local learning rate adaptation. In S. A. Solla, T. K. Leen, & K.-R. Müller (Eds.), Advances in neural information processing systems, 12 (pp. 789–795). Cambridge, MA: MIT Press.
16. Schraudolph, N. N., Yu, J., & Aberdeen, D. (2006). Fast online policy gradient learning with SMD gain vector adaptation. In Y. Weiss, B. Schölkopf, & J. Platt (Eds.), Advances in neural information processing systems, 18 (pp. 1185–1192). Cambridge, MA: MIT Press.
17. Smale, S., & Yao, Y. (2006). Online learning algorithms. Foundations of Computational Mathematics, 6(2), 145–170.
18. Smola, A. J., & Schölkopf, B. (2004). A tutorial on support vector regression. Statistics and Computing, 14, 199–222.
19. Sutton, R. S. (1981). Adaptation of learning rate parameters. Available online at http://www.cs.ualberta.ca/sutton/papers/sutton-81.pdf.
20. Sutton, R. S. (1992). Gain adaptation beats least squares? In Proceedings of the 7th Yale Workshop on Adaptive and Learning Systems (pp. 161–166). New Haven, CT: Yale University.
21. Vishwanathan, S. V. N., Schraudolph, N. N., Schmidt, M., & Murphy K. (2006). Accelerated training conditional random fields with stochastic gradient methods. In Proc. Intl. Conf. Machine Learning (pp. 969–976). New York: ACM.
22. Vishwanathan, S. V. N., Schraudolph, N. N., & Smola, A. J. (2006). Step size adaptation in reproducing kernel Hilbert space. Journal of Machine Learning Research, 7, 1107–1133.
23. Warmuth, M. K., & Jagota, A. (1998). Continuous and discrete time nonlinear gradient descent: Relative loss bounds and convergence. In Electronic Proceedings of the Fifth International Symposium on Artificial Intelligence and Mathematics. Available online at http://rutcor.rutgers.edu/amai.
24. Ying, Y., & Pontil, M. (2007, April 25). Online gradient descent learning algorithms. Foundations of Computational Mathematics. Online version.