Tuning-free step-size adaptation

ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings

Volumn , Issue , 2012, Pages 2121-2124

  Mahmood, Ashique Rupam ^a   Sutton, Richard S ^a   Degris, Thomas ^a   Pilarski, Patrick M ^a  

^a UNIVERSITY OF ALBERTA   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

3D HAND TRACKING; ADAPTATION METHODS; APPLICATION AREA; DOMAIN KNOWLEDGE; GRADIENT DESCENT; INCREMENTAL LEARNING; INPUT FEATURES; LMS ALGORITHMS; MANUAL INTERVENTION; PREDICTION PROBLEM; STEP SIZE;

LEARNING ALGORITHMS; REINFORCEMENT LEARNING; SIGNAL PROCESSING;

PARAMETER ESTIMATION;

EID: 84867615954     PISSN: 15206149     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICASSP.2012.6288330     Document Type: Conference Paper

References (16)

1. Almeida, L. B., Langlois, T., Amaral, J. D., and Plakhov, A. (1998). Parameter adaptation in stochastic optimization. In Saad, D. (Ed.), On-Line Learning in Neural Networks, pp. 111-134. Cambridge University Press.
2. Benveniste, A., Métivier, M., and Priouret, P. (1990). Adaptive algorithms and stochastic approximations. Springer-Verlag.
3. Bordes, A., Bottou, L., and Gallinari, P. (2009). SGD-QN: Careful quasi-Newton stochastic gradient descent. Journal of Machine Learning Research, 10:1737-1754.
4. Bray, M., Koller-Meier, E., Müller, P., Gool, L. V., and Schraudolph, N. N. (2004). 3D hand tracking by rapid stochastic gradient descent using a skinning model. In Proc. Intl. Conf. Artificial Neural Networks, pp. 59-68.
5. Buttfield, A., Ferrez, P. W., and Millán, R. J. del. (2006). Towards a robust BCI: Error potentials and online learning. IEEE Trans. Neural Sys. Rehab. Eng., 14:164-168.
6. Jacobs, R. A. (1988). Increased rates of convergence through learning rate adaptation. Neural Networks, 1(4):295-307.
7. Kushner H. J., and Yang, J. (1995). Analysis of adaptive step size SA algorithms for parameter tracking. IEEE Trans. Autom. Control, 40:1403-1410.
8. Mahmood A. (2010). Automatic step-size adaptation in incremental supervised learning. Master's thesis, Department of Computing Science, University of Alberta, Edmonton, AB T6G 2E8.
9. Murata, N.,Müller, K. R., Ziehe, A., and Amari, S. (1996). Adaptive on-line learning in changing environments. In: Advances in Neural Information Processing Systems 9, pp. 312-318. MIT press.
10. RLAI (2010). daylong04.crtrlog.gz. Critterbot. Retrieved June 1, 2010, from http://critterbot.rl-community.org/logs.
11. Schaal, S., Atkeson, C., and Vijayakumar, S. (2001). Scalable techniques from nonparametric statistics for real-time robot learning. Applied Intelligence, 17(1):49-60.
12. Schraudolph, N. N. (1999). Local gain adaptation in stochastic gradient descent. In: Proc. Intl. Conf. Artificial Neural Networks, pp. 569-574.
13. Sompolinsky, H., Barkai, N., and Seung, H. S. (1995). Online learning of dichotomies: Algorithms and learning curves. In Neural networks: The statistical mechanics perspective. Proceedings of the CTP-PBSRI Joint Workshop on Theoretical Physics, pp. 105-130. Singapore: World Scientific.
14. Sutton, R. S. (1981). Adaptation of learning rate parameters. In: Goal Seeking Components for Adaptive Intelligence: An Initial Assessment, by A. G. Barto and R. S. Sutton. Air Force Wright Aeronautical Laboratories Technical Report AFWAL-TR-81-1070. Wright-Patterson Air Force Base, Ohio 45433.
15. Sutton, R. S. (1992a). Adapting bias by gradient descent: An incremental version of delta-bar-delta. In Proc. 10th National Conference on Artificial Intelligence, pp. 171-176.
16. Sutton, R. S. (1992b). Gain adaptation beats least squares? In Proc. of the 7th Yale Workshop on Adaptive and Learning Systems, pp. 161-166.