On-line learning for very large data sets

Applied Stochastic Models in Business and Industry

Volumn 21, Issue 2, 2005, Pages 137-151

  Bottou, Léon ^a   Cun, Yann Le ^b  

^a NEC LABORATORIES AMERICA   (United States)

^b NEW YORK UNIVERSITY   (United States)

Author keywords

Convergence speed; Learning; Online learning; Stochastic optimization

Indexed keywords

DATA ACQUISITION; LEARNING ALGORITHMS; ONLINE SYSTEMS; PATTERN RECOGNITION; RANDOM PROCESSES;

CONVERGENCE SPEED; LEARNING; ONLINE LEARNING; STOCHASTIC OPTIMIZATION;

LEARNING SYSTEMS;

EID: 17444425307     PISSN: 15241904     EISSN: None     Source Type: Journal    
DOI: 10.1002/asmb.538     Document Type: Article

References (12)

1. Vapnik VN, Chervonenkis A. Theory of Pattern Recognition. Nauka: Moscow, 1974 (in Russian). (German translation: Theorie der Zeichenerkennung. Akademie Verlag: Berlin, 1979).
2. Boser B, Guyon I, Vapnik V. A training algorithm for optimal margin classifiers. In Proceedings of the Fifth Annual Workshop on Computational Learning Theory, vol. 5, Pittsburgh, Pennsylvania, USA, 1992; 144-152.
3. Drucker H, Schapire R, Simard P. Improving performance in neural networks using a boosting algorithm. In Advances in Neural Information Processing Systems 5, Hanson SJ, Cowan JD, Giles CL (eds). Morgan Kaufmann: San Mateo, CA, 1993; 42-49.
4. Freund Y, Schapire RE. Experiments with a new boosting algorithm. In Machine Learning: Proceedings of the Thirteenth International Conference, Bari, Italy, 1996; 148-156.
5. Bottou L, Cortes C, Denker JS, Drucker H, Guyon I, Jackel LD, Le Cun Y, Muller UA, Säckinger E, Simard P, Vapnik VN. Comparison of classifier methods: a case study in handwritten digit recognition. In Proceedings of the 13th International Conference on Pattern Recognition, Jerusalem, 1994.
6. Bottou L, Murata N. Stochastic approximations and efficient learning. In The Handbook of Brain Theory and Neural Networks (2nd edn), Arbib MA (ed.). The MIT Press: Cambridge, MA, 2002.
7. Dennis J, Schnabel RB. Numerical Methods For Unconstrained Optimization and Nonlinear Equations. Prentice-Hall, Inc.: Englewood Cliffs, NJ, 1983.
8. Benveniste A, Metivier M, Priouret P. Adaptive Algorithms and Stochastic Approximations. Springer: Berlin, New York, 1990.
9. Bottou L. Online algorithms and stochastic approximations. In Online Learning and Neural Networks, Saad D (ed.). Cambridge University Press: Cambridge, UK, 1998.
10. Le Cun Y, Bottou L, Orr OB, Müller KR. Efficient backprop. In Neural Networks, Tricks of the Trade, Lecture Notes in Computer Science, vol. 1524. Orr OB, Müller KR (eds). Springer: Berlin, 1998.
11. Murata N, Amari S. Statistical analysis of learning dynamics. Signal Processing 1999; 74(1):3-28.
12. Mann HB, Wald A. On stochastic limit and order relationships. Annals of Mathematical Statistics 1943; 14:217-226.