Gradient-based learning applied to document recognition

Proceedings of the IEEE

Volumn 86, Issue 11, 1998, Pages 2278-2323

  LeCun, Yann ^a,c   Bottou, Léon ^a   Bengio, Yoshua ^b   Haffner, Patrick ^a  

^a AT AND T LABS RESEARCH   (United States)

^b UNIVERSITÉ DE MONTRÉAL   (Canada)

^c IEEE   (United States)

Author keywords

Convolutional neural networks; Document recognition; Finite state transducers; Gradient based learning; Graph transformer networks; Machine learning; Neural networks; Optical character recognition (OCR)

Indexed keywords

BACKPROPAGATION; DIGITAL SIGNAL PROCESSING; LEARNING ALGORITHMS; LEARNING SYSTEMS; MULTILAYER NEURAL NETWORKS;

CONVOLUTIONAL NEURAL NETWORKS; GRADIENT-BASED LEARNING ALGORITHMS; GRAPH TRANSFORMER NETWORKS (GTN);

OPTICAL CHARACTER RECOGNITION;

EID: 0032203257     PISSN: 00189219     EISSN: None     Source Type: Journal    
DOI: 10.1109/5.726791     Document Type: Article

References (121)

1. R. O. Duda and P. E. Hart, Pattern Classification and Scene Analysis. New York: Wiley, 1973.
2. Y. LeCun, B. Boser, J. S. Denker, D. Henderson, R. E. Howard, W. Hubbard, and L. D. Jackel, "Backpropagation applied to handwritten zip code recognition," Neural Computation, vol. 1, no. 4, pp. 541-551, Winter 1989.
3. S. Seung, H. Sompolinsky, and N. Tishby, "Statistical mechanics of learning from examples," Phys. Rev. A, vol. 45, pp. 6056-6091, 1992.
4. V. N. Vapnik, E. Levin, and Y. LeCun, "Measuring the vedimension of a learning machine," Neural Computation, vol. 6, no. 5, pp. 851-876, 1994.
5. C. Cortes, L. Jackcl, S. Solla, V. N. Vapnik, and J. Denker, "Learning curves: Asymptotic values and rate of convergence," in Advances in Neural Information Processing Systems 6, J. D. Cowan, G. Tesauro, and J. Alspector, Eds. San Mateo, CA: Morgan Kaufmann, 1994, pp. 327-334.
6. V. N. Vapnik, The Nature of Statistical Learning Theory. New York: Springer, 1995.
7. _, Statistical Learning Theory. New York: Wiley, 1998.
8. W. H. Press, B. P. Flannery, S. A. Teukolsky, and W. T. Vetterling, Numerical Recipes: The Art of Scientific Computing. Cambridge, UK: Cambridge Univ., 1986.
9. S. I. Amari, "A theory of adaptive pattern classifiers," IEEE Trans. Electron. Comput., vol. EC-16, pp. 299-307, 1967.
10. Y. Tsypkin, Adaptation and Learning in Automatic Systems New York: Academic, 1971.
11. _, Foundations of the Theory of Learning Systems. New York: Academic, 1973.
12. M. Minsky and O. Selfridge, "Learning in random nets," in Proc. 4th London Symp. Information Theory, pp. 335-347, 1961.
13. D. H. Ackley, G. E. Hinton, and T. J. Sejnowski, "A learning algorithm for Boltzmann machines," Cognitive Sci., vol. 9, pp. 147-169, 1985.
14. G. E. Hinton and T. J. Sejnowski, "Learning and relearning in Boltzmann machines," in Parallel Distributed Processing: Explorations in the Microstructure of Cognition. volume I: Foundations, D. E. Rumelhart and J. L. McClelland, Eds. Cambridge, MA: MIT, 1986.
15. D. E. Rumelhart, G. E. Hinton, and R. J. Williams, "Learning internal representations by error propagation," in Parallel Distributed Processing: Explorations in the Microstructure of Cognition, vol. I. Cambridge, MA: Bradford Books, 1986, pp. 318-362,
16. A. E. Bryson, Jr. and Y.-C. Ho, Applied Optimal Control. London, UK: Blaisdell, 1969.
17. Y. LeCun, "A learning scheme for asymmetric threshold networks," in Proc. Cognitiva '85, Paris, France, 1985, pp. 599-604.
18. _, "Learning processes in an asymmetric threshold network," in Disordered Systems and Biological Organization, E. Bienenstock, F. Fogelman-Soulië, and G. Weisbuch, Eds. Les Houches, France: Springer-Verlag, 1986, pp. 233-240.
19. D. B. Parker, "Learning-logic," Sloan School Manage., MIT, Cambridge, MA, Tech. Rep., TR-47, Apr. 1985.
20. Y. LeCun, Modéles Connexionnistes de l'Apprentissage (Connectionist Learning Models), Ph.D. dissertation, Université P. et M. Curie (Paris 6), June 1987.
21. _, "A theoretical framework for back-propagation," in Proc. 1988 Connectionist Models Summer School, D. Touretzky, G. Hinton, and T. Sejnowski, Eds. Pittsburgh, PA: CMU, Morgan Kaufmann, 1988, pp. 21-28.
22. L. Bottou and P. Gallinari, "A framework for the cooperation of learning algorithms," in Advances in Neural Information Processing Systems, vol. 3, D. Touretzky and R. Lippmann, Eds. Denver, CO: Morgan Kaufmann, 1991.
23. C. Y. Suen, C. Nadal, R. Legault, T. A. Mai, and L. Lam, "Computer recognition of unconstrained handwritten numerals," Proc. IEEE, vol. 80, pp. 1162-1180, July 1992.
24. S. N. Srihari, "High-performance reading machines," Proc. IEEE., vol. 80, pp. 1120-1132, July 1992.
25. Y. LeCun, L. D. Jackel, B. Boser, J. S. Denker, H. P. Graf, I. Guyon, D. Henderson, R. E. Howard, and W. Hubbard, "Handwritten digit recognition: Applications of neural net chips and automatic learnins," IEEE Trans. Commun., vol. 37, pp. 41-46, Nov. 1989.
26. J. Keeler, D. Rumelhart, and W. K. Leow, "Integrated segmentation and recognition of hand-printed numerals," in Neural Information Processing Systems, R. P. Lippmann, J. M. Moody, and D. S. Touretzky, Eds. San Mateo, CA: Morgan Kaufmann, vol. 3, pp. 557-563, 1991.
27. O. Matan, C. J. C. Burges, Y. LeCun, and J. S. Denker, "Multidigit recognition using a space displacement neural network," vol. 4, in Neural Information Processing Systems, J. M. Moody, S. J. Hanson, and R. P. Lippman, Eds. San Mateo, CA: Morgan Kaufmann, 1992.
28. L. R. Rabiner, "A tutorial on hidden Markov models and selected applications in speech recognition," Proc. IEEE, vol. 77, pp. 257-286, Feb. 1989.
29. H. A. Bourland and N. Morgan, Connectionist Speech Recognition: A Hybrid Approach. Boston: Kluwer, 1994.
30. D. H. Hubel and T. N. Wiesel, "Receptive fields, binocular in-teraction, and functional architecture in the cat's visual cortex," J. Physiology (London), vol. 160, pp. 196-154, 1962.
31. K. Fukushima, "Cognition: A self-organizing multilayered neural network," Biological Cybern., vol. 20, pp. 121-136, 1975.
32. K. Fukushima and S. Miyake, "Neocognitron: A new algorithm for pattern recognition tolerant of deformations and shifts in position," Pattern Recognit., vol. 15, no. 6, pp. 455-469, Nov. 1982.
33. M. C. Mozer, The Perception of Multiple Objects: A Connectionist Approach. Cambridge, MA: MIT-Bradford Books, 1991.
34. Y. LeCun, "Generalization and network design strategies," in Connectionism in Perspective, R. Pfeifer, Z. Schreter, F. Fogelman, and L. Steels, Eds. Zurich, Switzerland: Elsevier, 1989.
35. Y. LeCun, B. Boser, J. S. Denker, D. Henderson, R. E. Howard, W. Hubbard, and L. D. Jackel, "Handwritten digit recognition with a back-propagation network," in Advances in Neural Information Processing Systems 2 (NIPS'89), David Touretzky, Ed. Denver, CO: Morgan Kaufmann, 1990.
36. G. L. Martin, "Centered-object integrated segmentation and recognition of overlapping hand-printed characters," Neural Computation, vol. 5, no. 3, pp. 419-429, 1993.
37. J. Wang and J. Jean, "Multi-resolution neural networks for omnifont character recognition," in Proc. Int. Conf. Neural Networks, vol. III, 1993, pp. 1588-1593.
38. Y. Bengio, Y. LcCun, C. Nohl, and C. Burges, "Lerec: A NN/HMM hybrid for on-line handwriting recognition," Neural Computation, vol. 7, no. 5, 1995.
39. S. Lawrence, C. L. Giles, A. C. Tsoi, and A. D. Back, "Face recognition: A convolutional neural network approach," IEEE Trans. Neural Networks, vol. 8, pp. 98-113, Jan. 1997.
40. K. J. Lang and G. E. Hinton, "A time delay neural network architecture for speech recognition," Carnegie-Mellon Univ., Pittsburgh, PA, Tech. Rep. CMU-CS-88-152, 1988.
41. A. H. Waibel, T. Hanazawa, G. Hinton, K. Shikano, and K. Lang, "Phoneme recognition using time-delay neural networks," IEEE Trans. Acoustics, Speech, Signal Processing, vol. 37, pp. 328-339, Mar. 1989.
42. L. Bottou, F. Fogelman, P. Blanchet, and J. S. Lienard, "Speaker independent isolated digit recognition: Multilayer perceptron versus dynamic time warpina," Neural Networks, vol. 3, pp. 453-165, 1990.
43. P. Haffner and A. H. Waibel, "Time-delay neural networks embedding time alignment: A performance analysis," in Proc. EUROSPEECH'91, 2nd Europ. Conf. Speech Communication and Technology, Genova, Italy.
44. I. Guyon, P. Albrecht, Y. LeCun, J. S. Denker, and W. Hubbard, "Design of a neural network character recognizer for a touch terminal," Pattern Recognit., vol. 24, no. 2, pp. 105-119, 1991.
45. J. Bromley, J. W. Bentz, L. bottou, I. Guyon, Y. LeCun, C. Moore, E. Säckinger, and R. Shah, "Signature verification using a Siamese time delay neural network," Int. J. Pattern Recognit. Artificial Intell., vol. 7, no. 4, pp. 669-687, Aug. 1993.
46. Y. LeCun, I. Kanter, and S. Solla, "Eigenvalues of covariance matrices: Application to neural-network learnine," Phys. Rev. Lett., vol. 66, no. 18, pp. 2396-2399, May 1991.
47. T. G. Dietterich and G. Bakiri, "Solving multiclass learning problems via error-correcting output codes," J. Artificial Intell. Res., vol. 2, pp. 263-286, 1995.
48. L. R. Bahl, P. F. Brown, P. V. de Souza, and R. L. Mercer, "Maximum mutual information of hidden Markov model parameters for speech recognition," in Proc. Int. Conf. Acoustics, Speech, Signal Processing, 1986, pp. 49-52.
49. _, "Speech recognition with continuous-parameter hidden Markov models," Comput., Speech Language, vol. 2, pp. 219-234, 1987.
50. B. H. Juang and S. Katagiri, "Discriminative learning for minimum error classification," IEEE Trans. Acoustics, Speech, Signal Processing, vol. 40, pp. 3043-3054, Dec. 1992.
51. Y. LeCun, L. D. Jackel, L. Bottou, A. Brunot, C. Cortes, J. S. Denker, H. Drucker, I. Guyon, U. A. Muller, E. Säckinger, P. Simard, and V. N. Vapnik, "Comparison of learning algorithms for handwritten digit recognition," in Int. Conf. Artificial Neural Networks, F. Fogelman and P. Gallinari, Eds. Paris: EC2 & Cie, 1995, pp. 53-60.
52. I. Guyon, I. Poujaud, L. Personnaz, G. Dreyfus, J. Denker, and Y. LeCun, "Comparing different neural net architectures for classifying handwritten digits," in Proc. IEEE IJCNN, Washington, DC, vol. II. 1989, pp. 127-132,.
53. R. Ott, "Construction of quadratic polynomial classifiers," in Proc. IEEE Int. Conf. Pattern Recognition, 1976, pp. 161-165.
54. J. Schürmann, "A multifont word recognition system for postal address reading," IEEE Trans. Comput., vol. C-27, pp. 721-732, Aug. 1978.
55. Y. Lee, "Handwritten digit recognition using k-nearest neighbor, radial-basis functions, and backpropagation neural networks," Neural Computation, vol. 3, no. 3, pp. 440-449, 1991.
56. D. Saad and S. A. Solla, "Dynamics of on-line gradient descent learning for multilayer neural networks," in Advances in Neural Information Processing Systems, vol. 8, D. S. Touretzky, M. C. Mozer, and M. E. Hasselmo, Eds. Cambridge, MA: MIT, 1996, pp. 302-308.
57. G. Cybenko, "Approximation by superpositions of sigmoidal functions," Math. Control, Signals, Syst., vol. 2, no. 4, pp. 303-314, 1989.
58. L. Bottou and V. N. Vapnik, "Local learning algorithms," Neural Computation, vol. 4, no. 6, pp. 888-900, 1992.
59. R. E. Schapire, "The strength of weak learnability," Machine Learning, vol. 5, no. 2, pp. 197-227, 1990.
60. H. Drucker, R. Schapire, and P. Simard, "Improving performance inneural networks using a boosting algorithm," in Advances in Neural Information Processing Systems 5, S. J. Hanson, J. D. Cowan, and C. L. Giles, Eds. San Mateo, CA: Morgan Kaufmann, 1993, pp. 42-49.
61. P. Simard, Y. LeCun, and J. Denker, "Efficient pattern recognition using a new transformation distance," in Advances in Neural Information Processing Systems, vol. 5, S. Hanson, J. Cowan, and L. Giles, Eds. San Mateo, CA: Morgan Kaufmann, 1993.
62. B. Böser, I. Guyon, and V. Vapnik, "A training algorithm for optimal margin classifiers," in Proc. 5th Annn. Workshop Computational Learning Theory, vol. 5, 1992, pp. 144-152.
63. C. J. C. Burges and B. Schoelkopf, "Improving the accuracy and speed of support vector machines," in Advances in Neural Information Processing Systems 9, M. Jordan, M. Mozer, and T. Petsche, Eds. Cambridge, MA: MIT, 1997.
64. E. Säckinger, B. Boser, J. Bromley, Y. LeCun, and L. D. Jackel, "Application of the ANNA neural network chip to high-speed character recognition," IEEE Trans. Neural Networks, vol. 3, no. 3, pp. 498-505, Mar. 1992.
65. J. S. Bridle, "Probabilistic interpretation of feedforward classification networks outputs, with relationship to statistical pattern recognition," in Neurocomputing, Algorithms, Architectures and Applications, F. Fogelman, J. Herault, and Y. Burnod, Eds. Les Arcs, France: Springer, 1989.
66. Y. LeCun, L. Bottou, and Y. Bengio, "Reading checks with graph transformer networks," in Proc. IEEE Int. Conf. Acoustics, Speech, Signal Processing. Munich, Germany, vol. 1, 1997, pp. 151-154,.
67. Y. Bengio, Neural Networks for Speech and Sequence Recognition. London, UK: International Thompson, 1996.
68. C. Burges, O. Matan, Y. LeCun, J. Denker, L. Jackel, C. Stenard, C. Nohl, and J. Ben, "Shortest path segmentation: A method for training a neural network to recognize character strings," in Proc. Int. Joint Conf. Neural Networks, Baltimore, MD, vol. 3, 1992, pp. 165-172.
69. T. M. Breuel, "A system for the off-line recognition of handwritten text," in Proc. IEEE ICPR'94, Jerusalem, pp. 129-134.
70. A. Viterbi, "Error bounds for convolutional codes and an asymptotically optimum decoding algorithm," IEEE Trans. Inform. Theory, vol. 15, pp. 260-269, Apr. 1967.
71. R. P. Lippmann and B. Gold, "Neural-net classifiers useful for speech recognition," in Proc. IEEE 1st Int. Conf. Neural Networks, San Diego, CA, June 1987, pp. 417-422.
72. H. Sakoe, R. Isotani, K. Yoshida, K. Iso, and T. Watanabe, "Speaker-independent word recognition using dynamic programming neural networks," in Proc. Int. Conf. Acoustics, Speech, Signal Processing, Glasgow, 1989, pp. 29-32.
73. J. S. Bridle, "Alphanets: A recurrent 'neural' network architecture with a hidden Markov model interpretation," Speech Commun., vol. 9, no. 1, pp. 83-92, 1990.
74. M. A. Franzini, K. F. Lee, and A. H. Waibel, "Connectionist viterbi training: A new hybrid method for continuous speech recognition," in Proc. Int. Conf. Acoustics, Speech, Signal Processing, Albuquerque, NM, 1990, pp. 425-428.
75. L. T. Niles and H. F. Silverman, "Combining hidden Markov models and neural network classifiers," in Proc. Int. Conf. Acoustics, Speech, Signal Processing, Albuquerque, NM, 1990, pp. 417-420.
76. X. Driancourt and L. Bottou, "MLP, LVQ and DP: Comparison & cooperation," in Proc. Int. Joint Conf. Neural Networks, Seattle, WA, vol. 2, 1991, pp. 815-819.
77. Y. Bengio, R. De Mori, G. Flammia, and R. Kompe, "Global optimization of a neural network-hidden Markov model hybrid," IEEE Trans. Neural Networks, vol. 3, pp. 252-259, March 1992.
78. P. Haffner and A. H. Waibel, "Multi-state time-delay neural networks for continuous speech recognition," vol. 4, in Advances in Neural Information Processing Systems. San Mateo, CA: Morgan Kaufmann, pp. 579-588, 1992.
79. Y. Bengio, P. Simard, and P. Frasconi, "Learning long-term dependencies with gradient descent is difficult," IEEE Trans. Neural Networks, vol. 5, no. 2, pp. 157-166, Mar. 1994.
80. T. Kohonen, G. Barna, and R. Chrisley, "Statistical pattern recognition with neural network: Benchmarking studies," in Proc. IEEE 2nd Int. Conf. Neural Networks, San Diego, CA, vol. 1, 1988, pp. 61-68.
81. P. Haffner, "Connectionist speech recognition with a global MMI algorithm," in Proc. EUROSPEECH'93, 3rd Europ. Conf. Speech Communication and Technology, Berlin, pp. 1929-1932.
82. J. S. Denker and C. J. Burges, "Image segmentation and recognition," in The Mathematics of Induction. Reading, MA: Addison Wesley, 1995.
83. L. Bottou, Une Approche théorique de l'Apprentissage Connexionniste: Applications á la Reconnaissance de la Parole, Ph.D. dissertation, Univ. Paris XI, France, 1991.
84. M. Rahim, Y. Bengio, and Y. LeCun, "Disriminative feature and model design for automatic speech recognition," in Proc. Eurospeech, Rhodes, Greece, 1997, pp. 75-78.
85. U. Bodenhausen, S. Manke, and A. Waibel, "Connectionist, architectural learning for high performance character and speech recognition," in Proc. Int. Conf. Acoustics, Speech, Signal Processing, Minneapolis, MN, vol. 1, 1993, pp. 625-628.
86. F. Pereira, M. Riley, and R. Sproat, "Weighted rational transductions and their application to human language processing," in ARPA Natural Language Processing Workshop, 1994.
87. M. Lades, J. C. Vorbrüggen, J. Buhmann, and C. von der Malsburg, "Distortion invariant object recognition in the dynamic link architecture," IEEE Trans. Comput, vol. 42, pp. 300-311, March 1993.
88. B. Boser, E. Säckinger, J. Bromley, Y. LeCun, and L. Jackel, "An analog neural network processor with programmable topology," IEEE J. Solid-Stale Circuits, vol. 26, pp. 2017-2025, Dec. 1991.
89. M. Schenkel, H. Weissman, I. Guyon, C. Nohl, and D. Henderson, "Recognition-based segmentation of on-line hand-printed words," in Advances in Neural Information Processing Systems 5, S. J. Hanson, J. D. Cowan, and C. L. Giles, Eds. Denver, CO: Morgan Kaufmann, 1993, pp. 723-730.
90. C. Dugust, L. Devillers, and X. Aubert, "Combining TDNN and HMM in a hybrid system for improved continuous-speech recognition," IEEE Trans. Speech Audio Processing, vol. 2, pp. 217-224, Jan. 1994.
91. O. Matan, H. S. Baird, J. Bromley, C. J. C. Burges, J. S. Denker, L. D. Jackel, Y. LeCun, E. P. D. Pednault, W. Satterfield, C. E. Stenard, and T. J. Thompson, "Reading handwritten digits: A ZIP code recognition system," IEEE Trans. Comput., vol. 25, no. 7, pp. 59-63, July 1992.
92. Y. Bengio and Y. LeCun, "Word normalization for on-line handwritten word recognition," in Proc. IEEE Int. Conf. Pattern Recognition, Jerusalem, 1994.
93. R. Vaillant, C. Monrocq, and Y. LeCun, "Original approach for the localization of objects in images," Proc. Inst. Elect. Eng., vol. 141, no. 4, pp. 245-250, Aug. 1994.
94. R. Wolf and J. Platt, "Postal address block location using a convolutional locator network," in Advances in Neural Information Processing Systems 6, J. D. Cowan, G. Tesauro, and J. Alspector, Eds. San Mateo, CA: Morgan Kaufmann, 1994, pp. 745-752.
95. S. Nowlan and J. Platt, "A convolutional neural network hand tracker," in Advances in Neural Information Processing Systems 7, G. Tesauro, D. Touretzky, and T. Leen, Eds. San Mateo, CA: Morgan Kaufmann, 1995, pp. 901-908.
96. H. A. Rowley, S. Baluja, and T. Kanade, "Neural network-based face detection," in Proc. IEEE CVPR'96, pp. 203-208.
97. E. Osuna, R. Freund, and F. Girosi, "Training support vector machines: An application to face detection," in Proc. IEEE CVPR'96, pp. 130-136.
98. H. Boudard and C. J. Wellekens, "Links between Markov models and multilayer perceptrons," in Advances in Neural Information Processing Systems, D. Touretzky, Ed. Denver: Morgan-Kaufmann, vol. 1, 1989, pp. 186-187.
99. Y. Bengio, R. De Mon, G. Flammia, and R. Kompe, "Neural network - Gaussian mixture hybrid for speech recognition or density estimation," in Advances in Neural Information Processing Systems 4, J. E. Moody, S. J. Hanson, and R. P. Lippmann, Eds. Denver, CO: Morgan Kaufmann, 1992, pp. 175-182.
100. F. C. N. Pereira and M. Riley, "Speech recognition by composition of weighted finite automata," in Finite-State Devices for Natural Lague Processing. Cambridge, MA: MIT, 1997.
101. M. Mohri, "Finite-state transducers in language and speech processine," Computational Linguistics, vol. 23, no. 2, pp. 269-311, 1997.
102. I. Guyon, M. Schenkel, and J. Denker, "Overview and synthesis of on-line cursive handwriting recognition techniques," in Handbook on Optical Character Recognition and Document Image Analysis, P. S. P. Wang and H. Bunke, Eds. New York: World Scientific, 1996.
103. M. Mohri and M. Riley, "Weighted determinization and minimization for large vocabulary recognition," in Proc. Eurospeech '97, Rhodes, Greece, pp. 131-134.
104. Y. Bengio and P. Frasconi, "An input/output HMM architecture," in Advances in Neural Information Processing Systems, vol. 7, G. Tesauro, D. Touretzky, and T. Leen, Eds. Cambridge, MA: MIT, pp. 427-434, 1996.
105. _, "Input/output HMM's for sequence processing," IEEE Trans. Neural Networks, vol. 7, no. 5, pp. 1231-1249, 1996.
106. M. Mohri, F. C. N. Pereira, and M. Riley, A Rational Design for a Weighted Finite-State Transducer Library (Lecture Notes in Computer Science). New York: Springer Verlag, 1997.
107. M. Rahim, C. H. Lee, and B. H. Juang, "Discriminative utterance verification for connected digits recognition," IEEE Trans. Speech Audio Processing, vol. 5, pp. 266-277, 1997.
108. M. Rahim, Y. Bengio, and Y. LeCun, "Discriminative feature and model design for automatic speech recognition," in Proc. Eurospeech '97, Rhodes, Greece.
109. S. Bengio and Y. Bengio, "An EM algorithm for asynchronous input/output hidden Markov models," in Proc. International Conference on Neural Information Processing, Hong-King, 1996, pp. 328-334.
110. C. Tappert, C. Suen, and T. Wakahara, "The state of the art in on-line handwriting recognition," IEEE Trans. Pattern Anal. Machine Intell., vol. 8, pp. 787-808, Dec. 1990.
111. S. Manke and U. Bodenhausen, "A connectionist recognizer for on-line cursive handwriting recognition," in Proc. Int. Conf. Acoustics, Speech, Signal Processing, Adelaide, vol. 2, 1994, pp. 633-636.
112. M. Gilloux and M. Leroux, "Recognition of cursive script amounts on postal checks," in Proc. Europ. Conf. Postal Technol., Nantes, France, June 1993, pp. 705-712.
113. D. Guillevic and C. Y. Suen, "Cursive script recognition applied to the processing of bank checks," in Proc. Int. Conf. Document Analysis Recognition, Montreal, Canada, Aug. 1995, pp. 11-14.
114. L. Lam, C. Y. Suen, D. Guillevic, N. W. Strathy, M. Cheriet, K. Liu, and J. N. Said, "Automatic processing of information on checks," in Int. Conf. Systems, Man, and Cybernetics, Vancouver, Canada, Oct. 1995, pp. 2353-2358.
115. C. J. C. Burges, J. I. Ben, J. S. Denker, Y. LeCun, and C. R. Nohl, "Off line recognition of handwritten postal words using neural networks," Int. J. Pattern Recognit. Artificial Intell., vol. 7, no. 4, p. 689, 1993.
116. Y. LeCun, Y. Bengio, D. Henderson, A. Weisbuch, H. Weissman, and L. Jackel, "On-line handwriting recognition with neural networks: Spatial representation versus temporal representation," in Proc. Int. Conf. Handwriting Drawing, 1993.
117. U. Müller, A. Gunzinger, and W. Guggenbühl, "Fast neural net simulation with a DSP processor array," IEEE Trans. Neural Networks, vol. 6, pp. 203-213, Jan. 1995.
118. R. Battiti, "First- and second-order methods for learning: Between steepest descent and Newton's method," Neural Computation, vol. 4, no. 2, pp. 141-166, 1992.
119. A. H. Kramer and A. Sangiovanni-Vincentelli, "Efficient par-allel learning algorithms for neural networks," in Advances in Neural Information Processing Systems, vol. 1, D. S. Touretzky, Ed. San Mateo, CA: Morgan Kaufmann, 1988, pp. 40-48.
120. M. Moller, Efficient Training of Feed-Forward Neural Networks, Ph.D. dissertation, Aarhus Univ., Aarhus, Denmark, 1993.
121. S. Becker and Y. LeCun, "Improving the convergence of back-propagation learning with second-order methods," Univ. Toronto Connectionist Res. Group, Toronto, Ontario, Canada, Tech. Rep. CRG-TR-88-5, Sept. 1988.