Modeling the manifolds of images of handwritten digits

IEEE Transactions on Neural Networks

Volumn 8, Issue 1, 1997, Pages 65-74

  Hinton, Geoffrey E ^a   Dayan, Peter ^b   Revow, Michael ^a  

^a UNIVERSITY OF TORONTO   (Canada)

^b MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

Autoencoder; Density estimation; Factor analysis; Minimum description length; Principal components

Indexed keywords

APPROXIMATION THEORY; COMPUTER SIMULATION; IMAGE PROCESSING; NEURAL NETWORKS; PROBABILITY DENSITY FUNCTION; STATISTICAL METHODS;

AUTOENCODER; DENSITY ESTIMATION; FACTOR ANALYSIS; HANDWRITTEN DIGITS; MINIMUM DESCRIPTION LENGTH; PRINCIPAL COMPONENTS ANALYSIS;

CHARACTER RECOGNITION;

EID: 0030737323     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/72.554192     Document Type: Article

References (28)

1. Y. Le Cun, B. Boser, J. S. Denker, D. Henderson, R. E. Howard, W. Hubbard, and L. D. Jackel, "Handwritten digit recognition with a backpropagation network," in Advances in Neural Information Processing Systems, D. S. Touretzky, Ed., vol. 2. San Mateo, CA: Morgan Kaufmann, 1990, pp. 396-404.
2. D. E Specht, "Probabilistic neural networks," Neural Networks, vol. 3, no. 1, pp. 109-118, 1990.
3. C. Bishop, Neural Networks for Pattern Recognition. New York: Oxford. Univ. Press, 1975.
4. B. Widrow, "The 'rubber-mask' technique - I: Pattern measurement and analysis," Pattern Recognition, vol. 5, pp. 175-197, 1973.
5. M. A. Fischler and R. A. Elschlager, "The representation and matching of pictorial structures," IEEE Trans. Comput., vol. C-22, pp. 67-92, 1973.
6. M. Revow, C. K. I. Williams, and G. E. Hinton, "Using generative models for handwritten digit recognition," IEEE Trans. Pattern Anal. Machine Intell., vol. 18, pp. 592-606, 1996.
7. P. Simard, Y. Le Cun, and J. Denker, "Efficient pattern recognition using a new transformation distance," in Advances in Neural Information Processing Systems 5, J. D. Cowan, S. J. Hanson, and C. L. Giles, Eds. San Mateo, CA: Morgan Kaufmann, 1993, pp. 50-58.
8. P. Simard, "Efficient computation of complex distance metrcis using hierarchial filtering," in Advances in Neural Information Processing Systems 6, J. D. Cowan, G. Tesauro, and J Alspector, Eds. San Mateo, CA: Morgan Kaufmann, 1994, pp. 168-175.
9. G. E. Hinton, M. Revow, and P. Dayan, "Recognizing handwritten digits using mixtures of linear models," in Advances in Neural Information Processing Systems 7, G. Tesauro, D. S. Touretzky, and T. K. Leen, Eds. Cambridge, MA: MIT Press, 1995, pp. 1015-1022.
10. C. Bregler and S. M. Omohundro, "Nonlinear image interpolation using manifold learning," in Advances in Neural Information Processing Systems 7, G. Tesauro, D. S. Touretzky, and T. K. Leen, Eds. Cambridge, MA: MIT Press, 1995, pp. 971-980.
11. N. Kambhatla and T. K. Leen, "Fast nonlinear dimension reduction," in Advances in Neural Information Processing Systems 6, J. D. Cowan, G. Tesauro, and J. Alspector, Eds. San Mateo, CA: Morgan Kaufmann, 1994, pp. 152-159.
12. K. Sung and T. Poggio, "Example-based learning for view-based human face detection," AI Memo 1521, CBCL paper 112, 1995.
13. T. Hastie and P. Simard, "Learning prototype models for tangent distance," in Advances in Neural Information Processing Systems 7, G. Tesauro, D. S. Touretzky, and T. K. Leen, Eds. Cambridge, MA: MIT Press, 1995, pp. 999-1006.
14. A. P. Dempster, N. M. Laird, and D. B. Rubin, "Maximum likelihood from incomplete data via the EM algorithm," in Proc. Roy. Statist. Soc., vol. B-39, 1977, pp. 1-38.
15. G. E. Hinton and R. Zemel, "Autoencoders, minimum description length and helmholtz free energy," in Advances in Neural Information Processing Systems 6, J. Cowan, G. Tesauro, and J. Alspector, Eds. San Mateo, CA: Morgan Kauffmann, 1994.
16. R. S. Zemel, "A minimum description length framework for unsupervised learning," Ph.D. dissertation, Dep. Comput. Sci., Univ. Toronto, Canada, 1993.
17. H. Schwenk and M. Milgram, "Transformation invariant autoassociation with application to handwritten character recognition," in Advances in Neural Information Processing Systems 7, G. Tesauro, D. S. Touretzky, and T. K. Leen, Eds. Cambridge, MA: MIT Press, 1995, pp. 991-998.
18. B. S. Everitt, An Introduction to Latent Variable Models. London: Chapman and Hall, 1984.
19. K. V. Mardia, J. T. Kent, and J. M. Bibby, Multivariate Analysis. New York: Academic, 1979.
20. R. M. Neal and P. Dayan, "Factor analysis using delta-rule wake-sleep learning," Dep. Statist., Univ. Toronto, Canada, Tech. Rep. 9607, 1996 [Online]. Available ftp://ftp.cs.toronto.edu/pub/radford/ws-fa.ps.Z
21. G. E. Hinton, P. Dayan, B. J. Frey, and R. M. Neal, "The wake-sleep algorithm for unsupervised neural networks," Sci., vol. 268, pp. 1158-1161, 1995.
22. P. Dayan, G. E. Hinton, R. M. Neal, and R. S. Zemel, "The Helmholtz machine," Neural Computa., vol. 7, no. 7, pp. 889-904, 1995.
23. D. B. Rubin and D. T. Thayer, "EM algorithms for ML factor analysis," Psychometrika, vol. 47, no. 1, pp. 69-76, 1982.
24. T. Hastie and R. Tibshirani, "Discriminant adaptive nearest neighbor clasification," IEEE Trans. Pattern Anal. Machine Intell., vol. 18, pp. 607-616, 1995.
25. H. Schwenk and M. Milgram, "Learning discriminant tangent models for handwritten character recognition," in ICANN*95. New York: Springer-Verlag, 1995.
26. J. J Hull, "A database for handwritten text recognition research," IEEE Trans. Pattern Anal. Machine Intell., vol. 16, pp. 550-554, 1994.
27. J. L. Fleiss, Statistical Methods for Rates and Proportions, 2nd ed. New York: Wiley, 1981.
28. P. Simard, B. Victorri, Y. LeCun, and J. Denker, "A formalism for specifying selected invariances in an adaptive network," in Advances in Neural Information Processing Systems 4, J. E. Moody, S. J. Hanson, and S. J. Lippmann, Eds. San Mateo, CA: Morgan Kaufmann, 1992, pp. 895-903.