Pattern Classification Using Neural Networks

IEEE Communications Magazine

Volumn 27, Issue 11, 1989, Pages 47-50

  Lippmann, Richard P ^a  

^a NONE

Author keywords

[No Author keywords available]

Indexed keywords

PATTERN RECOGNITION;

DECISION-TREE CLASSIFIERS; FEATURE-MAP CLASSIFIER; HYPERSPHERE CLASSIFIERS; LEARNING VECTOR QUANTIZER; NEAREST-NEIGHBOR CLASSIFIERS; PATTERN CLASSIFICATION;

SYSTEMS SCIENCE AND CYBERNETICS;

EID: 0024771475     PISSN: 01636804     EISSN: None     Source Type: Journal    
DOI: 10.1109/35.41401     Document Type: Article

References (92)

1. R. O. Duda and P. E. Hart, Pattern Classification and Scene Analysis, NY: John Wiley and Sons, 1973.
2. K. Fukunaga, Introduction to Statistical Pattern Recognition, NY: Academic Press, 1972.
3. R. P. Gorman and T. J. Sejnowski, “Analysis of Hidden Units in a Layered Network Trained to Classify Sonar Targets,” Neural Networks, vol. 1, pp. 75-89, 1988.
4. B. Kammerer and W. Kupper, “Experiments for Isolated-Word Recognition with Single and Multi-Layer Perceptrons,” Neural Networks, vol. 1, supp. 1, p. 302, Abstracts of 1st Annual INNS Meeting, Boston, 1988.
5. K.J. Lang and G. E. Hinton, “The Development of the Time-Delay Neural Network Architecture for Speech Recognition,” Tech. Rep. CMU-CS-88-152, Carnegie-Mellon University, 1988.
6. R. P. Lippmann, “Review of Neural Networks for Speech Recognition,” Neural Comp., vol. 1 (1), pp. 1-38, 1989.
7. S. M. Peeling and R. K. Moore, “Experiments in Isolated Digit Recognition Using the Multi-Layer Perceptron,” Tech. Rep. 4073, Royal Speech and Radar Estab., Malvern, Worcester, Great Britain, Dec. 1987.
8. A. Waibel, H. Sawai, and K. Shikano, “Modularity and Scaling in Large Phonemic Neural Nets,” Tech. Rep. TR-1-0034, ATR Interpreting Telephony Research Laboratories, Japan, Aug. 1988.
9. R. P. Lippmann, “An Introduction to Computing with Neural Nets,” IEEE ASSP Mag., vol. 4 (2), pp. 4-22, Apr. 1987.
10. A. R. Barron and R. L. Barron, “Statistical Learning Networks: A Unifying View,” 1988 Symp. on the Interface: Statistics and Comp. Sci., Reston, VA, Apr. 21-23, 1988.
11. R. Gnanadesikan and J. R. Kettenring, “Discriminant Analysis and Clustering,” Stat. Sci., vol. 4(1), pp. 34-69, 1989.
12. A. K. Jain, “Advances in Statistical Pattern Recognition,” Pattern Recognition Theory and Applications, P.A. Devijver and J. Kittler, eds., pp. 1-19, Series F: Computer and Systems Sciences, vol. 30, NY: Springer Verlag, 1989.
13. G. E. Hinton, “Connectionist Learning Procedures,” Tech. Rep. CMU-CS-87-115, Carnegie Mellon University, Computer Science Department, June 1987.
14. T. Kohonen, G. Barna, and R. Chrisley, “Statistical Pattern Recognition with Neural Networks: Bench Marking Studies,” IEEE Annual lnt'l. Conf. on Neural Networks, San Diego, July 1988.
15. J. Pearson and R. Lippmann, ed., “Adaptive Knowledge Processing,” DARPA Neural Network Study, Ch. II, pp. 55-178, AFCEA International Press, Fairfax, VA, 1988.
16. E. B. Baum and D. Haussler, “What Size Net Gives Valid Generalization?” Advances in Neural Info. Processing Syst. 1, D.S. Touretzky, ed., San Mateo, CA: Morgan Kauffman, 1989.
17. S. Judd, “Learning in Neural Networks,” COLT ′88 Proc. of the 1988 Workshop on Computational Learning Theory, D. Haussler and L Pitt, eds., pp. 2-8, San Mateo, CA: Morgan Kaufmann, 1988.
18. L. G. Valiant, “Functionality in Neural Nets,” COLT 88 Proc. of the 1988 Workshop on Computational Learning Theory, D. Haussler and L. Pitt, eds., pp. 28-39, San Mateo, CA: Morgan Kaufmann, 1988.
19. G. Nagy, “Candide's Practical Principles of Experimental Pattern Recognition,” IEEE Trans, on Pattern Anal, and Machine Intel., vol. PAMI-5(2), pp. 199-200, 1983
20. D. E. Rumelhart, G. E. Hinton, and R. J. Williams, “Learning Internal Representations by Error Propagation,” Parallel Distributed Processing, D.E. Rumelhart and J.L. McClelland, eds., Ch. 8, Cambridge, MA: MIT Press, 1986.
21. D. H. Ackley, G. E. Hinton, and T.J. Sejnowski, “A Learning Algorithm for Boltzmann Machines,” Cognitive Science, vol. 9, pp. 147-160, 1985.
22. L. Breiman, J. H. Friedman, R. A. Olshen, and C. J. Stone, “Classification and Regression Trees,” Wadsworth International Group, Belmont, CA, 1984.
23. J. R. Quinlan, “Simplifying Decision Trees,” Int'l. J. Man-Machine Studies, vol. 27, pp. 221-234, 1987.
24. C. L. Giles and T. Maxwell, “Learning, Invariance, and Generalization in High-Order Networks,” Applied Optics, vol. 26, pp. 4,972-4,978, Dec. 1987.
25. N. J. Nilsson, “Learning Machines,” McGraw Hill, N.Y., 1965.
26. S. Farlow, “Self-Organizing Methods in Modeling,” Marcel Dekker, 1984.
27. D. J. Hand, ed., “Kernel Discriminant Analysis,” John Wiley and Sons Ltd., New York, NY, 1982.
28. J. Moody and C. Darken, “Fast Learning in Networks of Locally-Tuned Processing Units,” Tech. Rep. YALEU/DCS/RR-654, Yale Computer Science Department, New Haven, CT, Oct. 1988.
29. A. Rojer and E. Schwartz, “A Multiple-Map Model for Pattern Classification,” Neural Computation, vol. 1(1), pp. 104-115, 1989.
30. F. H. Glanz and W. T. Miller, “Shape Recognition using a CMAC-Based Learning System,” SPIE Proc. Intelligent Robots and Computer Vision, Cambridge, MA, 1987.
31. J. Moody, “Fast Learning in Multi-Resolution Hierarchies,” Tech. Rep. YALEU/DCS/RR-681, Yale Computer Science Department, New Haven, CT, Feb. 1989.
32. D. S. Broomhead and D. Lowe, “Radial Basis Functions, Multi-Variable Functional Interpolation and Adaptive Networks,” Tech. Rep. RSRE, Memo. no. 4, 148, Royal Speech and Radar Establishment, Malvern, Worcester, Great Britain, Mar. 1988.
33. M. Niranjan and F. Fallside, “Neural Networks and Radial Basis Functions in Classifying Static Speech Patterns,” Tech. Rep. CUED/FINFENG/TR 22, Cambridge University Engineering Department, 1988.
34. S. Renals and R. Rohwer, “Phoneme Classification Experiments using Radial Basis Functions,” Proc. Int'l. Joint Conf. on Neural Networks, pp. 1.461-1.467, IEEE, Washington, DC, June 1989.
35. S. M. Omohundro, “Efficient Algorithms with Neural Network Behavior,” Complex Systems, vol. 1, pp. 273-347, 1987.
36. W. M. Huang and R. P. Lippmann, “Neural Net and Traditional Classifiers,” Neural Info. Processing Syst., D. Anderson, ed., pp. 387-396, NY: American Institute of Physics, 1988.
37. T. Kohonen, “An Introduction to Neural Computing,” Neural Networks, vol. 1, pp. 3-16, 1988.
38. D L. Reilly, L.N. Cooper, and C. Elbaum, “A Neural Model for Category Learning,” Bio. Cybernetics, vol. 45, pp. 35-41, 1982.
39. D. L. Reilly, C. Scofield, C. Elbaum, and L. N. Cooper, “Learning System Architectures Composed of Multiple Learning Modules,” IEEE 1st Int'l. Conf. on Neural Networks, June 1987.
40. G. A. Carpenter and S. Grossberg, “ART 2: Self-Organization of Stable Category Recognition Codes for Analog Input Patterns,” Applied Optics, vol. 26, pp. 4,919-4,930, 1987.
41. C Stanfill and D. Waltz, “Toward Memory-Based Reasoning,” Commun. of the ACM, vol. 29(12), pp. 1,213-1,228, Dec. 1986.
42. D. Waltz, C. Stanfill, S. Smith, and R. Thau, “Very Large Database Applications of the Connection Machine System,” Proc. Nat'l. Computer Conf., pp. 159-165, 1987.
43. D. Wolpert, “Alternative Generalizers to Neural Nets,” Neural Networks, vol. 1, supp. 1, p. 474, Abstracts of 1st Annual INNS Meeting, Boston, 1988.
44. R. Crick, “The Recent Excitement about Neural Nets,” Nature, vol. 337, pp. 129-132, 1989.
45. T. J. Sejnowski and C. M. Rosenberg, “Parallel Networks that Learn to Pronounce English Text,” Complex Systems, vol. 1, pp. 145-168, 1987.
46. N. Qian and T. J. Sejnowski, “Predicting the Secondary Structure of Globular Proteins using Neural Network Models,” J. Molecular Biology, vol. 202, pp. 865-884, 1988.
47. G. Tesauro and T. J. Sejnowski, “A Neural Network that Learns to Play Backgammon,” Neural Information Processing Systems, D. Anderson, ed., pp. 794-803, American Institute of Physics, New York, 1988.
48. D. Zipser and R. A. Andersen, “A Back Propagation Programmed Network that Simulates Response Properties of a Subset of Posterior Parietal Neurons,” Nature, vol. 331, pp. 679-684, 1988.
49. R. P. Lippmann and P. E. Beckman, “Adaptive Neural-Net Processing for Signal Detection in Non-Gaussian Noise,” Advances in Neural Info. Processing Syst. 1, D.S. Touretzky, ed., San Mateo, CA: Morgan Kauffman, 1989.
50. S. Tamura and A. Waibel, “Noise Reduction using Connectionist Models,” Proc. IEEE Int'l. Conf. Acoustics, Speech and Signal Processing, pp. 553-556, Apr. 1988.
51. S. J. Hanson and D. J. Burr, “Knowledge Representation in Connectionist Networks,” Tech. Rep., Bell Communications Research, Morristown, New Jersey, Feb. 1987.
52. I. D. Longstaff and J. F. Cross, “A Pattern Recognition Approach to Understanding the Multi-Layer Perceptron,” Memo. 3,936, Royal Signals and Radar Establishment, July 1986.
53. A Lapedes and R. Farber, “How Neural Nets Work,” Neural Information Processing Systems, D. Anderson, ed., pp. 442-456, American Institute of Physics, New York, 1988.
54. A. Wieland and R. Leighton, “Geometric Analysis of Neural Network Capabilities,” IEEE 1st Int'l. Conf. on Neural Networks, pp. 111-385, June 1987.
55. G. Cybenko, “Approximation by Superpositions of a Sigmoidal Function,” Mathematics of Control, Signals, and Systems, 2(4), 1989, To Appear.
56. M. R. Brady, R. Raghavan, and J. Slawny, “Gradient Descent Fails to Separate,” In IEEE Annual International Conference on Neural Networks, pages1649-1656, IEEE, San Diego, July 1988.
57. E. D Sontag and H. J Sussmann, “Back-Propagation Separates when Perceptrons Do,” Tech. Rep. SYCON-88-12, Rutgers Center for Systems and Control, December 1988.
58. B. S. Winner and J. S. Denker, “Strategies for Teaching Layered Networks Classification Tasks,” In D. Anderson, editor, Neural Information Processing Systems, pp. 850-859, American Institute of Physics, 1988.
59. D. J. Burr, “Experiments on Neural Net Recognition of Spoken and Written Text,” IEEE Transactions on Acoustics, Speech and Signal Processing, 36: pp. 1162-1168, 1988.
60. H. Sawai et al., “Parallelism, Hierarchy, Scaling in Time-Delay Neural Networks for Spotting Japanese Phonemes/CVsyllables,” In Proceedings International Joint Conference on Neural Networks, pp. 11.81-11.88, IEEE, Washington DC, June 1989.
61. S. Becker and Y. Le Cun, “Improving the Convergence of Back-Propagation Learning with Second Order Methods,” Proc. of the 1988 Connectionist Models Summer School, D. Touretzky, G. Hinton, and T. Sejnowski, eds., pp. 29-37, San Mateo, CA: Morgan Kauffman, 1989.
62. L. W. Chan and F. Fallside, “An Adaptive Training Algorithm for Back-Propagation Networks,” Comp. Speech and Language, vol. 2, pp. 205-218, 1987.
63. S. E. Fahlman, “Faster-Learning Variations on Back-Propagation: An Empirical Study,” Proc. of the 1988 Connectionist Models Summer School, D. Touretzky, G. Hinton, and T. Sejnowski, eds., pp. 38-51, San Mateo, CA: Morgan Kauffman, 1989.
64. S. J. Hanson and D. J. Burr, “Minkowski-r Back-Propagation: Learning in Connectionist Models with Non-Euclidean Error Signals,” Neural Information Processing Systems, D. Anderson, ed., pp. 348-357, NY: American Institute of Physics, 1988.
65. R. A.Jacobs, “Increased Rates of Convergence Through Learning Rate Adaptation,” Neural Networks, vol. 1, pp. 295-307, 1988.
66. H. C. Leung and V. W. Zue, “Some Phonetic Recognition Experiments using Artificial Neural Nets,” Proc. IEEE Int′1. Conf. on Acoustics, Speech and Signal Processing, Volume 1: Speech Processing, Apr. 1988.
67. R. L. Watrous, “Learning Algorithms for Connectionist Networks: Applied Gradient Methods of Nonlinear Optimization,” Tech. Rep. MS-CIS-87-51, Line Lab 72, University of Pennsylvania, June 1986.
68. S. J. Hanson and L. Y. Pratt, “Some Comparisons of Constraints for Minimal Network Construction with Back-Propagation,” Advances in Neural Information Processing Systems 1, D. S. Touretzky, ed., San Mateo, CA: Morgan Kauffman, 1989.
69. M. C. Mozer and P. Smolensky, “Skeletonization: a Technique for Trimming the Fat from a Network Via Relevance Assessment,” Advances in Neural Information Processing Systems 7, D. S. Touretzky, ed., Morgan Kauffman, San Mateo, CA, 1989.
70. J R. Quinlan, “Induction of Decision Trees,” Machine Learning, vol. 1, pp. 81-106, 1986.
71. D. H. Fisher and K. B. McKusick, “An Empirical Comparison of ID3 and Back-Propagation,” Tech. Rep. TR CS-88-14, Department of Computer Science, Vanderbilt University, Nashville, TN, 1988.
72. A. K. Jain and B. Chandrasekaran, “Dimensionality and Sample Size Consideration in Pattern Recognition Practice,” Handbook of Statistics: Vol. 2, P. R. Krishnaiah and L. N. Kanal, eds., pp. 835-855, North Holland, 1982.
73. G. V. Trunk, “A Problem of Dimensionality: a Simple Example,” IEEE Trans, on Pattern Analysis and Machine Intel., vol. PAMI-1, no. 3, pp. 306-307, 1979.
74. M. Stone, “Cross-Validation Choice and Assessment of Statistical Predictions,” J. of the Royal Statistical Soc., vol. B-36, pp. 111-147, 1974.
75. B. Efron, ed., The Jackknife, the Bootstrap and Other Resampling Plans, Soc. for Industrial and Applied Mathematics, Philadelphia, PA, 1982.
76. J. R. Quinlan and R. Rivest, “Inferring Decision Trees using the Minimum Descriptive Length Principle,” Info, and Computation, to appear, 1989.
77. J. Rissanen, “A Universal Prior for Integers and Estimation by Minimum Descriptive Length,” The Annals of Statistics, vol. 11, pp. 416431, 1983.
78. A. R. Barron, “Predicted Squared Error: a Criterion for Automatic Model Selection,” Self-Organizing Methods in Modeling, S. J. Farlow, ed., pp. 87-103, NY: Marcel Dekker, 1984.
79. M. A. Minsky and S. A. Papert, Perceptrons—Expanded Edition, Cambridge, MA: MIT Press, 1988.
80. T. Poggio, “On Optimal Nonlinear Associative Recall,” Bio. Cybernetics, vol. 19, p. 201, 1975.
81. A. R. Barron, “Adaptive Learning Networks: Development and Application in the United States of Algorithms Related to GMDH,” Self-Organizing Methods in Modeling, S. J. Farlow, ed., pp. 25-65, NY: Marcel Dekker, 1984.
82. J. D. Farmer and J.J. Sidorowich, “Exploiting Chaos to Predict the Future and Reduce Noise,” Tech. Rep. LA-UR-88-901, Los Alamos National Laboratory, Los Alamos, New Mexico, Mar. 1988.
83. T. Kohonen, Self-Organization and Associative Memory, Berlin: Springer-Verlag, 1984.
84. E. McDermott and S. Katagiri, “Phoneme Recognition using Kohonen's Learning Vector Quantization,” ATR Workshop on Neural Networks and Parallel Distributed Processing, Osaka, Japan, July 1988.
85. B. G. Batchelor, “Classification and Data Analysis in Vector Space,” Pattern Recognition, B. G. Batchelor, ed., ch. 4, pp. 67-116, London: Plenum Press, 1978.
86. S. E. Hampson and D. J. Volper, “Disjunctive Models of Boolean Category Learning,” Bio. Cybernetics, vol. 56, pp. 121-137, 1987.
87. C. L. Scofield, D. L. Reilly, C. Elbaum, and L. N. Cooper, “Pattern Class Degeneracy in an Unrestricted Storage Density Memory,” Neural Info. Processing Syst., pp. 674-682, American Institute of Physics, 1988.
88. E. Collins, S. Ghosh, and C. Scofield, “Appendix G: Risk Analysis,” DARPA Neural Network Study, pp. 429-443, Fairfax, VA: AFCEA International Press, 1988.
89. I. Guyon, I. Poujand, L. Personnaz, and G. Dreyfus, “Comparing Different Neural Network Architectures for Classifying Handwritten Digits,” Proc., Int'l. Joint Conf. on Neural Networks, pp. 11.127-11.132, Washington DC: IEEE, June 1989.
90. Y. Lee, “Classifiers: Adaptive Modules in Pattern Recognition Systems,” Master's thesis, Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science, Cambridge, MA, May 1989.
91. Y. Lee and R. P. Lippmann, “Practical Characteristics of Neural Network and Conventional Pattern Classifiers on Artificial and Speech Problems,” Proc., Neural Info. Processing Syst.—Natural and Synthetic Conf., Denver, CO, Nov. 1989.
92. M. F. Tenorio and W. T. Lee, “Self Organizing Neural Networks for the Identification Problem,” Advances in Neural Information Processing Systems 1, D. S. Touretzky, ed., pp. 57-64, San Mateo, CA: Morgan Kauffman, 1989.