Small sample size problems in designing artificial neural networks

Machine Intelligence and Pattern Recognition

Volumn 11, Issue C, 1991, Pages 33-50

  Raudys, Šarūnas ^a   Jain, Anil K ^b  

^a Inst for Mathematics/Cybernetics   (Lithuania)

^b MICHIGAN STATE UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84974761680     PISSN: 09230459     EISSN: None     Source Type: Book Series    
DOI: 10.1016/B978-0-444-88740-5.50008-6     Document Type: Chapter

References (53)

1. Allais, D.C., The problem of too many measurements in pattern recognition. IEEE Int. Con. Rec., 1966, 124–130 (part 7).
2. Burr, D.J., Experiments on neural net recognition of spoken and written text. IEEE Trans. Acoust., Speech, Signal Process. 36:7 (July 1988), 1162–1168.
3. Chandrasekaran, B., Jain, A.K., On balancing decision functions. J. Cybernet. Inform. Sci. 2 (1979), 12–15.
4. Cover, T.M., Geometrical and statistical properties of systems of linear inequalities with application in pattern recognition. IEEE Trans. Elect. Comp. 14 (1965), 326–334.
5. Cun, Y.L., Jackel, L.D., Guyon, I., Henderson, D., Boser, B., Howard, R.E., Denker, J.S., Hubbard, W., Graf, H.P., Handwritten digit recognition: Applications of neural network chips and automatic learning. IEEE Communications Magazine, Nov. 1989, 41–46.
6. Cybenko, G., Approximation by superpositions of sigmoidal function. Mathematics of Control, Signals, and Systems, 2(4), 1989.
7. Deev, A.D., Representation of statistics of discriminant analysis and asymptotic expansions in dimensionalities comparable with sample size. Report of Academy of Sciences of the USSR 195:No. 4 (1970), 756–762 (in Russian).
8. Devijver, D.A., Kitter, J., Pattern Recognition: A Statistical Approach, 1983, Wiley, N.Y.
9. Duin, R.P.W., On the Accuracy of Statistical Pattern Recognizers, 1978, Dutch Efficiency Bureau, Pijnacker, The Netherlands.
10. S. Dutta and S. Shekhar. Bond rating: A non-conservative application of neural networks. In Proc. 1988 IEEE Int. Conf. on Neural Networks, pages II443–II450, San Diego, California, July 1988.
11. Fisher, R.A., The use of multiple measurements in taxonomic problems. Ann. of Eugenics 7:No. 2 (1936), 179–188.
12. Foley, D.H., Considerations of sample and feature size. IEEE Trans. Inf. Theory IT-18:No.5 (1972), 618–626.
13. Fukunaga, K., Statistical Pattern Recognition. Young, T.Y., Fu, K.S., (eds.) Handbook of Pattern Recognition and Image Processing, 1986, Academic, New York, 3–32.
14. P. Gallinari, S. Thiria, and F. Fogllman Soulie. Multilayer perceptrons and data analysis. In Proc. 1988 IEEE Int. Conf. on Neural Networks, pages I391–I399, San Diego, California, July 1988.
15. Hand, D.J., Recent advances in error rate estimation. Pattern Recognition Letters 5 (1986), 335–346.
16. J. Higashino, B.L. Degreefand, and E.H.J. Person. Numerical analysis and adaptation method for learning rate of back propagation. In Proceedings of IEEE IJCNN1990, pages I627–I630. Lawrence Erlbaum Associates, Inc., 1990.
17. Huang, W.M., Lippmann, R.P., Neural net and traditional classifiers. Anderson, D., (eds.) Neural Info. Processing Syst, 1988, American Institute of Physics, New York, 387–396.
18. Ivakhanenko, A.G., Perceptrons, 1975, Naukova Dumka, Kijev.
19. Jain, A.K., Chandrasekaran, B., Dimensionality and sample size considerations in pattern recognition practice. Krishnaiah, P.R., Kanal, L.N., (eds.) Handbook of Statistics 2, 1982, North Holland, 835–855.
20. Jain, A.K., Dubes, R.C., Chen, C.C., Bootstrap techniques for error estimation. IEEE Trans. Pattern Anal. Machine Intell. PAMI-9:9 (1987), 628–630.
21. Jain, A.K., Ramaswami, M.D., Classifier design with Parzen windows. Gelsema, E.S., Kanal, L.N., (eds.) Pattern Recognition and Artificial Intellegence, 1988, Elsevier, 211–228.
22. Jain, A.K., Waller, W.G., On the optimal number of features in the classification of multivariate Gaussian data. Pattern Recognition 10 (1978), 365–374.
23. Juskevicius, K., Investigation of the sensitivity of a minimum distance piecewise linear classifier to the limitation of learning sample size. Statistical Problems of Control, Issue 61, 1983, Inst. of Math. and Cyb. Press, Vilnius, 89–129.
24. Kanal, L., Chandrasekaran, B., On dimensionality and sample size in statistical pattern recognition. Pattern Recognition 3 (1971), 238–255.
25. A. Khotanzad and J. H. Lu. Distortion invariant character recognition by a multilayer perceptron and back-propagation learning. In Proc. 1988 IEEE Int. Conf. on Neural Networks, pages I625–I632, San Diego, California, July 1988.
26. Koford, J.S., Groner, G.F., The use of an adaptive threshold element to design a linear optimal pattern classifier. IEEE Trans. Inf. Theory IT-12 (1966), 42–50.
27. T. Kohonen, G. Barna, and R. Chrisley. Statistical pattern recognition with neural networks: Benchmarking studies. In Proc. 1988 IEEE Int. Conf. on Neural Networks, pages I61–I68, San Diego, California, July 1988.
28. S.Y. Kung and J.N. Hwang. An algebraic projection analysis for optimal hidden units size and learning rates in back propagation learning. In Proc. 1988 IEEE Int. Conf. on Neural Networks, pages I363–I370, San Diego, California, July 1988.
29. Lbov, G.S., On representativeness of the sample size while choosing the effective measurement system. Zagoraiko, N.G., (eds.) Computing Systems Issue 22, 1966, Inst. of Math. Press, Novosibirsk, 39–58.
30. Lippmann, R.P., An introduction to computing with neural nets. IEEE ASSP Magazine 4:2 (Apr. 1988), 4–22.
31. Lippmann, R.P., Pattern classification using neural networks. IEEE Communications Magazine, Nov. 1989, 47–64.
32. Minsky, M., Papert, S., Perceptrons: An introduction to computational geometry, 1969, MIT Press, Cambridge, MA.
33. J.R. Movellan. Error functions to improve noise resistance and generalization in back propagation networks. In Proceedings of IEEE IJCNN1990, pages I557–I560. Lawrence Erlbaum Associates, Inc., 1990.
34. Y. Nisikawa, H. Kita, and A. Kawamura. NN/I: A neural network which divides and learns environments. In Proceedings of IEEE IJCNN1990, pages I684–I687. Lawrence Erlbaum Associates, Inc., 1990.
35. Pikelis, V., Comparison of methods of computing the expected classification errors. Automation and Remote Control, 1976, 59–63 (No.5), (in Russian).
36. Pivoriunas, V., Raudys, S., On the accuracy of “learning-one-out estimate”. Statistical Problems of Control, Issue 27, 1978, Inst. of Math. and Cyb. Press, Vilnius, 53–70 (in Russian).
37. S. Raudys. On the problems of sample size in pattern recognition. In Proc. 2nd All-Union Conf. on Statistical Methods in control theory, pages 64–67, Moscow: Nauka, 1970. (in Russian).
38. S. Raudys. Statistical Classification in the Case of Essentially Limited Sample Size. PhD thesis, Inst. of Math. and Cyb.,Lithuanian Acad. of Sci., Vilnius, 1977. (in Russian).
39. S. Raudys. Comparison of the estimates of the probability of misclassification. In Proc. 4th Int Joint Conf. Pattern Recognition, pages 280–282, Kyoto, Nov. 1978.
40. Raudys, S., Determination of optimal dimensionality in statistical pattern recognition. Pattern Recognition 11 (1979), 263–270.
41. Raudys, S., Influence of sample size on the accuracy of model selection in pattern recognition. Statistical Problems of Control, Issue 50, 1981, Inst. of Math. and Cyb. Press, Vilnius, 9–30 (in Russian).
42. S. Raudys and A. K. Jain. Small sample size effects in statistical pattern recognition: Recommendations for practitioners. To appear in IEEE Trans. on Pattern Anal. Machine Intell., 1991.
43. Raudys, S., Pikelis, V., On dimensionality, sample size, classification error and complexity of classification algorithm in pattern recognition. IEEE Trans. Pattern Anal. Machine Intell. PAMI-2:3 (1980), 242–252.
44. Raudys, S., Pikelis, V., Collective selection of the best version of a pattern recognition system. Pattern Recognition Letters 1 (1982), 7–13.
45. S. Raudys, V. Pikelis, and D. Stasaitis. An influence of the number of initial and final features, and correlation between them on the accuracy of feature selection. In Statistical Problems of Control, Issue 74. Inst. of Math. and Cyb. Press, Vilnius, 1986. (in Russian).
46. Rosenblatt, R., Principles of Neurodynamics, 1959, Spartan Books, New York.
47. Rumelhart, D.E., Hinton, G.E., Williams, R.J., Learning internal representations by error propagation. Rumelhart, D.E., McClelland, J.L., (eds.) Parallel Distributed Processing, 1986, MIT Press, Cambridge, MA.
48. A.G. Senin. Analysis of optimality criteria used in pattern recognition. Avtometrika No.5, Novosibirsk, pages 20–25, 1971.
49. Smith, F.W., Small-sample optimality of design techniques for linear classifiers of Gaussian patterns. IEEE Trans. Inf. Theory IT-18:No.1 (1972), 118–126.
50. Tsypkin, Y.Z., Adaptation, learning and self-learning in automatic systems. Automatic and Remote Control(USSR J.), 1966, 23–61 (No.1).
51. Vapnik, V.N., Estimation of Dependences based on Empirical Data, 1979, Nauka, Moscow Springer NY, 1982.
52. Widrow, B., Winter, R.C., Neural nets for adaptive filtering and adaptive pattern recognition. IEEE Trans. Comput., March 1988, 25–39.
53. A. Wieland and R. Leighton. Geometric analysis of neural network capabilities. In IEEE 1st Int'l Conf. on Neural Networks, pages III–385, June 1987.