Evolution and generalization of a single neurone: I. Single-layer perceptron as seven statistical classifiers

Neural Networks

Volumn 11, Issue 2, 1998, Pages 283-296

  Raudys, Šarunas ^a  

^a VILNIUS UNIVERSITY   (Lithuania)

Author keywords

Complexity; Dimensionality; Generalization error; Initialization; Overtraining; Sample size; Scissors effect; Single layer perceptron; Statistical classification

Indexed keywords

BACKPROPAGATION; COMPUTATIONAL COMPLEXITY; DECISION THEORY; ERROR ANALYSIS; MATRIX ALGEBRA; OPTIMIZATION; STATISTICAL METHODS;

GENERALIZATION ERROR; GENERALIZED FISHER DISCRIMINANT FUNCTION; STANDARD FISHER LINEAR DISCRIMINANT FUNCTION; STATISTICAL CLASSIFICATION;

NEURAL NETWORKS;

ALGORITHM; ARTICLE; COMPLEX FORMATION; COVARIANCE; DISCRIMINANT ANALYSIS; EVOLUTION; LEARNING; NERVE CELL NETWORK; NONHUMAN; PATTERN RECOGNITION; PRIORITY JOURNAL; STATISTICAL ANALYSIS;

EID: 0032029282     PISSN: 08936080     EISSN: None     Source Type: Journal    
DOI: 10.1016/S0893-6080(97)00135-4     Document Type: Article

References (38)

1. Abramson N., Braverman D. Learning to recognise patterns in a random environment. IRE Transactions on Information Theory. IT-8:1962;58-63.
2. Amari S. A theory of adaptive pattern classifiers. IEEE Transactions on Electronic Computers. EC-16:1967;299-307.
3. Anderson T.W. Classification by multivariate analysis. Psychometrika. 16:1951;31-50.
4. Anderson T.W., Bahadur R.R. Classification into two multivariate normal distributions with different covariance matrices. Ann. Math. Stat. 33:1962;420-431.
5. Babu C.C., Chen W.C. An optimal algorithm for pattern classification. Int. J. Control. 13(3):1971;577-586.
6. Boser, B., Guyon, I. and Vapnik, V. (1992). A training algorithm for optimal margin classifiers. In Proc. of the 5th Annual Workshop on Computational Learning Theory, Vol. 5 (pp. 144-152). ACM: Pittsburgh.
7. Cortes C., Vapnik V. Support-vector networks. Machine Learning. 20:1995;273-297.
8. Di Pillo P.J. Biased discriminant analysis: evaluation of the optimum probability of misclassification. Commun. Statist. Theory and methods. A8(14):1979;1447-1457.
9. Do-Tu H., Installe M. Learning algorithms for non-parametrical solution to the minimum error classification problem. IEEE Transactions on Computers. C-27:1978;648-659.
10. Duin, R.P.W. (1995, June). Small sample size generalization. Paper presented at 9th Scandinavian Conference on Image Analysis, (SCIA '95), Uppsala, Sweden, Vol. 2, 957-964.
11. Fisher R.A. The use of multiple measurements in taxonomic problems. Ann. of Eugenics (London). 7(2):1936;179-188.
12. Friedman J.M. Regularized discriminant analysis. J. American Statistical Association. 84:1989;165-175.
13. Fukunaga, K. (1990). Introduction to statistical pattern recognition. New York: Academic Press.
14. Geisser S. Posterior odds for multivariate normal classifications. J. Royal Stat. Soc., Ser. B. 21(1):1964;69-76.
15. Glucksman H. On improvement of a linear separation by extending the adaptive process with a stricter criterion. IEEE Transactions on Electronic Computers. EC-15(6):1966;941-944.
16. Gupta A.K. On the equivalence of two Classification Rules. Biometrical Journal. 19(5):1977;365-367.
17. Hoerl A.E., Kennard R.W. Ridge regression: biased estimation for orthogonal problems. Technometrics. 12:1970;55-67.
18. Ibaraki T., Muroga S. Adaptive linear classifier by linear programming. IEEE Transactions on Systems Science and Cybernetics. SSC-6:1970;53-62.
19. Keehn D. A note on learning for Gaussian properties. IEEE Transactions on Information Theory. IT-11:1965;126-131.
20. Koford J.S., Groner G.F. The use of an adaptive threshold element to design a linear optimal pattern classifier. IEEE Transactions on Information Theory. IT-12:1966;42-50.
21. Malinovskij, L.G. (1979). Hypotheses on subspaces in the problem of discriminant analysis of normal populations (pp. 195-206). Moscow: Nauka (in Russian).
22. McLachlan, G.J. (1992). Discriminant analysis and statistical pattern recognition. New York: Wiley.
23. Miyake, A. (1979). Mathematical aspects of optimal linear discriminant function. In COMPSAC'79, Proc. IEEE Comput. Soc. 3rd Int. Comput. Software and Applic. Conference, Chicago, IL (pp. 161-166). New York, N.Y.: IEEE.
24. Patterson D.W., Mattson R.L. A method of finding linear discriminant functions for a class of performance criteria. IEEE Transactions on Information Theory. IT-12:1966;380-387.
25. Pietrantonio H., Jurs P.C. Iterative least squares development of discriminant functions for spectroscopic data analysis by pattern recognition. Pattern Recognition. 4:1972;391-400.
26. Randles R.H., Brofitt J.D., Ramberg I.S., Hogg R.V. Generalised linear and quadratic discriminant functions using robust estimates. J. of American Statistical Association. 73(363):1978;564-568.
27. Raudys, S. (1967). On determining the training sample size of a linear classifier. In N. Zagoruiko (Ed.) Computing systems, Vol. 28 (pp. 79-87). Novosibirsk: Nauka, Institute of Mathematics, Academy of Sciences USSR (in Russian).
28. Raudys S. On the amount of a priori information in designing the classification algorithm. Proc. Acad. Sci. USSR, Technical. Cybernetics. 4:1972;168-174. (in Russian).
29. Raudys, S. (1991). Methods for overcoming dimensionality problems in statistical pattern recognition. A review. Zavodskaya Laboratorya, 3, 45 and 49-55 (in Russian). Paris: EC2&Cie.
30. Raudys, S. (1995, October). A negative weight decay or antiregularisation. In Proc. ICANN'95, Vol. 2, Paris (pp. 449-454).
31. Raudys, S. and Skurikhina, M. (1994, May). Small sample properties of ridge estimate of the covariance matrix in statistical and neural net classification. In New trends in probability and statistics, multivariate statistics and matrices in statistics, Proc. of the 5th Tartu Conference, Vol. 3, Tartu-Puhajarve, Estonia (pp. 237-245). Vilnius: TEV.
32. Rumelhart, D.E., Hinton, G.E. and Williams, R.L. (1986). Learning internal representations by error propagation. In Parallel distributed processing: explorations in the microstructure of cognition, Vol. 1 (pp. 318-362). Cambridge, MA: Bradford Books.
33. Schurmann, J. (1977). Polynomklassifikatoren fur Zeichenerkennung. Munchen/Wien: R. Oldenbourg Verlag.
34. Vapnik, V.N. and Chervonenkis, D.Ya. (1974). Theory of pattern recognition - statistical learning problems. Moscow: Nauka (in Russian).
35. Warmack R.E., Gonzales R.C. An algorithm for optimal solution of linear inequalities and its application to pattern recognition. IEEE Transactions on Computers. C-22:1973;1065-1075.
36. Widrow, B. and Hoff, M.E. (1960). Adaptive switching circuits. WESCON Convent. Record.
37. Wolf A.C. The estimation of the optimum linear decision function with a sequential random method. IEEE Transactions on Information Theory. IT-12:1966;312-315.
38. Yau S.S., Schumpert J.M. Design of pattern classifiers with the updating property using stochastic approximation techniques. IEEE Transactions on Computers. C-17:1968;908-1003.