Training MLPs layer by layer using an objective function for internal representations

Neural Networks

Volumn 9, Issue 1, 1996, Pages 83-97

  Lengellé, Régis ^b   Denœux, Thierry ^a  

^a UNIVERSITÉ DE TECHNOLOGIE DE COMPIÈGNE   (France)

^b NONE

Author keywords

Approximation; Architecture; Classification; Constructive algorithm; Discriminant analysis; Internal representation; Learning algorithm; Multilayer perceptrons

Indexed keywords

APPROXIMATION THEORY; BACKPROPAGATION; COMPUTATIONAL METHODS; COMPUTER SIMULATION; ENCODING (SYMBOLS); ERRORS; LEARNING ALGORITHMS; MATHEMATICAL MODELS; OPTIMIZATION; PERFORMANCE; STATISTICAL METHODS;

CONSTRUCTIVE ALGORITHM; DISCRIMINANT ANALYSIS; INTERNAL REPRESENTATION; MULTILAYER PERCEPTRONS;

NEURAL NETWORKS;

ALGORITHM; ARTICLE; ARTIFICIAL NEURAL NETWORK; LEARNING; PRIORITY JOURNAL;

EID: 0029972717     PISSN: 08936080     EISSN: None     Source Type: Journal    
DOI: 10.1016/0893-6080(95)00096-8     Document Type: Article

References (30)

1. Asoh, H., & Otsu, N. (1990). An approximation of nonlinear discriminant analysis by multilayer neural networks. In Proceedings IJCNN'90, San Diego (pp. III-211-216).
2. Baum, E. B., & Haussler, D (1989). What size net gives valid generalization. Neural Computing, 1(1), 151-160.
3. Baum, E. B., & Lang, K. J. (1991). Constructing hidden units using examples and queries. In R. P. Lippman, J. E. Moody, & D. S. Touretzky (Eds.), Neural information processing 3 (pp. 904-910). San Mateo, CA: Morgan Kaufmann.
4. Burton, R. M., & Farris, W. G. (1991). Reliable evaluation of neural networks. Neural Networks, 4, 411-415.
5. Courrieu, P. (1991). A distributed search algorithm for hard optimization. Technical Report TA9101, CREPCO, Université de Provence, F-13621 Aix-en-Provence Cedex 1.
6. Der Megreditchian, G (1983). Une identité algébrique intéressante pour la statistique mathématique. La Météorologie, VI(32), 5-14.
7. Duda, R. O., & Hart, P. E. (1973). Pattern classification and scene analysis. New York: John Wiley.
8. Fahlman, S. E., & Lebiere, C. (1990). The cascade-correlation learning architecture. In D. S. Touretzky (Ed.), Advances in neural information processing systems 2 (pp. 524-532). San Mateo, CA: Morgan Kaufmann.
9. Fukunaga, K. (1972). Introduction to statistical pattern recognition. Electrical Science. New York: Academic Press.
10. Gallinari, P., Thiria, S., & Fogelman-Soulie, F. (1988). Multilayer perceptrons and data analysis. In Proc. ICNN'88, San Diego (pp. 391-398).
11. Gallinari, P., Thiria, S., Badran, F., & Fogelman-Soulie, F. (1991). On the relations between discriminant analysis and multilayer perceptrons. Neural Networks, 4, 349-360.
12. Grossman, T., Meir, R., & Domany, E. (1989). Learning by choice of internal representations. In D. S. Touretzky, (Ed.), Advances in neural information processing systems 1 (pp. 73-80). San Mateo, CA: Morgan Kaufmann.
13. Haussler, D. (1988). Quantifying inductive bias: AI learning algorithms and valiant's learning framework. Artificial Intelligence, 36, 177-221.
14. Hinton, G. E. (1990). Connectionist learning procedures. In Y. Kodratoff & R. Michalski (Eds), Machine learning III. (pp. 555-610). San Mateo, CA: Morgan Kaufmann.
15. Hirose, Y., Yamashita, K., Hijiya, S. (1991). Back-propagation algorithm which varies the number of hidden units. Neural Networks, 4(1), 61-66.
16. Hornik, K. (1991). Approximation capabilities of multilayer feedforward networks. Neural Networks, 4, 251-257.
17. Kohonen, T., Chrisley, R., & Barna, G. (1989). Statistical pattern recognition with neural networks: Benchmarking studies. In L. Personnaz & G. Dreyfus (Eds.), Neural networks from models to applications. Paris: I.D.S.T.
18. Krogh, A., Thorebegsson, G. I., & Hertz, J. A. (1990). A cost function for internal representations. In D. S. Touretzky (Ed.) Advances in neural information processing systems 2 (pp. 733-740). San Mateo, CA: Morgan Kaufmann.
19. Lengellé, R., & Denoeux, T (1992). Optimizing multilayer networks layer per layer without back-propagation. In Igor Aleksander & John Taylor (Eds.), Artificial neural networks II (pp. 995-998). Amsterdam: North-Holland.
20. Marchand, M., Golea, M., & Ruján, P. (1990). A convergence theorem for sequential learning in two-layer perceptrons. Europhysics Letters, 11(6), 487-492.
21. Mézard, M., & Nadal, J. P. (1989). Learning in feedforward layered networks: The tiling algorithm. Journal of Physics A: Mathematical and General, 22, 2191-2203.
22. Rechstschaffen, A. & Kales, A. (1968) A manual of standardized terminology, techniques and scoring system for sleep stages of human subjects. Public Health Service, Washington, DC: U.S. Government Printing Office.
23. Rosenblatt, F. (1958). The perceptron: A probabilistic model for information storage and organization in the brain. Psychological Review, 65, 386-408.
24. Rumelhart, D. E. Hinton, G. E., & Williams, R. J. (1986). Learning internal representations by error propagation. In D. E. Rumelhart & J. McClelland (Eds), Parallel distributed processing. Cambridge, MA: MIT press.
25. Schaltenbrand, N., Lengellé, R., & Macher, J. P. (1993). Neural network model: Application to automatic analysis of human sleep. Computers and Biomedical Research, 26, 157-171.
26. Sontag, E. D. (1991). Remarks on interpolation and recognition using neural nets. In R. L. Lippman, J. E. Moody, & D. S. Touretzky (Eds.), Advances in neural information processing systems 3 (pp. 939-945). San Mateo, CA: Morgan Kaufmann.
27. Vapnik, V. N. (1983) Estimation of dependences based on empirical data. Springer series in statistics. Berlin: Springer-Verlag.
28. Webb, A. R., & Lowe, D. (1990). The optimised internal representation of multilayer classifier networks performs non linear discriminant analysis. Neural Networks, 3, 367-375.
29. Weigend, A.S., Rumelhart, D.E., & Huberman, B.A. (1991). Generalization by weight-elimination with application to forecasting. In R. P. Lippman, J. E. Moody, & D. S. Touretzky (Eds), Neural Information Processing, 3, (pp. 875-882). San Mateo, CA: Morgan Kaufmann.
30. Zollner, R., Schmitz, H. J., Wünsch, F., & Krey, U. (1992). Fast generating algorithm for a general three-layer perceptron. Neural Networks, 5, 771-777.