Convergent design of piecewise linear neural networks

Neurocomputing

Volumn 70, Issue 4-6, 2007, Pages 1022-1039

  Chandrasekaran, Hema ^a   Li, Jiang ^b   Delashmit, W H ^d   Narasimha, P L ^b   Yu, Changhua ^c   Manry, Michael T ^b  

^a NASA AMES RESEARCH CENTER   (United States)

^b University of Texas at Arlington   (United States)

^c FastVDO Inc   (United States)

^d LOCKHEED MARTIN   (United States)

Author keywords

Conjugate gradient method; Generalization; Local neural networks; Multilayer perceptron (MLP); Nonlinear networks; Piecewise linear function approximation

Indexed keywords

ALGORITHMS; APPROXIMATION THEORY; GRADIENT METHODS; NONLINEAR PROGRAMMING; PIECEWISE LINEAR TECHNIQUES;

CONJUGATE GRADIENT METHOD; LOCAL NEURAL NETWORKS; PIECEWISE LINEAR NETWORKS;

NEURAL NETWORKS;

ALGORITHM; ANALYTICAL ERROR; ARTICLE; ARTIFICIAL NEURAL NETWORK; CLUSTER ANALYSIS; CONTROLLED STUDY; CORRELATION ANALYSIS; LINEAR SYSTEM; MATHEMATICAL COMPUTING; NONLINEAR SYSTEM; NORMAL DISTRIBUTION; PERCEPTRON; PIECEWISE LINEAR NETWORK; PRIORITY JOURNAL; REGRESSION ANALYSIS; TIME SERIES ANALYSIS;

EID: 33846009462     PISSN: 09252312     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.neucom.2006.02.021     Document Type: Article

References (35)

1. Akaike H. A new look at the statistical model identification. IEEE Trans. Automat. Control AC 19 6 (1974) 716-723
2. Albus J.S. A new approach to manipulator control: the cerebellar model articulation controller (CMAC). J. Dyn. Systems Measure. Control-Trans. ASME 97 3 (1975) 220-227
3. Albus J.S. Data storage in the cerebellar model articulationcontroller (CMAC). J. Dyn. Systems Measure. Control-Trans. ASME 97 3 (1975) 228-233
4. Breiman L., Friedman J.H., Olshen R.A., and Stone C.J. Classification and Regression Trees (1984), Wadsworth, Belmont, CA
5. Changhua Yu., Manry M.T., and Li J. Effects of nonsingular preprocessing on feed forward network training. Int. J. Pattern Recogn. Artif. Intell. (2005)
6. Chen H.H., Manry M.T., and Chandrasekaran H. A neural network training algorithm utilizing multiple sets of linear equations. Neurocomputing 25 1-3 (1999) 55-72
7. Chien M., and Kuh E. Solving nonlinear resistive networks using piecewise-linear analysis and simplicial subdivision. IEEE Trans. Circuits Systems I CAS- 24 (1977) 305-317
8. Cormen T.C., Leiserson C.E., and Rivest R.L. Introduction to Algorithms (1992), The MIT Press, Cambridge, MA
9. W.H. Delashmit, M.T. Manry, New training algoriths for dependently initialized multilayer perceptrons, In: Conference Record of the 37th Annual Asilomar Conference on Signals, Systems, and Computers, 2003, pp. 581-585.
10. W.H. Delashmit, M.T. Manry, Recent developments in multilayer perceptron neural networks, In: Proceedings of the seventh Annual Memphis Area Engineering and Science Conference, MAESC, 2005.
11. Delve: Data for evaluating learning in valid experiments, URL: http://www.cs.utoronto.ca/_delve/data/datasets.html 〈http://www.cs.utoronto.ca/_delve/data/datasets.html〉
12. Dogaru R., Julian P., Chua L.O., and Glesner M. The simplicial neural cell and its mixed-signal circuit implementation: an efficient neuralnetwork architecture for intelligent signal processing in portable multimedia applications. IEEE Trans. Neural Network 13 4 (2002) 995-1008
13. Fletcher R. Conjugate direction methods. In: Murray W. (Ed). Numerical Methods for Unconstrained Optimization (1972), Academic Press, New York 73-86
14. Fogel D.B. An information criterion for optimal neural network selection. IEEE Trans. Neural Networks 2 5 (1991) 490-497
15. Friedman J.H. Multivariate adaptive regression splines. Ann. Stat. 19 1 (1991) 1-141
16. Fukunaga K. Introduction to Statistical Pattern Recognition (1990), Academic Press, Inc., New York
17. Fung K., Li Z., and Chen K.S. Backscattering from a randomly rough dielectric surface. IEEE Trans. Geosci. Remote Sensing 30 2 (1992) 356-369
18. Geman S., Bienenstock E., and Doursat R. Neural networks and the bias/variance dilemma. Neural Comput 4 1 (1992) 1-58
19. Gnanadesikan R. Methods for Statistical Data Analysis of Multivariate Observations (1997), Wiley, New York
20. Haykin S. Adaptive Filter Theory (1996), Prentice-Hall, Inc., Englewood Cliffs, NJ
21. Hu T.C. Combinatorial Algorithms (1982), Addison-Wesley Publishing Company, Inc., Reading, MA
22. K.K. Kim, A local approach for sizing the multilayer perceptron, Ph.D. Dissertation, The University of Texas at Arlington, 1996.
23. Koontz W.L.G., Narendra P.M., and Fukunaga K. Branch and bound clustering algorithm. IEEE Trans. Comput. C 24 9 (1975) 908-915
24. Kushner H.J., and Yin G.G. Stochastic Approximation Algorithms and Applications (1997), Springer, New York
25. Lapedes, R. Farber, Nonlinear signal processing using neural networks: prediction and system modeling, Technical Report LAUR- 87-2662, Los Alamos National Laboratory, Los Alamos, NM, 1987.
26. Lewis F.L. Optimal Estimation: with an Introduction to Stochastic Control Theory (1986), Wiley, New York
27. Li W., Lin J.-N., and Unbehauen R. Canonical representation of piecewise polynomial functions with nondegenerate linear domain partitions. IEEE Trans. Circuits Systems I: Fund. Theory Appl. 45 8 (1998) 838-848
28. Liu K., Subbarayan S., Shoults R.R., Manry M.T., Kwan C., Lewis F.L., and Naccarino J. Comparison of very short-term load forecasting techniques. IEEE Trans. Power Systems 11 2 (1996) 877-882
29. M.T. Manry, X. Guan, S.J. Apollo, L.S. Allen, W.D. Lyle, W. Gong, Output weight optimization for the multi-layer perceptron, Signals, systems and computers, Record of The 26th Asilomar Conference, Vol. 1 (1992) 502-506.
30. Narendra P.M., and Fukunaga K. Branch and bound algorithm for feature subset selection. IEEE Trans. Comput. C 26 9 (1977) 917-922
31. Oh Y., Sarabandi K., and Ulaby F.T. An empirical model and an inversion technique for radar scattering from bare soil surfaces. IEEE Trans. Geosci. Remote Sensing (1992) 370-381
32. Späth H. Cluster Analysis Algorithms for Data reduction and Classification of Objects (1980), Ellis Horwood Limited, Chichester, UK
33. Srinath M.D., Rajasekaran P.K., and Viswanathan R. Introduction to Statistical Signal Processing with Applications (1996), Prentice-Hall, Inc., Englewood Cliffs, NJ
34. S. Subbarayan, K. Kim, M.T. Manry, V. Devarajan, H. Chen, Modular neural network architecture using piecewise linear mapping, In: Thirtieth Asilomar Conference on Signals, Systems and Computers, Vol. 2, 1996, pp. 1171-1175.
35. Van Trees H.L. Detection, Estimation, and Modulation Theory-Part 1 (1968), Wiley, New York