Multilayer Potts perceptrons with Levenberg-Marquardt learning

IEEE Transactions on Neural Networks

Volumn 19, Issue 12, 2008, Pages 2032-2043

  Wu, Jian Ming ^a  

^a NATIONAL DONG HWA UNIVERSITY   (Taiwan)

Author keywords

Gaussian array; Population encoding; Post nonlinear projection; Potts encoding; Sparse coding; Supervised dimensionality reduction; Supervised learning

Indexed keywords

CYBERNETICS; EDUCATION; ELECTRIC LOADS; ENCODING (SYMBOLS); FUNCTIONS; MATHEMATICAL MODELS; MULTILAYERS; NEURAL NETWORKS; PATTERN RECOGNITION SYSTEMS; POLYNOMIAL APPROXIMATION; PROBABILITY DENSITY FUNCTION; PROGRAMMING THEORY; SUPERVISED LEARNING; TRELLIS CODES;

GAUSSIAN ARRAY; POPULATION ENCODING; POST NONLINEAR PROJECTION; POTTS ENCODING; SPARSE CODING; SUPERVISED DIMENSIONALITY REDUCTION;

TRANSFER FUNCTIONS;

ALGORITHM; ARTICLE; ARTIFICIAL INTELLIGENCE; AUTOMATED PATTERN RECOGNITION; COMPUTER SIMULATION; METHODOLOGY; THEORETICAL MODEL;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; COMPUTER SIMULATION; MODELS, THEORETICAL; PATTERN RECOGNITION, AUTOMATED;

EID: 57749207082     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/TNN.2008.2003271     Document Type: Article

References (34)

1. F. Rosenblatt, Principles of Neurodynamics: Perceptrons and the Theory of Brain Mechanisms. Washington, DC: Spartan Books, 1962.
2. D. E. Rumelhart, "Learning representations by back-propagation errors," Nature, vol. 323, p. 533, 1986.
3. P. J. Werbos, "Backpropagation: Past and the future," in Proc. IEEE Int. Conf. Neural Netw., Jul. 1988, vol. 1, pp. 343-353.
4. B. Widrow, Generalization and Information Storage in Networks of Adaline Neuron in Self-Organizing Systems, M. Yovitz, G. Jacobi, and G. Goldstein, Eds. Washington, DC: Spartan Books, 1962, pp. 435-461.
5. B. Widrow, "30 years of adaptive neural networks: Perceptron, Madaline, and backpropagation," Proc. IEEE, vol. 78, no. 9, pp. 1415-1442, Sep. 1990.
6. S. Ferrari and R. F. Stengel, "Smooth function approximation using neural networks," IEEE Trans. Neural Netw., vol. 16, no. 1, pp. 24-38, Jan. 2005.
7. J. M. Wu, Z. H. Lin, and P. H. Hsu, "Function approximation using generalized Adalines," IEEE Trans. Neural Netw., vol. 17, no. 3, pp. 541-558, May 2006.
8. S. Haykin, Neural Networks: A Comprehensive Foundation. Upper Saddle River, NJ: Prentice-Hall, 1999.
9. J. Hertz, A. Krogh, and R. G. Palmer, Introduction to the Theory of Neural Computation. Reading, MA: Addison-Wesley, 1991.
10. S. Haykin, Neural Networks: A Comprehensive Foundation. Upper Saddle River, NJ: Prentice-Hall, 1999.
11. C. M. Bishop, Neural Networks for Pattern Recognition. Oxford, U.K.: Clarendon Press, 1995.
12. T. Hayakawa, W. M. Haddad, N. Hovakimyan, and V. Chellaboina, "Neural network adaptive control for nonlinear nonnegative dynamical systems," IEEE Trans. Neural Netw., vol. 16, no. 2, pp. 399-413, Mar. 2005.
13. L. Ljung, System Identification - Theory for the User. Englewood Cliffs, NJ: Prentice-Hall, 1987.
14. R. Fletcher, Practical Methods of Optimization. New York: Wiley, 1987.
15. E. Barnard, "Optimization for training neural nets," IEEE Trans. Neural Netw., vol. 3, no. 2, pp. 232-240, Mar. 1992.
16. M. NØrgaard, O. Ravn, N. K. Poulsen, and L. K. Hansen, Neural Networks for Modelling and Control of Dynamic Systems. London, U.K.: Springer-Verlag, 2000.
17. M. T. Hagan and M. B. Menhaj, "Training feedforward networks with the Marquardt algorithm," IEEE Trans. Neural Netw., vol. 5, no. 6, pp. 989-993, Nov. 1994.
18. R. Battiti, "1st-order and 2nd-order methods for learning - Between steepest descent and Newton method," Neural Comput., vol. 4, pp. 141-166, 1992.
19. C. Charalambous, "Conjugate-gradient algorithm for efficient training of artificial neural networks," Inst. Electr. Eng. Proc.-G Circuits Devices Syst., vol. 139, pp. 301-310, 1992.
20. >P. Dayan and L. F. Abbott, Theoretical Neuroscience: Computational and Mathematical Modeling of Neural Systems. Cambridge, MA: MIT Press, 2001.
21. C. Peterson and B. Södergerbg, "A new method for mapping optimization problems onto neural network," Int. J. Neural Syst., vol. 1, pp. 3-22, 1989.
22. J. M. Wu, "Potts models with two sets of interactive dynamics," Neurocomputing, vol. 34, pp. 55-77, 2000.
23. J. M. Wu and S. J. Chiu, "Independent component analysis using Potts models," IEEE Trans. Neural Netw., vol. 12, no. 2, pp. 202-211, Mar. 2001.
24. J. M. Wu, "Natural discriminant analysis using interactive Potts models," Neural Computat., vol. 14, no. 3, pp. 689-713, Mar. 2002.
25. J. M. Wu and Z. H. Lin, "Learning generative models of natural images," Neural Netw., vol. 15, no. 3, pp. 337-347, 2002.
26. J. Cook, "The mean-field theory of a Q-state neural network model," J. Phys. A, Math. Gen., vol. 22, pp. 2000-2012, 1989.
27. I. Kanter, "Potts-Glass models of neural networks," Phys. Rev. A, Gen. Phys., vol. 37, pp. 2739-2742, 1988.
28. H. Rieger, "Storing an extensive number of grey-toned patterns in a neural network using multistate neurons," J. Phys. A, Math. Gen., vol. 23, pp. L1273-L1279, 1990.
29. H. Vogt and A. Zippelius, "Invariant recognition in Potts glass neural networks," J. Phys. A, Math. Gen., vol. 25, pp. 2209-2226, 1992.
30. Y. Nakamura, K. Torii, and T. Munaka, "Neural-network model composed of multidimensional spin neurons," Phys. Rev. B, Condens. Matter, vol. 51, pp. 1538-1546, 1995.
31. B. Kryzhanovsky and L. Litinskii, "Vector associative memory models," Autom. Remote Control, vol. 64, no. 11, pp. 1782-1793, 2003.
32. D. Erdogmus and J. C. Principe, "Convergence properties and data efficiency of the minimum error entropy criterion in adaline training," IEEE Trans. Signal Process., vol. 51, no. 7, pp. 1966-1978, Jul. 2003.
33. A. S. Weigend and N. A. Gersehnfield, Time Series Prediction: Forecasting the Future and Understanding the Past. Reading, MA: Addison-Wesley, 1993, vol. 15.
34. E. W. M. Lee, C. P. Lim, R. K. K. Yuen, and S. M. Lo, "A hybrid neural network model for noisy data regression," IEEE Trans. Syst. Man Cybern. B, Cybern., vol. 34, no. 2, pp. 951-960, Apr. 2004.