Inverting feedforward neural networks using linear and nonlinear programming

IEEE Transactions on Neural Networks

Volumn 10, Issue 6, 1999, Pages 1271-1290

  Lu, Bao Liang ^a,b   Kita, Hajime ^c   Nishikawa, Yoshikazu ^d  

^a IEEE

^b RIKEN BRAIN SCIENCE INSTITUTE   (Japan)

^c TOKYO INSTITUTE OF TECHNOLOGY   (Japan)

^d OSAKA INSTITUTE OF TECHNOLOGY   (Japan)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHMS; COMPUTER ARCHITECTURE; INVERSE PROBLEMS; ITERATIVE METHODS; LINEAR PROGRAMMING; MULTILAYER NEURAL NETWORKS; NONLINEAR PROGRAMMING; PROBLEM SOLVING;

RADIAL BASIS FUNCTION (RBF) NEURAL NETWORKS; SEPARABLE PROGRAMMING (SP);

FEEDFORWARD NEURAL NETWORKS;

EID: 0033332526     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/72.809074     Document Type: Article

References (44)

1. M. B. Bazaraa, H. D. Sherali and C. W. Shetty, Nonlinear Programming Theory and Algorithms, 2nd ed. New York: Wiley, 1993.
2. L. Behera, M. Gopal, and S. Chaudhury, "On adaptive trajectory tracking of a robot manipulators using inversion of its neural emulator," IEEE Trans. Neural Networks, vol. 7, pp. 1401-1414, 1996.
3. C. W. Bishop, Neural Networks for Pattern Recognition. Oxford, U.K.: Oxford Univ. Press, 1995.
4. C. W. Bishop and C. Legleye, "Estimating conditional probability densities for periodic variables," in Advances in Neural Information Processing Systems, G. Tesauro, D. S. Touretzky, and T. K. Leen, Eds., vol. 7. Cambridge, MA: MIT Press, 1995, pp. 641-648.
5. V. Chvátal, Linear Programming San Francisco, CA: W. H. Freeman, 1983.
6. D. Colton and R. Kress, Inverse Acoustic and Electromagnetic Scattering Theory. Berlin, Germany: Springer-Verlag, 1992.
7. D. E. Demers, "Learning to invert many-to-one mapping," Ph.D. dissertation, Univ. California, San Diego, 1992.
8. R. A. Fisher, "The use of multiple measurements in taxonomic problems," Ann. Eugenics, vol. 7, pt. 2, pp. 179-188, 1936.
9. C. L. Giles and T. Maxwell, "Learning, invariance, and generalization in high-order neural networks," Appl. Opt., vol. 26, pp. 4972-4978, 1987.
10. G. M. L. Gladwell, Inverse Problems in Vibration. Dordrecht, The Netherlands: Kluwer, 1986.
11. D. A. Hoskins, J. N. Hwang, and J. Vagners, "Iterative inversion of neural networks and its application to adaptive control," IEEE Trans. Neural Networks, vol. 3, pp. 292-301, 1992.
12. J. N. Hwang, J. J. Choi, S. Oh, and R. J. Mark, II, "Query based learning applied to partially trained multilayer perceptrons," IEEE Trans. Neural Networks, vol. 2, pp. 131-136, 1991.
13. S. Haykin, Neural Networks. New York: Macmillan, 1994.
14. Y. Iiguni, H. Sakai, and H. Tokumaru, "A real time learning algorithm for a multilayered neural network based on extended Kalman filter," IEEE Trans. Signal Processing, vol. 40, 1992.
15. R. A. Jacobs, M. I. Jordan, M. I. Nowlan, and G. E. Hinton, "Adaptive mixtures of local experts," Neural Comput., vol. 3, pp. 79-87, 1991.
16. M. I. Jordan and D. E. Rumelhart, "Forward models: supervised learning with a distal teacher," Cognitive Sci., vol. 16, pp. 307-354, 1992.
17. M. I. Jordan and R. A. Jacobs, "Hierarchical mixtures of experts and the EM algorithm," Neural Comput., vol. 6, pp. 181-214, 1994.
18. M. Kawato, "Computational schemes and neural network models for formation and control of multijoint arm trajectory," in Neural Networks for Control, W. T. Miller, R. S. Sutton, and P. J. Werbos Eds. Cambridge, MA: MIT Press, 1990, pp. 197-228.
19. J. Kindermann and A. Linden, "Inversion of neural networks by gradient descent," Parallel Computing, vol. 14, pp. 277-286, 1990.
20. A. Kirsch and R. Kress, "A numerical method for an inverse scattering problem," in Inverse and Ill-posed Problems, H. W. Engl and C. W. Groetsch, Eds. Boston: Academic, 1987.
21. M. A. Kramer and J. A. Leonard, "Diagnosis using backpropagation neural networks: Analysis and criticism," Computers Chem. Eng., vol. 14, pp. 1323-1338, 1990.
22. M. Kuperstein, "neural model of adaptive hand-eye coordination for single postures," Science, vol. 239, pp. 1308-1311, 1988.
23. J. A. Leonard and M. A. Kramer, "Radial basis function networks for classifying process faults," IEEE Contr. Syst. Mag., vol. 11, pp. 31-38, 1991.
24. S. Lee and R. M. Kil, "Inverse mapping of continuous functions using local and global information," IEEE Trans. Neural Networks, vol. 5, pp 409-423, 1994.
25. Y. Le Cun, Y. B. Boser, J. S. Denker, D. Henderson, R. E. Howard, W. Hubbard, and L. D. Jackel, "Backpropagation applied to handwritten ZIP code recognition," Neural Computation, vol. 1, no. 4, pp. 541-551, 1989.
26. A. Linden, and J. Kindermann, "Inversion of multilayer nets," in Proc. Int. Joint Conf. Neural Networks, Washington, D.C., 1989, pp. II425-II430.
27. A. Linden, "Iterative inversion of neural networks and its applications," in Handbook of Neural Computation, M. Fiesler and R. Beale Eds. Oxford, U.K.: Inst. Phys. Publishing and Oxford Univ. Press, 1997, pp. B521-B527.
28. B. L. Lu, H. Kita, and Y. Nishikawa, "Examining generalization by network inversions," in Proc. 1991 Annu. Conf. Japanese Neural Network Soc., Tokyo, Japan, 1991, pp. 27-28.
29. _, "A new method for inverting nonlinear multilayer feedforward networks," in Proc. IEEE Int. Conf. Ind. Electron. Contr. Instrumentation, Kobe, Japan, Oct. 28-Nov. 1, 1991, pp. 1349-1354.
30. B. L. Lu, Y. Bai, H. Kita, and Y. Nishikawa, "An efficient multilayer quadratic perceptron for pattern classification and function approximation," in Proc. Int. Joint Conf. Neural Networks, Nagoya, Japan, Oct. 25-29, 1993, pp. 1385-1388.
31. B. L. Lu, H. Kita, and Y. Nishikawa, "Inversion of feedforward neural networks by a separable programming," in Proc. World Congr. Neural Networks, Portland, OR, July 11-15, 1993, pp. IV415-IV420.
32. B. L. Lu, "Architectures, learning and inversion algorithms for multilayer neural networks," Ph.D. dissertation, Dept. Elect. Eng., Kyoto Univ., Japan, 1994.
33. B. L. Lu and K. Ito, "Regularization of inverse kinematics for redundant manipulators using neural network inversions," in Proc. IEEE Int. Conf. Neural Networks, Perth, Australia, Nov. 27-Dec. 1, 1995, pp. 2726-2731.
34. _, "Transformation of nonlinear programming problems into separable ones by multilayer neural networks," in Mathematics of Neural Networks: Models, Algorithms, and Applications, S. W. Ellacott, J. C. Mason, and I. J. Anderson, Eds. Boston, MA: Kluwer, 1997, pp. 235-239.
35. C. E. Miller, "The simplex method for local separable programming," in Recent Advances in Mathematical Programming, R. L. Graves and P. Wolfe, Eds. New York: McGraw-Hill, 1963, pp. 89-100.
36. W. T. Miller, "Sensor-based control of robotic manipulators using a general learning algorithm," IEEE J. Robot. Automat., vol. 3, pp. 157-165, 1987.
37. K. G. Mehrotra, C. K. Mohan, and S. Ranka, "Bounds on the number of samples needed for neural learning," IEEE Trans. Neural Networks, vol. 2, pp. 548-558, 1991.
38. S. Moon and J. N. Hwang, "Robust speech recognition based on joint model and feature space optimization of hidden Markov models," IEEE Trans. Neural Networks, vol. 8, pp. 194-204, 1997.
39. M. J. D. Powell, "A fast algorithm for nonlinearly constrained optimization calculations," in Lecture Notes in Mathematics 630, G. A. Watson, Ed. Berlin: Springer-Verlag, 1978.
40. S. S. Rao, Engineering Optimization: Theory and Practice, 3rd ed. New York: Wiley, 1996.
41. D. E. Rumelhart, G. E. Hinton, and R. J. Williams, "Learning internal representations by error propagation," in Parallel Distributed Processing: Exploration in the Microstructure of Cognition, vol. 1, D. E. Rumelhart, J. L. McClelland, and PDP Research Group, Eds. Cambridge, MA: MIT Press, 1986, pp. 318-362.
42. R. S. Scalero and N. Tepedelenlioglu, "A fast new algorithm for training feedforward neural networks," IEEE Trans. Signal Processing, vol. 40, 1992.
43. R. Shamir, "The efficiency of the simplex method: A survey," Management Sci., vol. 33, pp. 301-334, 1987.
44. R. J. Williams, "Inverting a connectionist network mapping by backpropagation of error," in Proc. 8th Annu. Conf. Cognitive Sci. Soc.. Hillsdale, NJ: Lawrence Erlbaum, 1986, pp. 859-865.