Operations and learning in neural networks for robust prediction

IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics

Volumn 27, Issue 3, 1997, Pages 402-411

  Kogiantis, Achilles G ^a,b   Papantoni Kazakos, Titsa ^a,c  

^a IEEE   (United States)

^b UNIVERSITY OF LOUISIANA AT LAFAYETTE   (United States)

^c University of Alabama   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

BACKPROPAGATION; CONVERGENCE OF NUMERICAL METHODS; DATA STRUCTURES; LEARNING ALGORITHMS; NUMERICAL METHODS; ONLINE SYSTEMS; PROBABILITY; ROBUSTNESS (CONTROL SYSTEMS);

ONLINE LEARNING ALGORITHM; STOCHASTIC APPROXIMATION METHODS; STOCHASTIC NEURAL NETWORK;

NEURAL NETWORKS;

EID: 0031166981     PISSN: 10834419     EISSN: None     Source Type: Journal    
DOI: 10.1109/3477.584948     Document Type: Article

References (42)

1. F. Abdelhamid, "Transformation of observations in stochastic approximation," Ann. Statist., vol. 1. pp. 1158-1174, Nov. 1973.
2. D. H. Ackley, G. E. Hinton, and T. J. Sejnowski, "A learning algorithm for Boltzman machines," Cognitive Sci., vol. 9, pp. 147-169, 1985.
3. S. Amari, K. Kurata, and H. Nagaoka, "Information geometry of Boltzman machines," IEEE Trans. Neural Networks, vol. 3, pp. 260-271, 1992.
4. A. R. Barron, F. W. va Straten, and R. L. Barron, "Adaptive learning network approach to weather forecasting: A summary," in Proc. IEEE Int. Conf. Cybernetics Soc., 1977, pp. 724-727.
5. R. Beran, "Adaptive autoregressive process," Ann. Inst. Statist. Math., vol. 28, pp. 77-89, 1976.
6. R. E. Blahut, "Hypothesis testing and information theory," IEEE Trans. Inform. Theory, vol. IT-20, pp. 405-417, 1987.
7. J. R. Blum, "Multidimensional stochastic approximation procedure," Ann. Math. Statist., vol. 25, pp. 737-744, 1954.
8. J. Elman and D. Zipser, "Learning the hidden structure of speech," J. Acoust. Soc. Amer., vol. 83, pp. 1615-1626, 1988.
9. V. Fabian, "On asymptotic normality in stochastic approximation," Ann. Math. Statist., vol. 39, pp. 1327-1332, 1968.
10. R. A. Fisher, "The goodness of fit of regression formulae and the distribution of regression coefficients," J. Roy. Stat. Soc., vol. 85, p. 597, 1922.
11. _, "Probability likelihood and quantity of information in the logic uncertain inference," in Proc. Roy. Soc., 1934, vol. A, p. 146.
12. L. Gerencser, "Parameter tracking of time-varying continuous-time linear stochastic systems," in Modeling, Identification and Robust Control, C. I. Byrnes and A. Lindquist, Eds. Amsterdam, The Netherlands: North-Holland, 1986, pp. 581-594.
13. P. Gorman and T. Sejnowski, "Analysis of hidden units in a layered network trained to classify sonar targets," Neural Networks, vol. 1, pp. 75-90, 1988.
14. B. Gustaffson, H. Wigstrom, W. C. Abramham, and Y. Y. Huang, "Long-term potentiation in the hippocampus using depolarizing current pulses as the conditioning stimulus to single volley synaptic potentials," J. Neuroscience, vol. 7, no. 3, pp. 774-780, 1987.
15. R. L. Kashyap and C. C. Blaydon, "Recovery of functions from noisy measurements taken at randomly selected points and its application to pattern classification," Proc. IEEE, vol. 54, pp. 1127-1129, 1966.
16. R. L. Kashyap, C. C. Blaydon, and K. S. Fu, "Stochastic approximation," in Adaptive Learning and Pattern Recognition Systems, J. M. Mendel and K. S. Ku, Eds. New York: Academic, 1970, pp. 329-355.
17. D. Kazakos and P. Papantoni-Kazakos, Detection and Estimation. Rockville, MD: Computer Science, 1989.
18. J. Kiefer and J. Wolfowitz, "Stochastic estimation of the maximum of a regression function," Ann. Math. Statist., vol. 23, pp. 462-466, 1952.
19. H. Kushner, "Asymptotic global behavior for stochastic approximations and diffusions with slowly decreasing noise effects: Global minimization via Monte Carlo," SIAM J. Appl. Math., vol. 47, pp. 169-185, 1987.
20. H. Kushner and D. Clark, Stochastic Approximation Methods for Constrained and Unconstrained Systems. Berlin, Germany: Springer-Verlag, 1978.
21. W. B. Levy, "A computational approach to hippocampal function," in Computational Models of Learning in Simple Neural Systems, R. D. Hawkins and G. H. Bower, Eds. Orlando, FL: Academic, 1989.
22. _, "Maximum entropy prediction in neural networks," in Int. J. Conf. Neural Networks, 1990, vol. 1, pp. 7-10.
23. W. B. Levy, C. M. Colbert, and N. L. Desmond, "Elemental adaptive processes of neurons and synapses: A statistical/computational perspective," in Neuroscience and Connectionist Models, M. A. Gluck and D. E. Rumelhart, Eds. Hillsdale, NJ: Lawrence Erlbaum, 1990, pp. 187-235.
24. W. B. Levy and O. Steward, "Temporal contiguity requirements for long-term associative potentiation/depression in the hippocampus," Neuroscience, vol. 8, pp. 791-797, 1983.
25. L. Ljung, "Analysis of recursive stochastic algorithms," IEEE Trans. Automat. Contr., vol. 22, pp. 551-575, 1977.
26. L. Ljung and T. Soderstrom, Theory and Practice of Recursive Identification. Cambridge, MA: MIT Press, 1983.
27. M. Minsky and S. Papert, Perceptrons Cambridge, MA: MIT Press, 1969.
28. D. A. Pados, P. Papantoni-Kazakos, D. Kazakos, and A. G. Koyiantis, "On-line threshold learning for Neyman-Pearson distributed detection," IEEE Trans. Syst., Man, Cybern., vol. 24, pp. 1519-1531, Oct. 1994.
29. D. A. Pados and P. Papantoni-Kazakos, "New nonleast-squares neural network learning algorithms for hypothesis testing," IEEE Trans. Neural Networks, vol. 6, pp. 596-609, May 1995.
30. P. Papantoni-Kazakos, D. Kazakos, and K. Birmiwal, "Predictive analog-to-digital conversion for resistance to data outliers," Inform. Comput., vol. 97, Mar. 1992.
31. H. Robbins and S. Monroe, "A stochastic approximation method," Ann. Math. Stat., vol. 22, pp. 400-407, 1951.
32. F. Rosenblatt, "The perceptron: A perceiving and recognizing automaton," Rep. 85-60-1, Cornell Aeronautical Laboratory, Buffalo, NY, 1957.
33. D. E. Rumelhart, G. E. Hinton, and R. J. Williams, "Learning internal representations by error propagation," in Parallel Distributed Processing: Explorations in the Microstructures of Cognition, D. E. Rumelhart and J. L. McClelland, Eds. Cambridge, MA: MIT Press, 1986, vol. 1, no. 1, pp. 318-362, 1986.
34. A. Wald, "Tests of statistical hypotheses concerning several parameters when the number of observations is large," Trans. Amer. Math. Soc., vol. 54, pp. 462-482, 1943.
35. P. Werbos, "Beyond regression: New tools for prediction and analysis in the behavioral sciences," Ph.D. dissertation, Dept. of Applied Mathematics, Harvard Univ., 1974.
36. H. White, "Some asymptotic results for learning in single hidden-layer feedforward network models," J. Amer. Statist. Assoc., vol. 84, pp. 1003-1013, Dec. 1989.
37. B. Widrow and M. E. Hoff, "Adaptive switching circuits," in 1960 IRE WESCON Conv. Rec., 1960, pp. 96-104.
38. B. Widrow, "Generalization and information storage in networks of adaline neurons," in Self-Organizing Systems, M. D. Yovits, G. T. Jacobi, and G. D. Goldstein, Eds. Washington, DC: Spartan, 1962, pp. 435-461.
39. N. Wiener, Extrapolation, Interpolation and Smoothing of Stationary Series. Cambridge, MA: MIT Press, 1949.
40. N. Wiener and P. Masani, "The prediction theory of multivariate stochastic processes," pt. I. "The regularity condition," Acta Math. vol. 98, pp. 111-150, 1957, pt. II.
41. "The linear predictor," Acta Math., vol. 99, pp. 93-137, 1958.
42. T. Y. Young and R. A. Westerberg, "Stochastic approximation with nonstationary regression function," IEEE Trans. Inform. Theory, vol. 18, pp. 518-519, 1972.