Stochastic neural networks with applications to nonlinear time series

Journal of the American Statistical Association

Volumn 96, Issue 455, 2001, Pages 968-981

  Lai, Tze Leung ^a   Po Shing Wong, Samuel ^b  

^a STANFORD UNIVERSITY   (United States)

^b HONG KONG UNIVERSITY OF SCIENCE AND TECHNOLOGY   (Hong Kong)

Author keywords

Expectation maximization algorithm; Hidden markov models; Model selection; Neural networks; Nonlinear stochastic systems

Indexed keywords

EID: 0442327789     PISSN: 01621459     EISSN: 1537274X     Source Type: Journal    
DOI: 10.1198/016214501753208636     Document Type: Article

References (49)

1. Anderson, J. A. (1994), Introduction to Practical Neural Modeling, Cambridge, MA: MIT Press.
2. Anderson, N. H., and Titterington, D. M. (1999), “Boltzmann Machines: Statistical Associations and Algorithms for Training,” in Neural Network Systems: Techniques and Applications (Vol. 3, Implementation Techniques), ed. C. T. Leondes, New York: Academic Press, pp. 51–89.
3. Barron, A. R. (1993), “Universal Approximation Bounds for Superpositions of a Sigmoid Function,” IEEE Transactions on Information Theory, 39, 930–945.
4. Baum, E. B., and Lang, K. S. (1991), “Constructing Hidden Units Using Examples and Queries,” Advances in Neural Information Processing Systems, 3, 904–910.
5. Chen, H. F., and Guo, L. (1991), Identification and Stochastic Adaptive Control, Boston: Birkhauser.
6. Chen, R., and Tsay, R. S. (1993a), “Functional-Coefficient Autoregressive Models,” Journal of the American Statistical Association, 88, 298–308.
7. Chen, R., and Tsay, R. S. (1993b), “Nonlinear Additive ARX Models,” Journal of the American Statistical Association, 88, 955-967.
8. Cox, D. R., and Snell, E. J. (1968), “A General Definition of Residuals” (with discussion), Journal of the Royal Statistical Society, Ser. B, 30, 248–275.
9. de Freitas, J. F. G., Niranjan, M., Gee, A. H., and Doucet, A. (2000), “Sequential Monte Carlo Methods to Train Neural Network Models,” Neural Computation, 12, 955–993.
10. Duflo, M. (1997), Random Iterative Models, New York: Springer-Verlag.
11. Durrett, R. (1996), Probability: Theory and Examples (2nd ed.), Belmont, CA: Duxbury.
12. Friedman, J. H. (1991), “Multivariate Adaptive Regression Splines” (with discussion), Annals of Statistics, 19, 1–141.
13. Friedman, J. H., and Steutzle, W. (1981), “Projection Pursuit Regression,” Journal of the American Statistical Association, 76, 817–823.
14. Gupta, M. M., and Rao, H. H. (1994), Neural Control Systems, New York: IEEE Press.
15. Hansen, L. P. (1982), “Large Sample Properties of Generalized Method of Moments Estimators,” Econometrica, 50, 1029–1054.
16. Hastie, T. J. and Tibshirani, R. J. (1990), Generalized Additive Models, London: Chapman and Hall.
17. Hertz, J., Krogh, A., and Palmer, R. (1991), Introduction to the Theory of Neural Computation, Redwood City, CA: Addison-Wesley.
18. Huerta, G., Jiang, W., and Tanner, M. (2000), “Time Series Modeling via Hierarchical Mixtures,” working paper, Northwestern University, Dept. of Statistics.
19. Hutchison, J., Lo, A., and Poggio, T. (1994), “A Nonparametric Approach to the Pricing and Hedging of Derivatives via Learning Networks,” Journal of Finance, 49, 851–889.
20. Jacobs, R. A., Jordan, M. I., Nowlan, S. J., and Hinton, G. E. (1991), “Adaptive Mixture of Local Experts,” Neural Computation, 3, 79–87.
21. Jordan, M., and Jacobs, R. (1994), “Hierachical Mixtures of Experts and the EM Algorithm,” Neural Computation, 6, 181–214.
22. Kantz, H., and Shreiber, T. (1997), Nonlinear Time Series Analysis, Cambridge, U.K.: Cambridge University Press.
23. LeCam, L., and Yang, G. L. (1990), Asymptotics in Statistics: Some Basic Concepts, New York: Springer-Verlag.
24. Lewis, P. A. W., and Stevens, J. G. (1991), “Nonlinear Modeling of Time Series Using Multivariate Adaptive Regression Splines (MARS),” Journal of the American Statistical Association, 86, 864–877.
25. Liu, J. S., and Chen, R. (1998), “Sequential Monte Carlo Methods for Dynamic Systems,” Journal of the American Statistical Association, 93, 1032–1044.
26. Liu, J. S., Chen, R., and Wong, W. H. (1998), “Rejection Control and Sequential Importance Sampling,” Journal of the American Statistical Association, 93, 1022–1031.
27. McCulloch, W. S., and Pitts, W. (1943), “A Logical Calculus of Ideas Imminent in Neural Activity,” Bulletin of Mathematical Biophysics, 5, 115–133.
28. Neal, R. (1992), “Connectionist Learning of Belief Networks,” Artificial Intelligence, 56, 71–113.
29. Neal, R. (1996), Bayesian Learning for Neural Networks, New York: Springer-Verlag.
30. Peretto, P. (1984), “Collective Properties of Neural Networks: A Statistical Physics Approach,” Biological Cybernetics, 50, 51–62.
31. Polycarpou, M. M. (1996), “Stable Adaptive Neural Control Scheme for Nonlinear Systems,” IEEE Transactions on Automatic Control, 41, 447–451.
32. Redner, R. A., and Walker, H. F. (1984), “Mixture Densities, Maximum Likelihood and the EM Algorithm,” SIAM Review, 26, 195–239.
33. Rumelhart, D. E., Hinton, G. E., and Williams, R. J. (1986), “Learning Representations by Backpropagation Errors,” Nature, 323, 533–536.
34. Sanner, R. M., and Slotine, J. J. (1992), “Gaussian Networks for Direct Adaptive Control,” IEEE Transactions on Neural Nets, 41, 447–451.
35. Saul, L. K., Jaakola, T., and Jordan, M. (1996), “Mean Field Theory for Sigmoid Belief Networks,” Journal of Artificial Intelligence Research, 4, 61–76.
36. Shumway, R. H., and Katzoff, M. J. (1992), “Nonlinear and Non-Gaussian Modeling for Mortality Rates,” unpublished manuscript, University of California at Davis, Dept. of Statistics.
37. Stroeve, S., Kappan, H. J., and Gielen, S. (1999), “Stimulus Segmentation in a Stochastic Neural Network with Exogenous Signals,” in Proceedings of the 1999 International Conference on Artificial Neural Networks, pp. 732–737.
38. Tong, H., and Lim, K. S. (1980), “Threshold Autoregression, Limit Cycles and Cyclical Data” (with discussion), Journal of the Royal Statistical Society, Ser. B, 42, 245–292.
39. Tong, H. (1990), Nonlinear Time Series: A Dynamical Systems Approach, Oxford, U.K.: Oxford University Press.
40. Tresp, V., and Hofmann, R. (1998), “Nonlinear Time-Series Prediction with Missing and Noisy Data,” Neural Computation, 10, 731–747.
41. Venables, W. N., and Ripley, B. D. (1994), Modern Applied Statistics with S-Plus, New York: Springer-Verlag.
42. Weigend, A., and Gershenfeld, N. (1993), Time Series Prediction: Forecasting the Future and Understanding the Past, Reading, MA: Addison-Wesley.
43. Weigend, A., Rumelhart, D., and Huberman, B. (1991), “Predicting Sunspots and Exchange Rates with Connectionist Networks,” in Nonlinear Modeling and Forecasting, (eds. M. Casdagli and S. Eubank,), Redwood City, CA: Addison Wesley, pp. 395–432.
44. White, H. (1981), “Consequences and Detection of Misspecified Nonlinear Regression Models,” Journal of the American Statistical Association, 76, 419–433.
45. White, H. (1989), “Some Asymptotic Results for Learning in Single Hidden-Layer Feedforward Network Models,” Journal of the American Statistical Association, 84, 1003–1013.
46. White, H. (1992), Artificial Neural Networks: Approximation and Learning Theory, Cambridge, MA: Blackwell.
47. Wong, W. H., and Liang, F. (1997), “Dynamic Weighting in Monte Carlo and Optimization,” Proceedings of National Academy of Sciences USA, 94, 14220–14224.
48. Wu, C. F. J. (1983), “On the Convergence Properties of the EM Algorithm,” Annals of Statistics, 11, 95–103.
49. Zbikowski, R., and Hunt, K. J. (1996), Neural Adaptive Control Technology, Cleveland: World.