Temporal BYY encoding, Markovian state spaces, and space dimension determination

IEEE Transactions on Neural Networks

Volumn 15, Issue 5, 2004, Pages 1276-1295

  Xu, Lei ^a  

^a CHINESE UNIVERSITY OF HONG KONG   (Hong Kong)

Author keywords

[No Author keywords available]

Indexed keywords

LEARNING SYSTEMS; MARKOV PROCESSES; MATHEMATICAL MODELS; MATRIX ALGEBRA; OPTIMIZATION; SIGNAL ENCODING;

GATED MULTITEMPORAL MODELS; HARMONY LEARNING; HIDDEN MARKOV MODEL; LINEAR STATE SPACES; SPACE DIMENSION; TEMPORAL BAYESIAN YING-YANG SYSTEM; TEMPORAL FACTOR ANALYSIS;

NEURAL NETWORKS;

EID: 3843066216     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/TNN.2004.833302     Document Type: Article

References (42)

1. S. Amari, "Information geometry of the EM and em algorithms for neural networks," Neural Netw., vol. 8, no. 9, pp. 1379-1408, 1995.
2. T. W. Anderson and H. Rubin, "Statistical inference in factor analysis," in Proc. 3rd Berkeley Symp. Mathemtaics Statistical Probabilities, vol. 5, Berkeley, CA, 1956, pp. 111-150.
3. H. Akaike, "A new look at the statistical model identification," IEEE Trans. Automat. Contr., vol. AC-19, pp. 714-723, June 1974.
4. H. Bozdogan, "Model selection and Akaike's informafion criterion: The general theory and its analytical extension," Psychometrika, vol. 52, pp. 345-370, 1987.
5. P. Baldi and S. Brunak, Bioinformatics: The Machine Learning Approach. Cambridge, MA: MIT Press, 2001.
6. R.G. Brown and P. Y. C. Hwang, Introduction to Random Signals and Applied Kalman Filtering. New York: Wiley, 1997.
7. K. C. Chiu and L. Xu et al., "Stock price and index forecasting by arbitrage pricing theory-based gaussian TFA learning," in Intelligent Data Engineering and Automated Learning - IDEAL 2002, H. yin et al., Eds. New York: Springer-Verlag, 2002, vol. LNCS 2412, pp. 366-371.
8. G. A. Carrpenter and S. Grossberg, "A massively parallel architecture for a self-organizing neural pattern recognition machine," Computer Vis. Graph. Image Process., vol. 37, pp. 54-115, 1987.
9. P. Dayan et al., "The Helmholtz machine," Neural Computat., vol. 7, no. 5, pp. 889-904, 1995.
10. T. Edelman and S. Smith, "The geometry of algorithms with orthogonal constraints," SIAM J. Matrix Anal. Appl., vol. 20, pp. 303-353, 1998.
11. G. E. Hinton and R. S. Zemel, "Autoencoders, minimum description length and Helmholtz free energy," Advances in NIPS, vol. 6, pp. 3-10, 1994.
12. X. L. Hu and L. Xu, "A comparative study of several cluster number selection criteria," in Intelligent Data Engineering and Automated Learning - IDEAL, Lecture Notes in Computer Science 2690, 2003, pp. 195-202.
13. X. L. Hu, "A comparative investigation on subspace dimension determination," Neural Netw., 2004, to be published.
14. R. E. Kalman, "A new approach to linear filtering and prediction problems," Trans. ASME, J Basic Eng., March 1960.
15. M. Kawamoto, "Cerebellum and motor cobtrol," in The Handbook of Brain Theory and Neural Networks, Second ed, M. Arbib, Ed. Cambridge, MA: MIT Press, 2002, pp. 190-195.
16. A. A. Neath and J. E. Cavanaugh, "Regression and time series model selection using variants of the Schwarz information criterion," Communications Statist. A, vol. 26, pp. 559-580, 1997.
17. L. Rabiner and B. H. Juang, Fundamentals of Speech Recognition. Englewood Cliffs, NJ: Prentice-Hall, 1993.
18. J. Rissanen, "Stochastic complexity and modeling," Ann. Statist., vol. 14, no. 3, pp. 1080-1100, 1986.
19. J. Rissanen, "Hypothesis selection and testing by the MDL principle," Comput. J., vol. 42, no. 4, pp. 260-269, 1999.
20. L. Saul and M. I. Jordan "Exploiting tractable structures in intractable networks," in Advances in Neural Information Processing Systems. Cambridge, MA: MIT Press, 1995, vol. 8, pp. 486-492.
21. G. Schwarz, "Estimating the dimension of a model," Ann. Statist., vol. 6, pp. 461-464, 1978.
22. M. Stone, "Cross-validation: A review," Math. Oper. Statist., vol. 9, pp. 127-140, 1978.
23. A. N. Tikhonov and V. Y. Arsenin, Solutions of ILL-Posed Problems. Washington, DC: Winston, 1977.
24. V. N. Vapnik, The Nature of Statistical Learning Theory. New York: Springer-Verlag, 1995.
25. C. S. Wallace and D. R. Dowe, "Minimum message length and Kolmogorov complexity," Comput. J., vol. 42, no. 4, pp. 270-280, 1999.
26. L. Xu, "Advances on BYY harmony learning: Information theoretic perspective, generalized projection geometry, and independent factor autodetermination," IEEE Trans. Neural Networks., vol. 15, pp. 885-902, July 2004.
27. L. Xu, "Independent component analysis and extensions with noise and time: A Bayesian Ying-Yang learning perspective," Neural Inform. Process. Lett. Rev., vol. 1, no. 1, pp. 1-52, 2003.
28. L. Xu, "BYY learning, regularized implementation, and model selection on modular networks with one hidden layer of binary units," Neurocomput., vol. 51, pp. 227-301, 2003.
29. L. Xu, "Data smoothing regularization, multi-sets-learning, and problem solving strategies," Neural Netw., vol. 15, no. 5-6, pp. 817-825, 2003.
30. L. Xu, "BYY harmony learning, structural RPCL, and topological self-organizing on mixture models," Neural Netw., vol. 15, pp. 1125-1151, 2000a.
31. L. Xu, "Bayesian Ying Yang harmony Learning," in The Handbook of Brain Theory and Neural Networks, Second ed, M. A. Arbib, Ed. Cambridge, MA: MIT Press, 2002b, pp. 1231-1237.
32. L. Xu, "BYY harmony learning, independent state space and generalized apt financial analyzes," IEEE Trans. Neural Networks, vol. 12, pp. 822-849, July 2001.
33. L. Xu, "Best harmony, unified RPCL and automated model selection for unsupervised and supervised learning on Gaussian mixtures, three-layer nets and ME-RBF-SVM models," Int. J. Neural Syst., vol. 11, no. 1, pp. 43-69, 2001.
34. L. Xu, "Temporal BYY learning for state space approach, hidden Markov model and blind source separation," IEEE Trans. Signal Processing, vol. 48, pp. 2132-2144, July 2000.
35. L. Xu, "Temporal Bayesian Ying-Yang dependence reduction, blind source separation and principal independent components," in Proc. Int. Joint Conf. Neural Networks '99, vol. 2, Washington, D.C., pp. 1071-1076.
36. L. Xu, "Bayesian kullback Ying-Yang dependence reduction theory," Neurocomput., vol. 22, no. 1-3, pp. 81-112, 1998.
37. L. Xu, "Bayesian Ying-Yang system and theory as a unified statistical learning approach: (v) temporal modeling for temporal perception and control," in Proc. Int. Conf. Neural Information Processing (ICONIP'98), vol. 2, Kitakyushu, Japan, Oct. 21-23, 1998, pp. 877-884.
38. L. Xu, C. C. Cheung, and S.-1. Amari, "Learned parametric mixture based ICA algorithm," Neurocomput., vol. 22, no. 1-3, pp. 69-80, 1998.
39. L. Xu, "RBF nets, mixture experts, and Bayesian Ying-Yang learning," Neurocomput., vol. 19, no. 1-3, pp. 223-257, 1998.
40. L. Xu, "A Unified Learning Scheme: Bayesian-Kullback Ying-Yang machine," in Advances in Neural Information Processing Systems, D. S. Touretzky et al., Eds. Cambridge, MA: MIT Press, 1995-1996, vol. 8, pp. 444-450.
41. L. Xu, A. Krzyzak, and E. Oja, "Rival penalized competitive learning for clustering analysis, RBF net and curve detection," IEEE Trans. Neural Networks, vol. 4, pp. 636-649, July 1993.
42. L. Xu, "Least mean square error reconstruction for self-organizing neural-nets," Neural Netw., vol. 6, pp. 627-648, 1991-93.