State-space models: From the EM algorithm to a gradient approach

Neural Computation

Volumn 19, Issue 4, 2007, Pages 1097-1111

  Olsson, Rasmus Kongsgaard ^a   Petersen, Kaare Brandt ^a   Lehn Schiøler, Tue ^a  

^a TECHNICAL UNIVERSITY OF DENMARK   (Denmark)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 34247192555     PISSN: 08997667     EISSN: 1530888X     Source Type: Journal    
DOI: 10.1162/neco.2007.19.4.1097     Document Type: Article

References (27)

1. Bermond, O., & Cardoso, J.-F. (1999). Approximate likelihood for noisy mixtures. In Proceedings of the ICA Conference, (pp. 325-330). Aussois, France.
2. Bishop, C. (1995). Neural networks for pattern recognition. New York: Oxford University Press.
3. Broyden, C. G. (1965). A class of methods for solving nonlinear simultanoeus equations. Math. Comput., 19, 577-593.
4. Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977). Maximum likelihood from incomplete data via the EM algorithm. Journal of Royal Statistics Society, Series B, 39, 1-38.
5. Digalakis, V., Rohlicek, J., & Ostendorf, M. (1993). ML estimation of a stochastic linear system with the EM algorithm and its application to speech recognition. IEEE Transactions on Speech and Audio Processing, 1(4), 431-442.
6. Gupta, N. K., & Mehra, R. K. (1974). Computational aspects of maximum likelihood estimation and reduction in sensitivity calculations. IEEE Transactions on Automatic Control, AC-19(1), 774-783.
7. Højen-Sørensen, P. A., Winther, O., & Hansen, L. K. (2002). Mean field approaches to independent component analysis. Neural Computation, 14, 889-918.
8. Honkela, A., Valpola, H., & Karhunen, J. (2003). Accelerating cyclic update algorithms for parameter estimation by pattern searches. Neural Processing Letters, 17(2), 191-203.
9. Jamshidian, M., & Jennrich, R. I. (1993). Conjugate gradient acceleration of the EM algorithm. Journal of the American Statistical Association, 88, 221-228.
10. Jamshidian, M., & Jennrich, R. I. (1997). Acceleration of the EM algorithm using quasi-Newton methods. Journal of Royal Statistical Society, B, 59(3), 569-587.
11. Lachlan, G. J., & Krishnan, T. (1997). The EM algorithm and extensions. New York: Wiley.
12. Lange, K. (1995). A quasi Newton acceleration of the EM algorithm. Statistica Sinica, 5, 1-18.
13. Lehn-Schiøler, T., Hansen, L. K., & Larsen, J. (2005). Mapping from speech to images using continuous state space models. In Lecture Notes in Computer Science (Vol. 3361, pp. 136-145). Berlin: Springer.
14. Meilijson, I. (1989). A fast improvement to the EM algorithm on its own terms. Journal of Royal Statistical Society, B, 51, 127-138.
15. Moulines, E., Cardoso, J., & Cassiat, E. (1997). Maximum likelihood for blind separation and deconvolution of noisy signals using mixture models. In D. S. Touretzky, M. C. Mozer, & M. E. Hasselmo (Eds.), Proc. ICASSP (Vol. 5, pp. 3617-3620). Cambridge, MA: MIT Press.
16. Neal, R. M., & Hinton, G. E. (1998). Learning in graphical models: A view of the EM algorithm that justifies incremental, sparse, and other variants. Dordrecht: Kluwer.
17. Nielsen, H. B. (2000). UCMINF - an algorithm for unconstrained nonlinear optimization. (Tech. Rep. IMM-Rep-2000-19). Lungby: Technical University of Denmark.
18. Olsson, R. K., & Hansen, L. K. (2004). Probabilistic blind deconvolution of non-stationary sources. In 12th European Signal Processing Conference (pp. 1697-1700). Vienna, Austria.
19. Olsson, R. K., & Hansen, L. K. (2005). A harmonic excitation state-space approach to blind separation of speech. In L. K. Saul, Y. Weiss, & L. Bottou (Eds.), Advances in neural information processing systems (Vol. 17, pp. 993-1000). Cambridge, MA: MIT Press.
20. Petersen, K. B., & Winther, O. (2005). The EM algorithm in independent component analysis. In International Conference on Acoustics, Speech, and Signal Processing. Piscataway, NJ: IEEE.
21. Redner, R. A., & Walker, H. F. (1984). Mixture densities, maximum likelihood and the EM algorithm. SIAM Review, 2(26), 195-239.
22. Roweis, S., & Ghahramani, Z. (1999). A unifying review of linear gaussian models. Neural Computation, 11, 305-345.
23. Salakhutdinov, R., & Roweis, S. (2003). Adaptive overrelaxed bound optimization methods. In International Conference on Machine Learning (pp. 664-671). New York: AAAI Press.
24. Salakhutdinov, R., Roweis, S. T., & Ghahramani, Z. (2003). Optimization with EM and expectation-conjugate-gradient. In International Conference on Machine Learning (Vol. 20, pp. 672-679). New York: AAAI Press.
25. Sandell, N. R., & Yared, K. I. (1978). Maximum likelihood identification of state space models for linear dynamic systems. (Tech. Rep. ESL-R-814). Cambridge, MA: Electronic Systems Laboratory, MIT.
26. Shumway, R., & Stoffer, D. (1982). An approach to time series smoothing and forecasting using the EM algorithm. J. Time Series Anal., 3(4), 253-264.
27. Xu, L., & Jordan, M. I. (1996). On convergence properties of the EM algorithm for gaussian mixtures. Neural Computation, (8), 129-151.