Variational learning for switching state-space models

Neural Computation

Volumn 12, Issue 4, 2000, Pages 831-864

  Ghahramani, Zoubin ^a   Hinton, Geoffrey E ^a  

^a UNIVERSITY COLLEGE LONDON   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHM; ARTICLE; ARTIFICIAL INTELLIGENCE; ARTIFICIAL NEURAL NETWORK; BREATHING MECHANICS; COMPUTER SIMULATION; HUMAN; PATHOPHYSIOLOGY; PHYSIOLOGY; PROBABILITY; SLEEP APNEA SYNDROME; STATISTICAL MODEL; STATISTICS;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; COMPUTER SIMULATION; HUMANS; LINEAR MODELS; MARKOV CHAINS; MODELS, ECONOMETRIC; MODELS, STATISTICAL; NEURAL NETWORKS (COMPUTER); RESPIRATORY MECHANICS; SLEEP APNEA SYNDROMES; STOCHASTIC PROCESSES;

EID: 0034170950     PISSN: 08997667     EISSN: None     Source Type: Journal    
DOI: 10.1162/089976600300015619     Document Type: Article

References (54)

1. Ackerson, G. A., & Fu, K. S. (1970). On state estimation in switching environments. IEEE Transactions on Automatic Control, AC-15(1):10-17.
2. Anderson, B. D. O., & Moore, J. B. (1979). Optimal filtering. Englewood Cliffs, NJ: Prentice-Hall.
3. Athaide, C. R. (1995). Likelihood evaluation and state estimation for nonlinear state space models. Unpublished doctoral dissertation, University of Pennsylvania.
4. Baldi, P., Chauvin, Y., Hunkapiller, T., & McClure, M. (1994). Hidden Markov models of biological primary sequence information. Proc. Nat. Acad. Sci. (USA), 91(3), 1059-1063.
5. Bar-Shalom, Y., & Li, X.-R. (1993). Estimation and tracking. Boston: Artech House.
6. Baum, L., Petrie, T., Soules, G., & Weiss, N. (1970). A maximization technique occurring in the statistical analysis of probabilistic functions of Markov chains. Annals of Mathematical Statistics, 41, 164-171.
7. Bengio, Y., & Frasconi, P. (1995). An input-output HMM architecture. In G. Tesauro, D. S. Touretzky, & T. K. Leen (Eds.), Advances in neural information processing systems, 7 (pp. 427-434). Cambridge, MA: MIT Press.
8. Cacciatore, T. W., & Nowlan, S. J. (1994). Mixtures of controllers for jump linear and non-linear plants. In J. D. Cowan, G. Tesauro, & J. Alspector (Eds.), Advances in neural information processing systems, 6 (pp. 719-726). San Mateo: Morgan Kaufmann Publishers.
9. Carter, C. K., & Kohn, R. (1994). On Gibbs sampling for state space models. Biometrika, 81, 541-553.
10. Chaer, W. S., Bishop, R. H., & Ghosh, J. (1997). A mixture-of-experts framework for adaptive Kalman filtering. IEEE Trans. on Systems, Man and Cybernetics.
11. Chang, C. B., & Athans, M. (1978). State estimation for discrete systems with switching parameters. IEEE Transactions on Aerospace and Electronic Systems, AES-14(3), 418-424.
12. Cover, T., & Thomas, J. (1991). Elements of information theory. New York: Wiley.
13. Dean, T., & Kanazawa, K. (1989). A model for reasoning about persistence and causation. Computational Intelligence, 5(3), 142-150.
14. Dempster, A., Laird, N., & Rubin, D. (1977). Maximum likelihood from incomplete data via the EM algorithm. J. Royal Statistical Society Series B, 39, 1-38.
15. Deng, L. (1993). A stochastic model of speech incorporating hierarchical nonstationarity. IEEE Trans. on Speech and Audio Processing, 1(4), 471-474.
16. Digalakis, V., Rohlicek, J. R., & Ostendorf, M. (1993). ML estimation of a stochastic linear system with the EM algorithm and its application to speech recognition. IEEE Transactions on Speech and Audio Processing, 1(4), 431-442.
17. Elliott, R. J., Aggoun, L., & Moore, J. B. (1995). Hidden Markov models: Estimation and control. New York: Springer-Verlag.
18. Fraser, A. M., & Dimitriadis, A. (1993). Forecasting probability densities by using hidden Markov models with mixed states. In A. S. Wiegand & N. A. Gershenfeld (Eds.), Time series prediction: Forecasting the future and understanding the past (pp. 265-282). Reading, MA: Addison-Wesley.
19. Ghahramani, Z. (1997). On structured variational approximations (Tech. Rep. No. CRG-TR-97-1). Toronto: Department of Computer Science, University of Toronto. Available online at: http://www.gatsby.ucl.ac.uk/zoubin/ papers/struct.ps.gz.
20. Ghahramani, Z., & Hinton, G. E. (1996a). Parameter estimation for linear dynamical systems (Tech. Rep. No. CRG-TR-96-2). Toronto: Department of Computer Science, University of Toronto. Available online at: http://www.gatsby. ucl.ac.uk/zoubin/papers/tr-96-2.ps.gz.
21. Ghahramani, Z., & Hinton, G. E. (1996b). The EM algorithm for mixtures of factor analyzers (Tech. Rep. No. CRG-TR-96-1). Toronto: Department of Computer Science, University of Toronto. Available online at: http://www. gatsby.ucl.ac.uk/zoubin/papers/tr-96-1.ps.gz.
22. Ghahramani, Z., & Jordan, M. I. (1997). Factorial hidden Markov models. Machine Learning, 29, 245-273.
23. Goodwin, G., & Sin, K. (1984). Adaptive filtering prediction and control. Englewood Cliffs, NJ: Prentice-Hall.
24. Hamilton, J. D. (1989). A new approach to the economic analysis of nonstationary time series and the business cycle. Econometrica, 57, 357-384.
25. Hamilton, J. D. (1994). Time series analysis. Princeton, NJ: Princeton University Press.
26. Harrison, P. J., & Stevens, C. F. (1976). Bayesian forecasting (with discussion). J. Royal Statistical Society B, 38, 205-247.
27. Hinton, G. E., Dayan, P., & Revow, M. (1997). Modeling the manifolds of images of handwritten digits. IEEE Transactions on Neural Networks, 8, 65-74.
28. Jacobs, R. A., Jordan, M. I., Nowlan, S. J., & Hinton, G. E. (1991). Adaptive mixture of local experts. Neural Computation, 3, 79-87.
29. Jordan, M. I., Ghahramani, Z., Jaakkola, T. S., & Saul, L. K. (1998). An introduction to variational methods in graphical models. In M. I. Jordan (Ed.), Learning in graphical models. Norwell, MA: Kluwer.
30. Jordan, M. I., & Jacobs, R. (1994). Hierarchical mixtures of experts and the EM algorithm. Neural Computation, 6, 181-214.
31. Juang, B. H., & Rabiner, L. R. (1991). Hidden Markov models for speech recognition. Technometrics, 33, 251-272.
32. Kadirkamanathan, V., & Kadirkamanathan, M. (1996). Recursive estimation of dynamic modular RBF networks. In D. Touretzky, M. Mozer, and M. Hasselmo (Eds.), Advances in neural information processing systems, 8 (pp. 239-245). Cambridge, MA: MIT Press.
33. Kalman, R. E., & Bucy, R. S. (1961). New results in linear filtering and prediction. Journal of Basic Engineering (ASME), 83D, 95-108.
34. Kanazawa, K., Koller, D., & Russell, S. J. (1995). Stochastic simulation algorithms for dynamic probabilistic networks. In P. Besnard & S. Hanks (Eds.), Uncertainty in artificial intelligence. Proceedings of the Eleventh Conference (pp. 346-351). San Mateo, CA: Morgan Kaufmann.
35. Kehagias, A., & Petrides, V. (1997). Time series segmentation using predictive modular neural networks. Neural Computation, 9(8), 1691-1710.
36. Kim, C.-J. (1994). Dynamic linear models with Markov-switching. J. Econometrics, 60, 1-22.
37. Lauritzen, S. L., & Spiegelhalter, D. J. (1988). Local computations with probabil-ities on graphical structures and their application to expert systems. J. Royal Statistical Society B, 157-224.
38. Ljung, L., & Söderström, T. (1983). Theory and practice of recursive identification. Cambridge, MA: MIT Press.
39. Meila, M., & Jordan, M. I. (1996). Learning fine motion by Markov mixtures of experts. In D. S. Touretzky, M. C. Mozer, & M. E. Hasselmo (Eds.), Advances in neural information processing systems, 8. Cambridge, MA: MIT Press.
40. Neal, R. M., & Hinton, G. E. (1998). A new view of the EM algorithm that justifies incremental, sparse, and other variants. In M. I. Jordan (Ed.), Learning in graphical models. Norwell, MA: Kluwer.
41. Parisi, G. (1988). Statistical field theory. Redwood City, CA: Addison-Wesley.
42. Pawelzik, K., Kohlmorgen, J., & Müller, K.-R. (1996). Annealed competition of experts for a segmentation and classification of switching dynamics. Neural Computation, 8(2), 340-356.
43. Pearl, J. (1988). Probabilistic reasoning in intelligent systems: Networks of plausible inference. San Mateo, CA: Morgan Kaufmann.
44. Rabiner, L. R., & Juang, B. H. (1986). An introduction to hidden Markov models. IEEE Acoustics, Speech & Signal Processing Magazine, 3, 4-16.
45. Rauch, H. E. (1963). Solutions to the linear smoothing problem. IEEE Transactions on Automatic Control, 8, 371-372.
46. Rigney, D., Goldberger, A., Ocasio, W., Ichimaru, Y., Moody, G., & Mark, R. (1993). Multi-channel physiological data: Description and analysis. In A. Weigend & N. Gershenfeld (Eds.), Time series prediction: Forecasting the future and understanding the past (pp. 105-129). Reading, MA: Addison-Wesley.
47. Roweis, S., & Ghahramani, Z. (1999). A unifying review of linear gaussian models. Neural Computation, 11(2), 305-345.
48. Saul, L. and Jordan, M. I. (1996). Exploiting tractable substructures in intractable networks. In D. Touretzky, M. Mozer, & M. Hasselmo (Eds.), Advances in neural information processing systems, 8. Cambridge, MA: MIT Press.
49. Shumway, R. H., & Stoffer, D. S. (1982). An approach to time series smoothing and forecasting using the EM algorithm. J. Time Series Analysis, 3(4), 253-264.
50. Shumway, R. H., & Stoffer, D. S. (1991). Dynamic linear models with switching. J. Amer. Stat. Assoc., 86, 763-769.
51. Smyth, P. (1994). Hidden Markov models for fault detection in dynamic systems. Pattern Recognition, 27(1), 149-164.
52. Smyth, P., Heckerman, D., & Jordan, M. I. (1997). Probabilistic independence networks for hidden Markov probability models. Neural Computation, 9, 227-269.
53. Ueda, N., & Nakano, R. (1995). Deterministic annealing variant of the EM algorithm. In G. Tesauro, D. Touretzky, & J. Alspector (Eds.), Advances in neural information processing systems, 7 (pp. 545-552). San Mateo, CA: Morgan Kaufmann.
54. Weigend, A., & Gershenfeld, N. (1993). Time series prediction: Forecasting the future and understanding the past. Reading, MA: Addison-Wesley.