Optimal kullback-leibler aggregation via spectral theory of markov chains

IEEE Transactions on Automatic Control

Volumn 56, Issue 12, 2011, Pages 2793-2808

  Deng, Kun ^a   Mehta, Prashant G ^a   Meyn, Sean P ^a  

^a UNIVERSITY OF ILLINOIS AT URBANA CHAMPAIGN   (United States)

Author keywords

Kullback Leibler (K L) divergence rate; Markov chain; Model reduction; Spectral theory

Indexed keywords

EIGENVALUE PROBLEM; KULLBACK LEIBLER DIVERGENCE; KULLBACK-LEIBLER; M-ARY PARTITIONS; MARKOV CHAIN; MARKOV CHAIN MODELS; MARKOV MODEL; MODEL REDUCTION; MODEL REDUCTION PROBLEMS; RECURSIVE ALGORITHMS; SPECTRAL THEORY;

EIGENVALUES AND EIGENFUNCTIONS;

MARKOV PROCESSES;

EID: 83255176310     PISSN: 00189286     EISSN: None     Source Type: Journal    
DOI: 10.1109/TAC.2011.2141350     Document Type: Article

References (34)

1. R. G. Phillips and P. V. Kokotovic, "A singular perturbation approach to modeling and control of Markov chains," IEEE Trans. Autom. Control., vol. AC-26, no. 5, pp. 1087-1094, Oct. 1981.
2. G. G. Yin and Q. Zhang, Continuous-time Markov Chains and Applications: A Singular Perturbation Approach. New York: Springer- Verlag, 1998, vol. 37, Stochastic Modelling and Applied Probability.
3. S. P. Meyn and R. L. Tweedie,Markov Chains and Stochastic Stability, 2nd ed. Cambridge, U.K.: Cambridge Univ. Press, 2009.
4. A. D. Wentzell, "The asymptotic behavior of the largest eigenvalue of a second order elliptic differential operator with a small parameter multiplying the highest derivatives," Dokl. Akad. Nauk SSSR, vol. 202, pp. 19-22, 1972.
5. O. Junge and M. Dellnitz, "Almost invariant sets in Chua's circuit," Int. J. Bif. Chaos, vol. 7, no. 11, pp. 2475-85, 1997. (Pubitemid 127338654)
6. P. Deuflhard, W. Huisinga, A. Fischer, and C. Schütte, "Identification of almost invariant aggregates in reversible nearly uncoupled Markov chains," Linear Algebra Appl., vol. 315, no. 1-3, pp. 39-59, 2000.
7. W. Huisinga, S. P. Meyn, and C. Schütte, "Phase transitions and metastability in Markovian and molecular systems," Ann. Appl. Probab., vol. 14, no. 1, pp. 419-458, 2004. (Pubitemid 41449262)
8. T. Runolfsson and Y. Ma, "Model reduction of nonreversible Markov chains," in Proc. 46th IEEE Conf. Decision Control, New Orleans, LA, 2007, pp. 3739-3744. (Pubitemid 351277730)
9. M. Fiedler, "A property of eigenvectors of nonnegative symmetric matrices and its application to graph theory," Czechoslovak Math. J., vol. 25, no. 100, pp. 619-633, 1975.
10. J. Shi and J. Malik, "Normalized cuts and image segmentation," IEEE Trans. Pattern Anal. Mach. Intell., vol. 22, no. 8, pp. 888-905, Aug. 2000.
11. B. Hendrickson and R. Leland, "Multidimensional spectral load balancing," Sandia National Laboratories, Albuquerque, NM, 1993, Tech. Rep., SAND93-0074.
12. S. Mahadevan and M. Maggioni, "Value function approximation using diffusion wavelets and laplacian eigenfunctions," in Advances in Neural Information Processing Systems. Cambridge, MA: MIT Press, 2006.
13. L. White, R. Mahony, and G. Brushe, "Lumpable Hidden Markov Models - Model reduction and reduced complexity filtering," IEEE Trans. Automat. Contr., vol. 45, no. 12, pp. 2297-2306, Dec. 2000. (Pubitemid 32190991)
14. S. Dey, "Reduced-complexity filtering for partially observed nearly completely decomposable Markov chains," IEEE Trans. Signal Process., vol. 48, no. 12, pp. 3334-3344, Dec. 2000. (Pubitemid 32085761)
15. P. J. Courtois, Decomposability: Queueing and Computer System Applications. New York, NY: Academic, 1977.
16. P. G. Mehta, M. Dorobantu, and A. Banaszuk, "Graph-based multiscale analysis of building system transport models," in Proc. Amer. Controls Conf., Minneapolis, MN, 2006, pp. 1110-1115. (Pubitemid 46530963)
17. U. Luxburg, "A tutorial on spectral clustering," Statist. Comput., vol. 17, no. 4, pp. 395-416, 2007.
18. T. M. Cover and J. A. Thomas, Elements of Information Theory, 1st ed. New York: Wiley, 1991.
19. Z. Rached, F. Alalaji, and L. L. Campbell, "The Kullback-Leibler divergence rate between Markov sources," IEEE Trans. Inf. Theory, vol. 50, no. 5, pp. 917-921, May 2004.
20. M. Donsker and S. Varadhan, "Asymptotic evaluation of certain Markov process expectations for large time. I. II," Comm. Pure Appl. Math., vol. 28, pp. 1-47, 1975.
21. P. J. Schweitzer, "A survey of aggregation-disaggregation in large Markov chains," in Numerical Solution of Markov Chains, W. J. Stewart, Ed. New York: Marcel Dekker, 1991, pp. 63-88.
22. W. E. T. Li and E.Vanden-Eijnden, "Optimal partition and effective dynamics of complex networks," PNAS, vol. 105, no. 23, pp. 7907-7912, 2008. (Pubitemid 351904680)
23. K. Deng, Y. Sun, P. G. Mehta, and S. P. Meyn, "An information-theoretic framework to aggregate a Markov chain," in Proc. Amer. Control Conf., St. Louis, MO, 2009, pp. 731-736.
24. M. Sarich, C. Schütte, and E. Vanden-Eijnden, "Optimal fuzzy aggregation of networks," Multiscale Modeling Simul., vol. 8, no. 4, pp. 1535-1561, 2010.
25. Y. Sun and P. G. Mehta, "The Kullback-Leiber rate pseudo-metric for comparing dynamical systems," IEEE Trans. Autom. Control, vol. 55, no. 7, pp. 1585-1598, Jul. 2010.
26. M. Vidyasagar, S. S. Mande, C. V. S. K. Reddy, and V. V. R. Rao, "The 4M (Mixed Memory Markov Model) algorithm for finding genes in prokaryotic genomes," IEEE Trans. Autom. Control, vol. 53, no. 1, pp. 26-37, Jan. 2008. (Pubitemid 351314537)
27. I. Kontoyiannis and S. P. Meyn, "Large deviations asymptotics and the spectral theory of multiplicatively regular Markov processes," Electron. J. Probab., vol. 10, no. 3, pp. 61-123 (electronic), 2005. (Pubitemid 40447601)
28. J. R. Kemeny and R. J. Snell, Finite Markov Chains. New York: Van Nostrand, 1960.
29. C. L. Beck, S. Lall, T. Liang, and M. West, "Model reduction, optimal prediction, and the Mori-Zwanzig representation of Markov chains," in Proc. IEEE Conf. Decision Control, Shanghai, China, 2009, pp. 3282-3287.
30. J. A. Fill, "Eigenvalue bounds on convergence to stationarity for nonreversible Markov chains, with an application to the exclusion process," Ann. Appl. Probab., vol. 1, no. 1, pp. 62-87, 1991.
31. W. Huisinga and B. Schmidt, "Metastability and dominant eigenvalues of transfer operators," in New Algorithms for Macromolecular Simulation. Berlin, Heidelberg, DEU: Springer, 2006, vol. 49, Lecture Notes in Computational Science and Engineering, pp. 167-182.
32. P. Deuflhard and M.Weber, "Robust Perron cluster analysis in conformation dynamics," Linear Algebra its Applicat., vol. 398, pp. 161-184, 2005. (Pubitemid 40220042)
33. M. Fazel, "Matrix rank minimization with applications," Ph.D. dissertation, Stanford Univ., Stanford, CA, 2002.
34. M. Meila and L. Xu, "Multiway cuts and spectral clustering," Tech. Rep. Univ. of Washington, Seattle, 2003.