A metric between probability distributions on finite sets of different cardinalities and applications to order reduction

IEEE Transactions on Automatic Control

Volumn 57, Issue 10, 2012, Pages 2464-2477

  Vidyasagar, Mathukumalli ^a  

^a The University of Texas at Dallas   (United States)

Author keywords

Computational complexity; entropy; greedy algorithms; mutual information; probability distributions

Indexed keywords

CARDINALITIES; FINITE SET; GREEDY ALGORITHMS; JOINT DISTRIBUTIONS; MARGINALS; MAXIMUM ENTROPY; METRIC DISTANCES; MINIMUM ENTROPY; MUTUAL INFORMATIONS; NP-HARD; ORDER REDUCTION; REDUCED-ORDER APPROXIMATION; UPPER BOUND;

COMPUTATIONAL COMPLEXITY; ENTROPY; MARKOV PROCESSES; OPTIMIZATION; PROBABILITY DISTRIBUTIONS; SET THEORY;

APPROXIMATION ALGORITHMS;

EID: 84866925305     PISSN: 00189286     EISSN: None     Source Type: Journal    
DOI: 10.1109/TAC.2012.2188423     Document Type: Article

References (19)

1. R. Cilibrasi and P. M. B. Vitányi, "Clustering by comparison," IEEE Trans. Inf. Thy., vol. 51, no. 4, pp. 1523-1545, Apr. 2005.
2. E. G. Coffman, Jr. and J. Csirik, Performance guarantees for one-dimensional bin packing, Ch. 32 in [11].
3. E. G. Coffman, Jr., J. Csirik, and J. Y.-T. Leung, Variants of classical one-dimensional bin packing Ch. 33 in [11].
4. E. G. Coffman, Jr., J. Csirik, and J. Y.-T. Leung, Variable-sized bin packing and bin covering Ch. 34 in [11].
5. E. G. Coffman, Jr. and G. S. Lueker, "Approximation algorithms for extensible b packing," in Proc. SODA, Jan. 2001, pp. 586-588.
6. T. M. Cover and J. A. Thomas, Elements of Information Theory, 2nd ed. New York: Wiley, 2006.
7. P. Dell'Olmo, H. Kellerer, M. G. Speranza, and Z. Tuza, "A 13/12 approximation algorithm for b packing with extendable bins," Inf. Process. Lett., vol. 65, pp. 229-233, 1998. (Pubitemid 128393571)
8. K. Deng, P. G. Mehta, and S. P. Meyn, "Optimal Kullback-Leibler aggregation via the spectral theory of Markov chains," in Proc. Amer. Control Conf., St. Louis, MO, 2009, pp. 731-736.
9. K. Deng, P. G. Mehta, and S. P. Meyn, "A simulation-based method for aggregatingMarkov chains," in Proc. IEEE Conf. Decision Control, Shanghai, China, 2009, pp. 4710-4716.
10. K. Deng, P. G. Mehta, and S. P. Meyn, "Optimal Kullback-Leibler aggregation via the spectral theory of Markov chains," IEEE Trans. Autom. Control, vol. 56, no. 12, pp. 2793-2808, Dec. 2011.
11. T. González, Ed., Handbook of Approximation Algorithms and Metaheuristics. London, U.K.: Chapman & Hall, CRC, 2007.
12. M. Li, X. Chen, X. Li, B. Ma, and P. M. B. Vitányi, "The similarity metric," IEEE Trans. Inf. Theory, vol. 50, no. 12, pp. 3250-3264, Dec. 2004.
13. M. Meila, "Comparing clusterings by the variation of information," in Proc. Learning Theory and Kernel Machines: 16th Annu. Conf. Learn. 7th Kernel Workshop, B. Schökopf, M.Warmuth, and M. K.Warmuth, Eds., 2003, pp. 173-187. (Pubitemid 37053203)
14. M. Meila, "Comparing clusterings-An information-based distance," J. Multivariate Anal., vol. 98, no. 5, pp. 873-895, 2007.
15. D. S. Ornstein, "An application of ergodic theory to probability theory," Ann. Probab., vol. 1, no. 1, pp. 43-65, 1973.
16. J. C. Spall and S. D. Hill, "Least-informative Bayesian prior distributions for finite samples based on information theory," IEEE Trans. Autom. Control, vol. 35, no. 5, pp. 580-583, May 1990.
17. M. Vidyasagar, "Kullback-Leibler divergence rate between probability distributions on sets of different cardinalities," in Proc. IEEE Conf. Decision Control, Atlanta, GA, 2010, pp. 947-953.
18. D. Yu and G. Zhang, "On-line extensible b packing with unequal b sizes," Lecture Notes Comput. Sci., vol. 2909, pp. 235-247, 2004.
19. M. Yue, "A simple proof of the inequality for the FFD bin-packing algorithm," Acta Mathematicae Applicatae Sinica, vol. 7, no. 4, pp. 321-331, Oct. 1991.