Mini-buckets: A general scheme for bounded inference

Journal of the ACM

Volumn 50, Issue 2, 2003, Pages 107-153

  Dechter, Rina ^a   Rish, Irina ^b  

^a UNIVERSITY OF CALIFORNIA   (United States)

^b IBM T J WATSON RESEARCH CENTER   (United States)

Author keywords

Accuracy complexity trade off; Approximation algorithms; Bayesian networks; Combinatorial optimization; Probabilistic inference

Indexed keywords

ACCURACY/COMPLEXITY TRADEOFF; APPROXIMATION ALGORITHM; BAYESIAN NETWORKS; COMBINATORIAL OPTIMIZATION;

ALGORITHMS; APPROXIMATION THEORY; DIAGNOSIS; OPTIMIZATION; PROBABILISTIC LOGICS; PROBLEM SOLVING;

COMPUTATIONAL COMPLEXITY;

EID: 1642312636     PISSN: 00045411     EISSN: None     Source Type: Journal    
DOI: 10.1145/636865.636866     Document Type: Article

References (86)

1. ARNBORG, S. A. 1985. Efficient algorithms for combinatorial problems on graphs with bounded decomposability-A survey. BIT 25, 2-23.
2. BERROU, G., GLAVIEUX, A., AND THITIMAJSHIMA, P. 1993. Near Shannon limit error-correcting coding: Turbo codes. In Proceedings of the 1993 International Conference on Communications (Geneva, May).
3. BERTELE, U., AND BRIOSCHI, F. 1972. Nonserial Dynamic Programming, Academic Press, New York.
4. BODDY, M., AND DEAN, T. L. 1989. Solving time-dependent planning problems. In Proceedings of the 11th International Joint Conference on Artificial Intelligence, pp. 979-984.
5. BODLAENDER, H. L. 1997. Treewidth: Algorithmic techniques and results. In Proceedings of 22nd International Symposium on Mathematical Foundations of Computer Science (MFCS'97). Lecture Notes in Computer Science, vol. 1295. Springer-Verlag, New York, pp. 19-36.
6. BODLAENDER, H. L., KOSTER, A. M., EIJKHOF, F. V., AND VAN DER GAAG, L. C. 2001. Pre-processing for triangulation of probabilistic networks. In Proceedings of 17th Annual Conference of Uncertainty in Artificial Intelligence (UAI01). pp. 32-39.
7. BOYEN, X., AND KOLLER, D. 1998. Tractable inference for complex stochastic processes. In Proceedings of 14th annual conference on Uncertainty in AI (UAI) (Madison, Wise), pp. 33-42.
8. CANNINGS, C., THOMPSON, E. A., AND SKOLNICK, H. H. 1978. Probability functions on complex pedigrees. Adv. Appl. Prob. 10, 26-61.
9. CHENG, J.-F. 1997. Iterative decoding. Ph.D. dissertation. Caltech, Pasadena, Calif.
10. COOPER, G. F., 1990. The computational complexity of probabilistic inference using Bayesian belief networks. Artif. Intel. 42, 2-3, 393-405.
11. DAGUM, P., AND LUBY, M. 1993. Approximating probabilistic inference in Bayesian belief networks is NP-hard. Artif. Intel 60, 1, 141-155.
12. D'AMBROSIO, B. 1994. Symbolic probabilistic inference in large BN2O networks. In Proceedings of the 10th Conference on Uncertainty in Artificial Intelligence, pp. 128-135.
13. DAVIS, M., AND PUTNAM, H. 1960. A computing procedure for quantification theory. J. ACM 7, 3.
14. DEAN, T. L., AND BODDY, M. 1988. An analysis of time-dependent planning. In Proceedings of the 7th National Conference on Artificial Intelligence. PP. 49-54.
15. DECHTER, R. 1992. Constraint networks. In Encyclopedia of Artificial Intelligence, 2nd ed. Wiley, New York, pp. 276-285.
16. DECHTER, R. 1996. Bucket elimination: A unifying framework for probabilistic inference. In Proceedings of 12th Conference on Uncertainty in Artificial Intelligence (UAI-96). pp. 211-219.
17. DECHTER, R. 1997a. Bucket elimination: A unifying framework for processing hard and soft constraints. CONSTRAINTS: An Int. J. 2, 51-55.
18. DECHTER, R. 1997b. Mini-buckets: A general scheme for generating approximations in automated reasoning. In Proceedings of the 15th International Joint Conference of Artificial Intelligence (IJCAI-97) (Japan). pp. 1297-1302.
19. DECHTER, R. 1998. Constraint satisfaction. In MIT Encyclopedia of the Cognitive Sciences (MITECS). MIT Cambridge, Mass.
20. DECHTER, R. 1999. Bucket elimination: A unifying framework for reasoning. Artif. Intel. 113, 41-85.
21. DECHTER, R., AND FROST, D. 1997. Backtracking algorithms for constraint satisfaction problems, a tutorial survey. Tech. rep., UCI.
22. DECHTER, R., KASK, K., AND MATEESCU, R. 2002. Iterative join-graph propagation. In Proceedings of UAI-02.
23. DECHTER, R., AND PEARL, J. 1987. Network-based heuristics for constraint satisfaction problems. Artif. Intel. 34, 1-38.
24. DECHTER, R., AND RISK, I. 1994. Directional resolution: The Davis-Putnam procedure, revisited. In Principles of Knowledge Representation and Reasoning (KR-94). pp. 134-145.
25. DECHTER, R., AND RISH, I. 1997. A scheme for approximating probabilistic inference. In Proceedings of the 13th Conference on Uncertainty in Artificial Intelligence (UAI97).
26. DECHTER, R., AND VAN BEEK, P. 1997. Local and global relational consistency. Theoret. Comput. Sci. pp. 283-308.
27. DRAPER, D. 1995. Localized partial evaluation of belief networks. Tech. rep. Ph.D. dissertation. University of Washington.
28. EL FATTAH, Y., AND DECHTER, R. 1995. Diagnosing tree-decomposable circuits. In Proceedings of the International Joint Conference of Artificial Intelligence (IJCAI-95) (Montreal, Ont., Canada, Aug.). pp. 1742-1748.
29. JENSEN, F., AND ANDERSEN, S. K., 1990. Approximations in Bayesian belief universes for knowledge-based systems. In Proceedings of the 6th Conference on Uncertainty in Artificial Intelligence.
30. FREUDER, E. C. 1978. Synthesizing constraint expressions. Commun. ACM 21, 11, 958-965.
31. FREUDER, E. C. 1982. A sufficient condition for backtrack-free search. J. ACM 21, 11, 958-965.
32. FREY, B. J. 1998. Graphical Models for Machine Learning and Digital Communication. MIT Press, Cambridge, Mass.
33. FREY, B. J. 1998. Graphical Models for Machine Learning and Digital Communication. MIT Press, Cambridge, Mass.
34. FREY, B. J., AND MACKAY, D. J. C. 1998. A revolution: belief propagation in graphs with cycles. Adv. Neural Inf. Proc. Syst. 10.
35. GALLAGER, R. G. 1965. A simple derivation of the coding theorem and some applications. IEEE Trans. Inf. Theory IT-11, 3-18.
36. GUO, H., AND HSU, W. 2002. A survey on algorithms for real-time Bayesian network inference. In Proceedings of the Joint AAAI-02/KDD-02/UAI-02 Workshop on Real-Time Decision Support and Diagnosis Systems (Edmonton, Alberta, Canada).
37. HECKERMAN, D. AND BREESE, J. 1995. Causal independence for probability assessment and inference using Bayesian networks. Tech. Rep. MSR-TR-94-08, Microsoft Research.
38. HORVITZ, E. 1987. Reasoning about beliefs and actions under computational resource constraints. In Proceedings of the 3rd Conference on Uncertainty in Artificial Intelligence (UAI-87) (Seattle, Wash.). pp. 301-324.
39. HORVITZ, E. 1988. Reasoning under varying and uncertain resource constraints. In Proceedings of the 4th Conference on Uncertainty in Artificial Intelligence (UAI-88). pp. 111-116.
40. HORVTTZ, E. 1990. Computation and action under bounded resources. Ph.D. dissertation. Stanford Univ., Stanford, Calif.
41. HORVITZ, E., SUERMONDT, H. J., AND COOPER, G. F. 1989. Bounded conditioning: Flexible inference for decisions under scarce resources. In Proceedings of the 5th Conference on Uncertainty in Artificial Intelligence (UAI-89). pp. 182-193.
42. KASK, K., RISH, I., AND DECHTER, R. 1998. Approximation algorithms for probabilistic decoding. In Proceedings of the 14th Conference on Uncertainty in Artificial Intelligence (UAI-98).
43. JAAKKOLA, T. S., AND JORDAN, M. I. 1996. Recursive algorithms for approximating probabilities in ' graphical models. Adv. Neural Inf. Proc. Syst. 9.
44. JORDAN, M. I., GHAHRAMANI, Z., JAAKKOLA, T. S., AND SAUL, L. K. 1998. An Introduction to Variational Methods for Graphical Models. Kluwer Academic Publishers.
45. KASK, K. 2000. New search heuristics for max-csp. In Proceedings of the Principles and Practice of Constraint Programming (CP2000). pp. 255-269.
46. KASK, K., AND DECHTER, R. 1999a. Mini-bucket heuristics for improved search. In Proceedings of 15th Conference on Uncertainty in Artificial Intelligence (UAI-99).
47. KASK, K., AND DECHTER, R. 1999b. Stochastic local search for bayesian networks. In Proceedings of Workshop on AI and Statistics (AISTAT'99). pp. 113-122.
48. KASK, K., AND DECHTER, R. 2001. A general scheme for automatic search heuristics from specification dependencies. Artif Intel.
49. KASK, K., DECHTER, R., LARROSA, J., AND FABIO, G. 2001. Bucket-tree elimination for automated reasoning. Artif. Intel. 125, 91-131.
50. KJAERULEF, U. 1990. Triangulation of graph-based algorithms giving small total state space. Tech. Rep. 90-09, Department of Mathematics and Computer Science, Univ. Aalborg, Aalborg, Denmark.
51. KJAERULFF, U. 1992. Optimal decomposition of probabilistic networks by simulated annealing. Stat. Comput. 2, 7-17.
52. KJAERULFF, U. 1994. Reduction of computational complexity in Bayesian networks through removal of week dependencies. In Proceedings of the 10th Conference on Uncertainty in Artificial Intelligence (UAI-94).
53. LARROSA, J. 2000. On the time complexity of bucket elimination algorithms. ICS Tech. rep.
54. LASSEZ, J.-L., AND MAHLER, M. 1992. On Fourier's algorithm for linear constraints. J. Automat. Reas. 9.
55. LAURITZEN, S. L., AND SPIEGELHALTER, D. J. 1988. Local computation with probabilities on graphical structures and their application to expert systems. J. Roy. Stat. Soc., Seri. B 50, 2, 157-224.
56. MACKAY, D. J. C. 1998. Introduction to Monte Carlo methods. In Learning in Graphical Models, M. I. Jordan, ed. Kluwer Academic Press.
57. MACKAY, D. J. C., AND NEAL, R. M. 1996. Near Shannon limit performance of low density parity check codes. Electron. Lett. 33, 457-458.
58. MACKWORTH, A. K. 1977. Consistency in networks of relations. Artif. Intel. 8,1, 99-118.
59. MATEESCU, R., DECHTER, R., AND KASK, K. 2002. Tree approximation for belief updating. In Proceedings of the 18th National Conference on Artificial Intelligence (AAAI-2002) (Edmonton, Alberta, Canada), pp. 553-559.
60. MCELIECE, R. J., MACKAY, D. J. C., AND CHENG, J. F. 1997. Turbo decoding as an instance of Pearl's belief propagation algorithm. IEEE J. Select. Areas Commun.
61. MILLER, R. A., MASARIE, F. E., AND MYERS, J. 1986. Quick medical reference (QMR) for diagnostic assistance. Medi. Comput. 3, 5, 34-38.
62. MILLER, R. A., POPLE, H. E., AND MYERS, J. D. 1982. Internist-1: An experimental computer-based diagnostic consultant for general internal medicine. New Eng. J. Med. 307, 468-476.
63. MONTANARI, U. 1972. On the optimal approximation of discrete functions with low-dimensional tables. Inf. Proc. 71, 1363-1368.
64. PARK, J. 2002. MAP complexity results and approximation methods. In Proceedings of the 18th Conference on Uncertainty in Artificial Intelligence (UAI) (San Francisco, Calif.) Morgan-Kaufmann, San Mateo, Calif., pp. 388-396.
65. PARKER, R., AND MILLER, R. 1987. Using causal knowledge to create simulated patient cases: the CPCS project as an extension of INTERNIST-1. In Proceedings of the 11th Symposium Computer Applications in Medical Care, pp. 473-480.
66. PEARL, J. 1988. Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan-Kaufmann, San Mateo, Calif.
67. PENG, Y., AND REGGIA, J. A. 1989. A connectionist model for diagnostic problem solving. IEEE Trans. Syst., Man and Cybernet.
68. POOLE, D. 1996. Probabilistic conflicts in a search algorithm for estimating posterior probabilities in Bayesian networks. Artif. Intel. 88, 69-100.
69. PRADHAN, M., PROVAN, G., MIDDLETON, B., AND HENRION, M. 1994. Knowledge engineering for large belief networks. In Proceedings of the 10th Conference on Uncertainty in Artificial Intelligence.
70. RISH, I., BRODIE, M., AND MA, S. 2002. Accuracy vs. efficiency trade-offs in probabilistic diagnosis. In Proceedings of the 18th National Conference on Artificial Intelligence (AAAI2002) (Edmonton, Alb., Canada).
71. RISH, I., AND DECHTER, R. 1998, On the impact of causal independence. Tech. rep., Information and Computer Science, Univ. California, Irvine, Irvine, Calif.
72. RISH, I., AND DECHTER, R. 2000. Resolution vs. search; Two strategies for SAT. J. Automat. Reas. 24, 1/2, 225-275.
73. ROBERTSON, N., AND SEYMOUR, P. 1995. Graph minor. XIII, The disjoint paths problem. Combinat. Theory, Ser. B 63, 65-110.
74. ROTH, D. 1996. On the hardness of approximate reasoning. AI J. 82, 1-2 (Apr.), 273-302.
75. SANTOS, E. 1991. On the generation of alternative explanations with implications for belief revision. In Proceedings of the 7th Annual Conference on Uncertainty in Artificial Intelligence (UAI-91). pp. 339-347.
76. SANTOS, E. J., AND SHIMONY, S. E. 1998. Deterministic approximation of marginal probabilities in Bayes nets. IEEE Trans. Syst. Man, Cybernet. 28, 4, 377-393.
77. SHACHTER, R. D. 1986. Evaluating influence diagrams. Oper. Res. 34.
78. SHANNON, C. E. 1948. A mathematical theory of communication. Bell Syst. Tech. J. 27, 379-423, 623-656.
79. SHIMONY, S. E., AND CHARNIACK, E. 1991. A new algorithm for finding MAP assignments to belief networks. In Uncertainty in Artificial Intelligence, P. Bonissone, M. Henrion, L. Kanal, and J. Lemmer, Eds. pp. 185-193.
80. SHWE, M., MIDDLETON, B. F., HECKERMAN, D. E., HENRION, M., HORVITZ, E. J., LEHMANN, H., AND COOPER, G. F. 1991. Probabilistic diagnosis using a reformulation of the Internist-1/QMR knowledge base: I. The probabilistic model and inference algorithms. Meth. Inf. Med. 30, 241-255.
81. TATMAN, J. A., AND SHACHTER, R. D. 1990. Dynamic programming and influence diagrams. IEEE Trans. Syst. Man, Cyberneti.
82. VAN ENGELEN, R. A. 1997. Approximating Bayesian belief networks by arc removal. IEEE Trans. Patt. Anal. Mach. Intel. 19, 8, 916-920.
83. WEISS, Y. 1997. Belief propagation and revision in networks with loops. In Proceedings of the NIPS-97 Workshop on Graphical Models.
84. WELLMAN, M. P., AND LIU, C. 1994. State-space abstraction for anytime evaluation of probabilistic networks. In Proceedings of the 10th Annual Conference on Uncertainty in Artificial Intelligence (UAI-94). pp. 567-574.
85. YEDIDIA, J., FREEMAN, W. T., AND WEISS, Y. 2001. Generalized belief propagation. In NIPS 13. MIT Press, Cambridge, Mass, pp. 689-695.
86. ZHANG, N. L., AND POOLE, D. 1996. Exploiting causal independence in Bayesian network inference. J. Artif. Intel. Res. 5, 301-328.