Belief propagation for continuous state spaces: Stochastic message-passing with quantitative guarantees

Journal of Machine Learning Research

Volumn 14, Issue , 2013, Pages 2799-2835

  Noorshams, Nima ^a   Wainwright, Martin J ^a  

^a UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

Graphical models; Low complexity belief propagation; Monte Carlo methods; Orthogonal basis expansion; Stochastic approximation; Sum product for continuous state spaces

Indexed keywords

CONTINUOUS STATE SPACE; GRAPHICAL MODEL; LOW-COMPLEXITY BELIEF; ORTHOGONAL BASIS; STOCHASTIC APPROXIMATIONS;

BACKPROPAGATION; FOURIER ANALYSIS; GRAPHIC METHODS; MONTE CARLO METHODS; STOCHASTIC MODELS; STOCHASTIC SYSTEMS; TREES (MATHEMATICS);

MESSAGE PASSING;

EID: 84885651092     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (33)

1. A. Agarwal, P. L. Bartlett, P. Ravikumar, andM. J. Wainwright. Information-theoretic lower bounds on the oracle complexity of stochastic convex optimization. IEEE Transactions on Information Theory, 58(5):3235-3249, May 2012.
2. S. M. Aji and R. J. McEliece. The generalized distributive law. IEEE Transactions on Information Theory, 46(2):325-343, March 2000.
3. M. S. Arulampalam, S. Maskell, N. Gordon, and T. Clapp. A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking. IEEE Transaction on Signal Processing, 50(2):174-188, 2002. (Pubitemid 34291500)
4. S. Baker, D. Scharstein, J. P. Lewis, S. Roth, M. Black, and R. Szeliski. A database and evaluation methodology for optical flow. International Journal of Computer Vision, 92(1):1-31, March 2011.
5. G. Boccignone, A. Marcelli, P. Napoletano, and M. Ferraro. Motion estimation via belief propagation. In Proceedings of the International Conference on Image Analysis and Processing, pages 55-60, 2007.
6. F. Chung and L. Lu. Concentration inequalities and martingale inequalities: A survey. Internet Mathematics, 3(1):79-127, 2006.
7. F. Clarke. Functional Analysis, Calculus of Variations and Optimal Control. Springer-Verlag, London, 2013.
8. J. Coughlan and H. Shen. Dynamic quantization for belief propagation in sparse spaces. Computer Vision and Image Understanding, 106(1):47-58, 2007. (Pubitemid 46528760)
9. A. Doucet, N. de Freitas, and N. Gordon. Sequential Monte Carlo Methods in Practice. Springer, New York, 2001.
10. R. Durrett. Probability: Theory and Examples. Duxbury Press, New York, NY, 1995.
11. M. Fabian, P. Habala, P. Hajek, V. Montesinos, and V. Zizler. Banach Space Theory: The Basis for Linear and Nonlinear Analysis. Springer, New York, 2011.
12. C. Gu. Smoothing Spline ANOVA Models. Springer Series in Statistics. Springer, New York, NY, 2002.
13. A. Ihler, J. Fisher, and A. S. Willsky. Loopy belief propagation: Convergence and effects of message errors. Journal of Machine Learning Research, 6:905-936, May 2005.
14. A. T. Ihler and D. McAllester. Particle belief propagation. In Proceedings of the Conference on Artificial Intelligence and Statistics, pages 256-263, 2009.
15. M. Isard. PAMPAS: Real-valued graphical models for computer vision. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, volume 1, pages 613-620, 2003.
16. M. Isard, J. MacCormick, and K. Achan. Continuously-adaptive discretization for message-passing algorithms. In Proceedings of the Advances in Neural Information Processing Systems, pages 737-744, 2009.
17. F. R. Kschischang, B. J. Frey, and H. A. Loeliger. Factor graphs and the sum-product algorithm. IEEE Transaction on Information Theory, 47(2):498-519, 2001. (Pubitemid 32318087)
18. H. A. Loeliger. An introduction to factor graphs. IEEE Signal Processing Magazine, 21(1):28-41, 2004.
19. D. G. Luenberger. Optimization by Vector Space Methods. Wiley, New York, 1969.
20. J. M. Mooij and H. J. Kappen. Sufficient conditions for convergence of the sum-product algorithm. IEEE Transactions on Information Theory, 53(12):4422-4437, December 2007.
21. A. S. Nemirovsky and D. B. Yudin. Problem Complexity and Method Efficiency in Optimization. New York, 1983.
22. N. Noorshams and M. J. Wainwright. Quantized stochastic belief propagation: Efficient messagepassing for continuous state spaces. In Proceedings of the IEEE International Symposium on Information Theory, 2012.
23. N. Noorshams and M. J. Wainwright. Stochastic belief propagation: A low-complexity alternative to the sum-product algorithms. IEEE Transactions on Information Theory, 59(4):1981-2000, April 2013.
24. F. Riesz and B.S. Nagy. Functional Analysis. Dover Publications Inc., New York, 1990.
25. B. D. Ripley. Stochastic Simulation. Wiley, New York, 1987.
26. T. G. Roosta, M. J. Wainwright, and S. S. Sastry. Convergence analysis of reweighted sum-product algorithms. IEEE Transactions on Signal Processing, 56(9):4293-4305, September 2008.
27. H. L. Royden. Real Analysis. Prentice-Hall, New Jersey, 1988.
28. L. Song, A. Gretton, D. Bickson, Y. Low, and C. Guestrin. Kernel belief propagation. In Proceedings of the Artificial Intelligence and Statistics, 2011.
29. I. Steinwart and A. Christmann. Support Vector Machines. Springer, New York, 2008.
30. E. B. Sudderth, A. T. Ihler, M. Israd, W. T. Freeman, and A. S. Willsky. Nonparametric belief propagation. Communications of the ACM Magazine, 53(10):95-103, 2010.
31. S. Tatikonda and M. I. Jordan. Loopy belief propagation and Gibbs measures. In Proc. Uncertainty in Artificial Intelligence, volume 18, pages 493-500, August 2002.
32. M. J. Wainwright and M. I. Jordan. Graphical Models, Exponential Families, and Variational Inference. Now Publishers Inc, Hanover, MA 02339, USA, 2008.
33. J. S. Yedidia, W. T. Freeman, and Y. Weiss. Constructing free energy approximations and generalized belief propagation algorithms. IEEE Transaction on Information Theory, 51(7):2282-2312, July 2005. (Pubitemid 41136394)