On measure concentration of random maximum a-posteriori perturbations

31st International Conference on Machine Learning, ICML 2014

Volumn 1, Issue , 2014, Pages 674-692

  Orabona, Francesco ^a   Hazan, Tamir ^b   Sarwate, Anand D ^c   Jaakkola, Tommi S ^d  

^a TOYOTA TECHNOLOGICAL INSTITUTE AT CHICAGO   (United States)

^b UNIVERSITY OF HAIFA   (Israel)

^c RUTGERS UNIVERSITY   (United States)

^d MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHMS; ARTIFICIAL INTELLIGENCE;

COMPUTATIONAL COSTS; GIBBS DISTRIBUTION; MAXIMUM A POSTERIORI; MEASURE CONCENTRATION; MONTE-CARLO ESTIMATION; NUMBER OF SAMPLES; POTENTIAL FUNCTION; SEQUENTIAL SAMPLING;

LEARNING SYSTEMS;

EID: 84919794589     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (32)

1. Azuma, K. Weighted sums of certain dependent random variables. Tdhoku Mathematical Journal, 19(3): 357-367, 1967.
2. Bartlett, P. L. and Mendelson, S. Rademacher and Gaussian complexities: Risk bounds and structural results. JMLR, 3:463-482, 2003.
3. Bernstein, S. The Theory of Probabilities. Gastehizdat Publishing House, Moscow, 1946.
4. Bobkov, S. and Ledoux, M. Poincare's inequalities and Talagrand's concentration phenomenon for the exponential measure. Probability Theory and Related Fields, 107(3): 383-400, March 1997.
5. Boucheron, S., Lugosi, G., andBousquet, O. Concentration inequalities. In Advanced Lectures on Machine Learning, pp. 208-240. Springer, 2004.
6. Bousquet, O. Concentration inequalities for sub-additive functions using the entropy method. In Stochastic inequalities and applications, pp. 213-247. Springer, 2003.
7. Brascamp, H. J. and Lieb, E. H. On extensions of the Brunn-Minkowski and Prekopa-Leindler theorems, including inequalities for log concave functions, and with an application to the diffusion equation. J. Func. Analysis, 22(4):366-389, August 1976.
8. Ermon, S., Gomes, C. P., Sabharwal, A., and Selman, B. Embed and project: Discrete sampling with universal hashing. In Advances in Neural Information Processing Systems, pp. 2085-2093, 2013.
9. Felzenszwalb, P.F. and Zabih, R. Dynamic programming and graph algorithms in computer vision. IEEE Trans. PAMI, 33(4):721-740, 2011.
10. Gumbel, E. J. and Lieblein, J. Statistical theory of extreme values and some practical applications: a series of lectures. Number 33 in National Bureau of Standards Applied Mathematics Series. US Govt. Print. Office, Washington, DC, 1954.
11. Hazan, T. and Jaakkola, T. On the partition function and random maximum a-posteriori perturbations. In ICML, 2012.
12. Hazan, T., Maji, S., J., Keshet, and Jaakkola, T. Learning efficient random maximum a-posteriori predictors with non-decomposable loss functions. Advances in Neural Information Processing Systems, 2013a.
13. Hazan, T., Maji, S., and Jaakkola, T. On sampling from the gibbs distribution with random maximum a-posteriori perturbations. Advances in Neural Information Processing Systems, 2013b.
14. Hoeffding, W. Probability inequalities for sums of bounded random variables. JASA, 58(301): 13-30, March 1963.
15. Huber, M. A bounding chain for swendsen-wang. Random Structures & Algorithms, 22(1):43-59, 2003.
16. Jerrum, M., Sinclair, A., and Vigoda, E. A polynomial-time approximation algorithm for the permanent of a matrix with nonnegative entries. JACM, 51 (4):671-697, 2004.
17. Keshet, J., McAllester, D., and Hazan, T. PAC-Bayesian approach for minimization of phoneme error rate. In ICASSP, 2011.
18. Koller, D. and Friedman, N. Probabilistic graphical models. MIT press, 2009.
19. Kolmogorov, V. Convergent tree-reweighted message passing for energy minimization. PAMI, 28(10), 2006.
20. Koo, T., Rush, A.M., Collins, M., Jaakkola, T., and Sontag, D. Dual decomposition for parsing with non-projective head automata. In EMNLP, 2010.
21. Ledoux, M. The Concentration of Measure Phenomenon, volume 89 of Mathematical Surveys and Monographs. American Mathematical Society, 2001.
22. Ledoux, M. and Talagrand, M. Probability in Banach Spaces: isoperimetry and processes, volume 23. Springer, 1991.
23. Maurey, B. Construction de suites symétriques. Comptes Rendus de l'Académie des Sciences, Paris, Série A-B, 288:A679-681, 1979.
24. McAllester, D. Simplified PAC-Bayesian margin bounds. Learning Theory and Kernel Machines, pp. 203-215, 2003.
25. McDiarmid, C. On the method of bounded differences. In Surveys in Combinatorics, number 141 in London Mathematical Society Lecture Note Series, pp. 148-188. Cambridge University Press, Cambridge, 1989.
26. Orabona, Francesco, Hazan, Tamir, Sarwate, Anand D, and Jaakkola, Tommi. On measure concentration of random maximum a-posteriori perturbations. arXiv: 1310.4227, 2013.
27. Papandreou, G. and Yuille, A. Perturb-and-MAP random fields: Using discrete optimization to learn and sample from energy models. In ICCV, 2011.
28. Pisier, G. Probabilistic methods in the geometry of Banach spaces. In Probabilty and Analysis, Varenna (Italy) 1985, volume 1206 of Lecture Notes in Mathematics. Springer, Berlin, 1986.
29. Sontag, D., Meltzer, T., Globerson, A., Jaakkola, T., and Weiss, Y. Tightening LP relaxations for MAP using message passing. In UAI, 2008.
30. Tarlow, D., Adams, R.P., and Zemel, R.S. Randomized optimum models for structured prediction. In AISTATS, pp. 21-23, 2012.
31. Valiant, L.G. The complexity of computing the permanent. Theoretical computer science, 8(2): 189-201, 1979.
32. Werner, T. High-arity interactions, polyhedral relaxations, and cutting plane algorithm for soft constraint optimisation (MAP-MRF). In CVPR, 2008.