On graduated optimization for stochastic non-convex problems

33rd International Conference on Machine Learning, ICML 2016

Volumn 4, Issue , 2016, Pages 2726-2739

  Hazan, Elad ^a   Levy, Kfir Y ^b   Shalev Shwartz, Shai ^c  

^a PRINCETON UNIVERSITY   (United States)

^b TECHNION ISRAEL INSTITUTE OF TECHNOLOGY   (Israel)

^c HEBREW UNIVERSITY OF JERUSALEM   (Israel)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE; CONVEX OPTIMIZATION; FUNCTIONS; GLOBAL OPTIMIZATION; HEURISTIC METHODS; LEARNING SYSTEMS; STOCHASTIC SYSTEMS;

APPROXIMATE SOLUTION; CONTINUATION METHOD; CONVERGENCE ANALYSIS; CONVERGENCE RATES; FIRST-ORDER ALGORITHMS; NONCONVEX FUNCTIONS; NONCONVEX OPTIMIZATION; OPTIMIZATION APPROACH;

OPTIMIZATION;

EID: 84998952820     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (21)

1. Allgower, Eugene L and Georg, Kurt. Numerical continuation methods, volume 13. Springer-Verlag Berlin, 1990.
2. Bengio, Yoshua. Learning deep architectures for ai. Foundations and trends in Machine Learning, 2(1): 1-127, 2009.
3. Blake, Andrew and Zisserman, Andrew. Visual reconstruction, volume 2. MIT press Cambridge, 1987.
4. Boccuto, A, Discepoli, M, Gerace, I, Pandolfi, R, and Pucci, P. A gnc algorithm for deblurring images with interacting discontinuities. Proc. VISIMAI, pp. 296-310, 2002.
5. Brox, Thomas and Malik, Jitendra. Large displacement optical flow: descriptor matching in variational motion estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(3):500-513, 2011.
6. Chapelle, Olivier and Wu, Mingrui. Gradient descent optimization of smoothed information retrieval metrics. Information retrieval, 13(3):216-235, 2010.
7. Chapelle, Olivier, Chi, Mingmin, and Zien, Alexander. A continuation method for semi-supervised svms. In Pro ceedings of the 23rd international conference on Machine learning, pp. 185-192. ACM, 2006.
8. Chaudhuri, Swarat and Solar-Lezama, Armando. Smoothing a program soundly and robustly. In Computer Aided Verification, pp. 277-292. Springer, 2011.
9. Dauphin, Yann N, Pascanu, Razvan, Gulcehre, Caglar, Cho, Kyunghyun, Ganguli, Surya, and Bengio, Yoshua. Identifying and attacking the saddle point problem in high-dimensional non-convex optimization. In Advances in Neural Information Processing Systems, pp. 2933- 2941, 2014.
10. Erhan, Dumitru, Manzagol, Pierre-Antoine, Bengio, Yoshua, Bengio, Samy, and Vincent, Pascal. The difficulty of training deep architectures and the effect of unsupervised pre-training. In International Conference on Artificial Intelligence and Statistics, pp. 153-160, 2009.
11. Flaxman, Abraham, Kalai, Adam Tauman, and McMahan, H. Brendan. Online convex optimization in the bandit setting: gradient descent without a gradient. In SODA, pp. 385-394, 2005.
12. Hinton, Geoffrey, Osindero, Simon, and Teh, Yee-Whye. A fast learning algorithm for deep belief nets. Neural computation, 18(7): 1527-1554, 2006.
13. Mobahi, Hossein and Fisher III, John W. On the link between Gaussian homotopy continuation and convex envelopes. In Energy Minimization Methods in Computer Vision and Pattern Recognition, pp. 43-56. Springer, 2015a.
14. Mobahi, Hossein and Fisher HI, John W. A theoretical analysis of optimization by Gaussian continuation. 2015b.
15. Nikolova, Mila, Ng, Michael K, and Tam, Chi-Pan. Fast nonconvex nonsmooth minimization methods for image restoration and reconstruction. IEEE Transactions on Image Processing, 19(12):3073, 2010.
16. Rakhlin, Alexander, Shamir, Ohad, and Sridharan, Karthik. Making gradient descent optimal for strongly convex stochastic optimization. arXiv preprint arXiv. l 109.5647, 2011.
17. Terzopoulos, Demetri. The computation of visible-surface representations. IEEE Transactions on Pattern Analysis and Machine Intelligence, 10(4):417-438, 1988.
18. Wu, Zhijun. The effective energy transformation scheme as a special continuation approach to global optimization with application to molecular conformation. SIAM Journal on Optimization, 6(3):748-768, 1996.
19. Yuille, AL. Energy functions for early vision and analog networks. Biological Cybernetics, 61(2): 115-123, 1989.
20. Yuille, Alan L, Geiger, Davi, and Biilthoff, H. Stereo integration, mean field theory and psychophysics. In Computer Vision ECCV90, pp. 71-82. Springer, 1990.
21. Zaslavskiy, Mikhail, Bach, Francis, and Vert, J-P. A path following algorithm for the graph matching problem. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 31(12):2227-2242, 2009.