Deep learning without poor local minima

Advances in Neural Information Processing Systems

Volumn , Issue , 2016, Pages 586-594

  Kawaguchi, Kenji ^a  

^a MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

CONVEX OPTIMIZATION; DEEP LEARNING; EIGENVALUES AND EIGENFUNCTIONS; GLOBAL OPTIMIZATION; LEARNING SYSTEMS;

INDEPENDENCE ASSUMPTION; LEARNING THEORY; LINEAR NEURAL NETWORK; MACHINE LEARNING MODELS; NONCONVEX OPTIMIZATION; NONLINEAR NEURAL NETWORKS; THEORETICAL FOUNDATIONS; THEORY AND PRACTICE;

DEEP NEURAL NETWORKS;

EID: 85019256625     PISSN: 10495258     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (15)

1. Baldi, Pierre. 1989. Linear learning: Landscapes and algorithms. In Advances in neural information processing systems. pp. 65-72.
2. Baldi, Pierre, & Hornik, Kurt. 1989. Neural networks and principal component analysis: Learning from examples without local minima. Neural networks, 2(1), 53-58.
3. Baldi, Pierre, & Lu, Zhiqin. 2012. Complex-valued autoencoders. Neural Networks, 33, 136-147.
4. Blum, Avrim L, & Rivest, Ronald L. 1992. Training a 3-node neural network is NP-complete. Neural Networks, 5(1), 117-127.
5. Choromanska, Anna, Henaff, MIkael, Mathieu, Michael, Ben Arous, Gerard, & LeCun, Yann. 2015a. The Loss Surfaces of Multilayer Networks. In Proceedings of the Eighteenth International Conference on Artificial Intelligence and Statistics. pp. 192-204.
6. Choromanska, Anna, LeCun, Yann, & Arous, Gérard Ben. 2015b. Open Problem: The landscape of the loss surfaces of multilayer networks. In Proceedings of The 28th Conference on Learning Theory. pp. 1756-1760.
7. Dauphin, Yann N, Pascanu, Razvan, Gulcehre, Caglar, Cho, Kyunghyun, Ganguli, Surya, & Bengio, Yoshua. 2014. Identifying and attacking the saddle point problem in high-dimensional non-convex optimization. In Advances in Neural Information Processing Systems. pp. 2933-2941.
8. Ge, Rong, Huang, Furong, Jin, Chi, & Yuan, Yang. 2015. Escaping From Saddle Points-Online Stochastic Gradient for Tensor Decomposition. In Proceedings of The 28th Conference on Learning Theory. pp. 797-842.
9. Goodfellow, Ian, Bengio, Yoshua, & Courville, Aaron. 2016. Deep Learning. Book in preparation for MIT Press. http://www.deeplearningbook.org.
10. Livni, Roi, Shalev-Shwartz, Shai, & Shamir, Ohad. 2014. On the computational efficiency of training neural networks. In Advances in Neural Information Processing Systems. pp. 855-863.
11. Mhaskar, Hrushikesh, Liao, Qianli, & Poggio, Tomaso. 2016. Learning Real and Boolean Functions: When Is Deep Better Than Shallow. Massachusetts Institute of Technology CBMM Memo No. 45.
12. Murty, Katta G, & Kabadi, Santosh N. 1987. Some NP-complete problems in quadratic and nonlinear programming. Mathematical programming, 39(2), 117-129.
13. Rockafellar, R Tyrrell, & Wets, Roger J-B. 2009. Variational analysis. Vol. 317. Springer Science & Business Media.
14. Saxe, Andrew M, McClelland, James L, & Ganguli, Surya. 2014. Exact solutions to the nonlinear dynamics of learning in deep linear neural networks. In International Conference on Learning Representations.
15. Zhang, Fuzhen. 2006. The Schur complement and its applications. Vol. 4. Springer Science & Business Media.