On the computational efficiency of training neural networks

Advances in Neural Information Processing Systems

Volumn 1, Issue January, 2014, Pages 855-863

  Livni, Roi ^a   Shalev Shwartz, Shai ^a   Shamir, Ohad ^b  

^a HEBREW UNIVERSITY OF JERUSALEM   (Israel)

^b WEIZMANN INSTITUTE OF SCIENCE   (Israel)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTATIONAL EFFICIENCY; INFORMATION SCIENCE;

ACTIVATION FUNCTIONS;

COMPLEX NETWORKS;

EID: 84937952927     PISSN: 10495258     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (24)

1. A. Andoni, R. Panigrahy, G. Valiant, and L. Zhang. Learning polynomials with neural networks. In ICML, 2014.
2. A. Andoni, R. Panigrahy, G. Valiant, and L. Zhang. Learning sparse polynomial functions. In SODA, 2014.
3. M. Anthony and P. Bartlett. Neural Network Learning - Theoretical Foundations. Cambridge University Press, 2002.
4. S. Arora, A. Bhaskara, R. Ge, and T. Ma. Provable bounds for learning some deep representations. arXiv preprint arXiv:1310.6343, 2013.
5. P. Auer, M. Herbster, and M. Warmuth. Exponentially many local minima for single neurons. In NIPS, 1996.
6. P. L. Bartlett and S. Ben-David. Hardness results for neural network approximation problems. Theor. Comput. Sci., 284(1):53-66, 2002.
7. Y. Bengio, A. Courville, and P. Vincent. Representation learning: A review and new perspectives. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35:1798-1828, 2013.
8. E. Blais, R. O'Donnell, and K. Wimmer. Polynomial regression under arbitrary product distributions. Machine Learning, 80(2-3):273-294, 2010.
9. A. Blum and R. Rivest. Training a 3-node neural network is np-complete. Neural Networks, 5(1):117-127, 1992.
10. G. Dahl, T. Sainath, and G. Hinton. Improving deep neural networks for lvcsr using rectified linear units and dropout. In ICASSP, 2013.
11. N. Dalal and B. Triggs. Histograms of oriented gradients for human detection. In CVPR, 2005.
12. A. Daniely, N. Linial, and S. Shalev-Shwartz. From average case complexity to improper learning complexity. In FOCS, 2014.
13. A. Kalai, A. Klivans, Y. Mansour, and R. Servedio. Agnostically learning halfspaces. SIAM J. Comput., 37(6):1777-1805, 2008.
14. A. Kalai, A. Samorodnitsky, and S.-H. Teng. Learning and smoothed analysis. In FOCS, 2009.
15. A. Klivans and A. Sherstov. Cryptographic hardness for learning intersections of halfspaces. In FOCS, 2006.
16. A. Krizhevsky, I. Sutskever, and G. Hinton. Imagenet classification with deep convolutional neural networks. In NIPS, 2012.
17. Q. V. Le, M.-A. Ranzato, R. Monga, M. Devin, G. Corrado, K. Chen, J. Dean, and A. Y. Ng. Building high-level features using large scale unsupervised learning. In ICML, 2012.
18. N. Pippenger and M. Fischer. Relations among complexity measures. Journal of the ACM (JACM), 26(2):361-381, 1979.
19. S. Shalev-Shwartz and S. Ben-David. Understanding Machine Learning: From Theory to Algorithms. Cambridge University Press, 2014.
20. S. Shalev-Shwartz, A. Gonen, and O. Shamir. Large-scale convex minimization with a low-rank constraint. In ICML, 2011.
21. S. Shalev-Shwartz, O. Shamir, and K. Sridharan. Learning kernel-based halfspaces with the 0-1 loss. SIAM Journal on Computing, 40(6):1623-1646, 2011.
22. M. Sipser. Introduction to the Theory of Computation. Thomson Course Technology, 2006.
23. I. Sutskever, J. Martens, G. Dahl, and G. Hinton. On the importance of initialization and momentum in deep learning. In ICML, 2013.
24. M. Zeiler and R. Fergus. Visualizing and understanding convolutional neural networks. arXiv preprint arXiv:1311.2901, 2013.