Nonparametric neural networks

5th International Conference on Learning Representations, ICLR 2017 - Conference Track Proceedings

Volumn , Issue , 2017, Pages

  Philipp, George ^a   Carbonell, Jaime G ^a  

^a CARNEGIE MELLON UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

GRADIENT METHODS;

GLOBAL SEARCH; GRADIENT DESCENT; NETWORK GROWTH; NON-PARAMETRIC; NON-PROBABILISTIC; OPTIMIZATION ALGORITHMS; PRIOR INFORMATION; REDUNDANT UNITS;

OPTIMIZATION;

EID: 85052107291     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (42)

1. Jose M. Alvarez and Mathieu Salzmann. Learning the number of neurons in deep networks. In NIPS, 2016.
2. Timur Ash. Dynamic node creation in backpropagation networks. Institute for Cognitive Science, UCSD, Technical Report 8901, 1989.
3. Lei Jimmy Ba and Rich Caruana. Do deep nets really need to be deep? In NIPS, 2014.
4. Amir Back and Marc Teboulle. A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J Imaging Sciences, 2:183-202, 2006.
5. James Bergstra and Yoshua Bengio. Random search for hyper-parameter optimization. JMLR, 13: 281-305, 2012.
6. Tianqi Chen, Ian Goodfellow, and Jonathon Shlens. Net2net: accelerating learning via knowledge transfer. In ICLR, 2016.
7. Maxwell D. Collins and Pushmeet Kohli. Memory bounded deep convolutional networks. CoRR, pp. abs/1412.1442, 2014.
8. George E. Dahl, Tara N. Sainath, and Geoffrey E. Hinton. Improving deep neural networks for lvcsr using rectified linear units and dropout. In ICASSP, 2013.
9. Juan F. De Freitas. Bayesian methods for neural networks. PhD thesis, Trinity College, University of Cambridge, 2003.
10. John Duchi, Elad Hazan, and Yoram Singer. Adaptive subgradient methods for online learning and stochastic optimization. JMLR, 12:2121-2159, 2011.
11. Scott Fahlman and Christian Lebiere. The cascade-correlation learning architecture. In NIPS, 1990.
12. Jiashi Feng and Trevor Darrell. Learning the structure of deep convolutional networks. In ICCV, 2015.
13. Michael Figurnov, Aijan Ibraimova, Dmitry Vetrov, and Pushmeet Kohli. Perforatedcnns: Acceleration through elimination of redundant convolutions. In NIPS, 2016.
14. Stephen Gallant. Three constructive algorithms for network learning. In Conference of the Cognitive Learning Society, 1986.
15. Yiwen Guo, Anbang Yao, and Yurong Chen. Dynamic network durgery for efficient dnns. In NIPS, 2016.
16. Tamir Hazan and Tommi Jaakkola. Steps toward deep kernel methods from infinite neural networks. arXiv preprint arXiv:1508.05133, 2015.
17. Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531, 2015.
18. Frank Hutter, Holger H. Hoos, and Kevin Leyton-Brown. Sequential model-based optimization for general algorithm configuration (extended version). Tech. Rep. TR-2009-01, University of British Columbia, Department of Computer Science, 2009.
19. Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In ICML, 2015.
20. Diederik P. Kingma and Jimmy Lei Ba. Adam: A method for stochastic optimization. In ICLR, 2015.
21. Aaron Klein, Stefan Falkner, Jost Tobias Springenberg, and Frank Hutter. Dsd: Dense-sparse-dense training for deep neural networks. In ICLR, 2017.
22. Jan Larsen, Claus Svarer, Lars Nonboe Andersen, and Lars Kai Hansen. Adaptive regularization in neural network modeling. Neural Networks: Tricks of the Trade, 2nd Ed., 7700:111-130, 1998.
23. Jelena Luketina, Mathias Berglund, Klaus Greff, and Raiko Tapani. Scalable gradient-based tuning of continuous regularization hyperparameters. In ICML, 2016.
24. Dougal Maclaurin, David Duvenaud, and Ryan P. Adams. Gradient-based hyperparameter optimization through reversible learning. In ICML, 2015.
25. David McKay. A practical bayesian framework for backpropagation networks. Neural Computation, 4:448-472, 1992.
26. Pavlo Molchanov, Stephen Tyree, Tero Karras, Timo Aila, and Jan Kautz. Pruning convolutional neural networks for efficient inference. In ICLR, 2017.
27. Gaurav Pandey and Ambedkar Dukkipati. Learning by stretching deep networks. In ICML, 2014.
28. Adriana Romero, Nicolas Ballas, Samira Ebrahimi Kahou, Antoine Chassang, Carlo Gatta, and Yoshua Bengio. Fitnets: Hints for thin deep nets. In ICLR, 2015.
29. Andrew Saxe, Pang Wei Koh, Zhenghao Chen, Maneesh Bhand, Bipin Suresh, and Andrew Ng. Computing with infinite networks. In ICML, 2011.
30. Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. In ICLR, 2015.
31. Jasper Snoek, Hugo Larochelle, and Ryan P. Adams. Practical bayesian optimization of machine learning algorithms. In NIPS, 2012.
32. Jasper Snoek, Oren Rippel, Kevin Swersky, Ryan Kiros, Nadathur Satish, Narayanan Sundaram, Md. Mostofa Ali Patwary, Prabhat, and Ryan P. Adams. Scalable bayesian optimization using deep neural networks. In ICML, 2015.
33. Jost Tobias Springenberg, Aaron Klein, Stefan Falkner, and Frank Hutter. Bayesian optimization with robust bayesian neural networks. In NIPS, 2016.
34. Ilya Sutskever, James Martens, George Dahl, and Geoffrey Hinton. On the importance of initialization and momentum in deep learning. In ICML, 2013.
35. Robert Tibshirani. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society, Series B, 58:267-288, 1996.
36. Tijmen Tieleman and Geoffrey Hinton. Lecture 6.5 - rmsprop, coursera: Neural networks for machine learning. 2012.
37. Tao Wei, Changhu Wang, Yong Rui, and Chang Wen Chen. Network morphism. In ICML, 2016.
38. Wei Wen, Chunpeng Wu, Wandan Wang, Yiran Chen, and Hai Li. Learning structured sparsity in deep neural networks. In NIPS, 2016.
39. Christopher K. I. Williams. Computing with infinite networks. In NIPS, 1997.
40. Ming Yuan and Yin Lin. Model selection and estimation in regression with grouped variables. Journal of the Royal Statistical Society, Series B, 68:49-67, 2006.
41. Matthew D. Zeiler. Adadelta: An adaptive learning rate method. arXiv preprint arXiv:1212.5701, 2012.
42. Barret Zoph and Quoc V. Le. Neural architecture search with reinforcement learning. In ICLR, 2017.