On the complexity of shallow and deep neural network classifiers

22nd European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning, ESANN 2014 - Proceedings

Volumn , Issue , 2014, Pages 371-376

  Bianchini, Monica ^a   Scarselli, Franco ^a  

^a UNIVERSITY OF SIENA   (Italy)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE; LEARNING SYSTEMS; NETWORK ARCHITECTURE; NEURAL NETWORKS;

ACTIVATION FUNCTIONS; DEEP ARCHITECTURES; DEEP NEURAL NETWORKS; FEED-FORWARD NETWORK; HIDDEN UNITS; HIGH COMPLEXITY; NETWORK COMPLEXITY; UPPER AND LOWER BOUNDS;

COMPLEX NETWORKS;

EID: 84962018700     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (17)

1. Y. Bengio, Y. LeCun, R. Salakhutdinov, and H. Larochelle, editors. Proceedings of the Deep Learning Workshop: Foundations and Future Directions (NIPS 2007). 2007.
2. H. Lee, M. Ranzato, Y. Bengio, G.E. Hinton, Y. LeCun, and A.Y. Ng, editors. Proceedings of the Deep Learning and Unsupervised Feature Learning Workshop (NIPS 2010). 2010.
3. K. Yu, R. Salakhutdinov, Y. LeCun, G.E. Hinton, and Y. Bengio, editors. Proceedings of the Workshop on Learning Feature Hierarchies (ICML 2009). 2009.
4. Y. Bengio. Learning deep architectures for AI. Foundations and Trends in Machine Learning, 2(1):1-127, 2009.
5. Y. Bengio and O. Delalleau. Shallow vs. deep sum-products networks. In Advances in Neural Information Processing Systems, volume 24, pages 666-674, 2011.
6. O. Delalleau and Y. Bengio. On the expressive power of deep architectures. In Proceedings of the 22nd International Conference on Algorithmic Learning Theory, pages 18-36, 2011.
7. M. Bianchini and F. Scarselli. On the complexity of neural network classifiers: A comparison between shallow and deep architectures. IEEE Transactions on Neural Networks, 2014. To be published.
8. G.E. Bredon. Topology and Geometry, Graduate Texts in Mathematics. Springer, 1993.
9. F. Scarselli and A.C. Tsoi. Universal approximation using feedforward neural networks: A survey of some existing methods, and some new results. Neural Networks, 11:15-37, 1998.
10. B. Hammer. On the approximation capability of recurrent neural networks. Neurocom- puting, 31(1-4):107-123, 2000.
11. M. Bianchini, M. Maggini, L. Sarti, and F. Scarselli. Recursive neural networks for processing graphs with labelled edges: Theory and applications. Neural Networks, 18(8):1040-1050, 2005.
12. F. Scarselli, M. Gori, A.C. Tsoi, M. Hagenbuchner, and G. Monfardini. Computational capabilities of graph neural networks. IEEE Transactions on Neural Networks, 20(1):81-102, 2009.
13. V. Kurkova, P. Savicky, and K. Hlavackova. Representations and rates of approximation of real-valued boolean functions by neural networks. Neural Networks, 11(4):651-659, 1998.
14. J. Håstad. Almost optimal lower bounds for small depth circuits. In Proceedings of the 18th Annual ACM Symposium on Theory of Computing, pages 6-20. ACM, 1986.
15. J. Håstad and M. Goldmann. On the power of small-depth threshold circuits. Computational Complexity, 1(2):113-129, 1991.
16. P.L. Bartlett and W. Maass. Vapnik-Chervonenkis dimension of neural nets. In M.A. Arbib, editor, The Handbook of Brain Theory and Neural Networks, pages 1188-1192. Cambridge, MA: MIT Press, 2003. Second Edition.
17. M. Maillot, M. Aupetit, and G. Govaert. A generative model that learns Betti num- bers from a data set. In ESANN2012, 15th European Symposium on Artificial Neural Networks, pages 537-542, Bruges, Belgium, 2012.