Hybrid evolution of convolutional networks

Proceedings - 10th International Conference on Machine Learning and Applications, ICMLA 2011

Volumn 1, Issue , 2011, Pages 293-297

  Cheung, Brian ^a   Sable, Carl ^a  

^a Cooper Union ^*  (United States)

Author keywords

convolutional networks; evolution; image classification; neural networks; second order methods

Indexed keywords

ABSOLUTE VALUES; ARCHITECTURAL PARAMETERS; CONVOLUTIONAL NETWORKS; EVOLUTION; EVOLUTIONARY SEARCH; EXPONENTIAL INCREASE; FITNESS EVALUATIONS; HYBRID EVOLUTION; IMAGE DATASETS; LEVENBERG-MARQUARDT; LOCAL CONTRAST; NETWORK MODELS; NEURAL NETWORK MODEL; SECOND ORDERS; TIME COST;

IMAGE CLASSIFICATION; LEARNING SYSTEMS; NEURAL NETWORKS;

EVOLUTIONARY ALGORITHMS;

EID: 84857883444     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICMLA.2011.73     Document Type: Conference Paper

References (22)

1. G. E. Hinton, S. Osindero, and Y.-W. Teh, "A fast learning algorithm for deep belief nets," Neural Computation, vol. 18, no. 7, pp. 1527-1554, 2006. [Online]. Available: http://www.mitpressjournals.org/doi/abs/10.1162/neco. 2006.18.7.1527
2. H. Larochelle, D. Erhan, A. C. Courville, J. Bergstra, and Y. Bengio, "An empirical evaluation of deep architectures on problems with many factors of variation," in ICML, ser. ACM International Conference Proceeding Series, Z. Ghahramani, Ed., vol. 227. ACM, 2007, pp. 473-480. [Online]. Available: http://doi.acm.org/10.1145/1273496.1273556
3. Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, "Gradient-based learning applied to document recognition," Proceedings of the IEEE, vol. 86, no. 11, pp. 2278-2324, November 1998.
4. A. M. Saxe, P. W. Koh, K. Kavukcuoglu, Z. Chen, M. Bhand, B. Suresh, and A. Y. Ng, "On random weights and unsupervised feature learning,"in The 28th International Conference on Machine Learning, 2011.
5. D. Erhan, Y. Bengio, A. Courville, P.-A. Manzagol, P. Vincent, and S. Bengio, "Why does unsupervised pre-training help deep learning?"J. Mach. Learn. Res., vol. 11, pp. 625-660, March 2010. [Online]. Available: http://portal.acm.org/citation.cfm?id=1756006.1756025
6. A. Coates, H. Lee, and A. Y. Ng, "An analysis of single-layer networks in unsupervised feature learning," in NIPS Workshop on Deep Learning and Unsupervised Feature Learning, 2010.
7. X. Yao and Y. Liu, "A new evolutionary system for evolving artificial neural networks," IEEE Transactions on Neural Networks, vol. 8, no. 3, pp. 694-713, May 1997. (Pubitemid 127767812)
8. X. Yao, "Evolving artificial neural networks," Proceedings of the IEEE, vol. 87, no. 9, pp. 1423-1447, Sep. 1999.
9. Y. Bengio and X. Glorot, "Understanding the difficulty of training deep feedforward neural networks," in Proceedings of AISTATS 2010, vol. 9, May 2010, pp. 249-256.
10. Y. LeCun, F.-J. Huang, and L. Bottou, "Learning methods for generic object recognition with invariance to pose and lighting," in Proceedings of CVPR'04. IEEE Press, 2004.
11. K. Jarrett, M. Ranzato, K. Kavukcuoglu, and Y. LeCun, "High-accuracy object recognition with a new convolutional net architecture and learning algorithm," in The Learning Workshop, 2009.
12. K. Jarrett, K. Kavukcuoglu, M. Ranzato, and Y. LeCun, "What is the best multi-stage architecture for object recognition?" in Proc. International Conference on Computer Vision (ICCV'09). IEEE, 2009.
13. N. Pinto, D. D. Cox, and J. J. DiCarlo, "Why is real-world visual object recognition hard?" PLoS Comput Biol, vol. 4, no. 1, p. e27, 01 2008. [Online]. Available: http://dx.plos.org/10.1371%2Fjournal.pcbi.0040027
14. Y. LeCun, L. Bottou, G. Orr, and K. Muller, "Efficient backprop," in Neural Networks: Tricks of the trade, G. Orr and M. K., Eds. Springer, 1998.
15. J. Martens, "Deep learning via hessian-free optimization," in ICML, J. Fürnkranz and T. Joachims, Eds. Omnipress, 2010, pp. 735-742. [Online]. Available: http://www.icml2010.org/papers/458.pdf
16. A. Ranganathan, "The levenberg-marquardt algorithm," Jun. 08 2004. [Online]. Available: http://citeseer.ist.psu.edu/638988.html;http://www. cc.gatech.edu/people/home/ananth/lmtut.pdf
17. P. J. Angeline, "Adaptive and self-adaptive evolutionary computations," in Computational Intelligence: A Dynamic Systems Perspective, M. Palaniswami and Y. Attikiouzel, Eds. IEEE Press, 1995, pp. 152-163. [Online]. Available: http://citeseer.ist.psu.edu/cache/papers/cs/1007/ http:zSzzSzwww.natural-selection.comzSzpeoplezSzpjazSzdocszSzicec95.pdf/ angeline95adaptive.pdf
18. P. Sermanet, K. Kavukcuoglu, and Y. LeCun, "Eblearn: Open-source energy-based learning in c++," in Proc. International Conference on Tools with Artificial Intelligence (ICTAI'09). IEEE, 2009.
19. Y. Tang, "Robust visual recognition using multilayer generative neural networks," 2010, unpublished.
20. S. Rifai, X. Muller, X. Glorot, G. Mesnil, Y. Bengio, and P. Vincent, "Learning invariant features through local space contraction," CoRR, vol. abs/1104.4153, 2011, informal publication. [Online]. Available: http://arxiv.org/abs/1104.4153
21. P. Vincent, H. Larochelle, I. Lajoie, Y. Bengio, and P.-A. Manzagol, "Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion," Journal of Machine Learning Research, vol. 11, pp. 3371-3408, 2010. [Online]. Available: http://portal.acm.org/citation.cfm?id=1953039
22. R. Memisevic, C. Zach, G. Hinton, and M. Pollefeys, "Gated softmax classification," in Advances in Neural Information Processing Systems 23, J. Lafferty, C. K. I. Williams, J. Shawe-Taylor, R. Zemel, and A. Culotta, Eds., 2010, pp. 1603-1611.