Neural networks with few multiplications

4th International Conference on Learning Representations, ICLR 2016 - Conference Track Proceedings

Volumn , Issue , 2016, Pages

  Lin, Zhouhan ^a   Courbariaux, Matthieu ^a   Memisevic, Roland ^a   Bengio, Yoshua ^a  

^a UNIVERSITÉ DE MONTRÉAL   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

CLASSIFICATION (OF INFORMATION); DIGITAL ARITHMETIC; GRADIENT METHODS; LEARNING ALGORITHMS; STOCHASTIC SYSTEMS;

BINARIZE; CLASSIFICATION PERFORMANCE; FLOATING POINT MULTIPLICATION; HIDDEN STATE; SIGN-CHANGE; STOCHASTIC GRADIENT DESCENT;

DEEP LEARNING;

EID: 85083953250     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (18)

1. Bastien, Frédéric, Lamblin, Pascal, Pascanu, Razvan, Bergstra, James, Goodfellow, Ian J., Bergeron, Arnaud, Bouchard, Nicolas, and Bengio, Yoshua. Theano: new features and speed improvements. Deep Learning and Unsupervised Feature Learning NIPS 2012 Workshop, 2012.
2. Burge, Peter S., van Daalen, Max R., Rising, Barry J. P., and Shawe-Taylor, John S. Stochastic bit-stream neural networks. In Maass, Wolfgang and Bishop, Christopher M. (eds.), Pulsed Neural Networks, pp. 337–352. MIT Press, Cambridge, MA, USA, 1999. ISBN 0-626-13350-4. URL http://dl.acm.org/citation.cfm?id=296533.296552.
3. Cheng, Zhiyong, Soudry, Daniel, Mao, Zexi, and Lan, Zhenzhong. Training binary multilayer neural networks for image classification using expectation backpropagation. arXiv preprint arXiv:1503.03562, 2015.
4. Courbariaux, Matthieu, Bengio, Yoshua, and David, Jean-Pierre. Binaryconnect: Training deep neural networks with binary weights during propagations. arXiv preprint arXiv:1511.00363, 2015.
5. Gulcehre, Caglar, Firat, Orhan, Xu, Kelvin, Cho, Kyunghyun, Barrault, Loic, Lin, Huei-Chi, Bougares, Fethi, Schwenk, Holger, and Bengio, Yoshua. On using monolingual corpora in neural machine translation. arXiv preprint arXiv:1503.03535, 2015.
6. Jeavons, Peter, Cohen, David A., and Shawe-Taylor, John. Generating binary sequences for stochastic computing. Information Theory, IEEE Transactions on, 40(3):716–720, 1994.
7. Kim, Minje and Paris, Smaragdis. Bitwise neural networks. In Proceedings of The 31st International Conference on Machine Learning, pp. 0–0, 2015.
8. Krizhevsky, Alex and Hinton, Geoffrey. Learning multiple layers of features from tiny images, 2009.
9. Krizhevsky, Alex, Sutskever, Ilya, and Hinton, Geoffrey E. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems, pp. 1097–1105, 2012.
10. Kwan, Hon Keung and Tang, CZ. Multiplierless multilayer feedforward neural network design suitable for continuous input-output mapping. Electronics Letters, 29(14):1259–1260, 1993.
11. Le, Quoc V. Building high-level features using large scale unsupervised learning. In Acoustics, Speech and Signal Processing (ICASSP), 2013 IEEE International Conference on, pp. 8595–8598. IEEE, 2013.
12. LeCun, Yann, Bottou, Léon, Bengio, Yoshua, and Haffner, Patrick. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278–2324, 1998.
13. Machado, Emerson Lopes, Miosso, Cristiano Jacques, von Borries, Ricardo, Coutinho, Murilo, Berger, Pedro de Azevedo, Marques, Thiago, and Jacobi, Ricardo Pezzuol. Computational cost reduction in learned transform classifications. arXiv preprint arXiv:1504.06779, 2015.
14. Marchesi, Michele, Orlandi, Gianni, Piazza, Francesco, and Uncini, Aurelio. Fast neural networks without multipliers. Neural Networks, IEEE Transactions on, 4(1):53–62, 1993.
15. Neelakantan, Arvind, Vilnis, Luke, Le, Quoc V, Sutskever, Ilya, Kaiser, Lukasz, Kurach, Karol, and Martens, James. Adding gradient noise improves learning for very deep networks. arXiv preprint arXiv:1511.06807, 2015.
16. Netzer, Yuval, Wang, Tao, Coates, Adam, Bissacco, Alessandro, Wu, Bo, and Ng, Andrew Y. Reading digits in natural images with unsupervised feature learning. In NIPS workshop on deep learning and unsupervised feature learning, pp. 5. Granada, Spain, 2011.
17. Simard, Patrice Y and Graf, Hans Peter. Backpropagation without multiplication. In Advances in Neural Information Processing Systems, pp. 232–239, 1994.
18. van Daalen, Max, Jeavons, Pete, Shawe-Taylor, John, and Cohen, Dave. Device for generating binary sequences for stochastic computing. Electronics Letters, 29(1):80–81, 1993.