Accelerating deep learning with shrinkage and recall

Proceedings of the International Conference on Parallel and Distributed Systems - ICPADS

Volumn 0, Issue , 2016, Pages 963-970

  Zheng, Shuai ^b   Vishnu, Abhinav ^a   Ding, Chris ^b  

^a PACIFIC NORTHWEST NATIONAL LABORATORY   (United States)

^b University of Texas at Arlington   (United States)

Author keywords

Convolution Neural Network (CNN); Deep Belief Network (DBN); Deep Learning; Deep Neural Network (DNN)

Indexed keywords

CONVOLUTION; MACHINE LEARNING; REGRESSION ANALYSIS; SHRINKAGE; SUPPORT VECTOR MACHINES;

CLASSIFICATION PERFORMANCE; CONVOLUTION NEURAL NETWORK; DEEP BELIEF NETWORK (DBN); MACHINE LEARNING MODELS; MULTIPLE LAYERS; SCREENING TECHNIQUES; SPEED UP; SUPPORT VECTOR MACHINES ALGORITHMS;

DEEP NEURAL NETWORKS;

EID: 85018500482     PISSN: 15219097     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICPADS.2016.0129     Document Type: Conference Paper

References (27)

1. G. E. Hinton and R. R. Salakhutdinov, "Reducing the dimensionality of data with neural networks, " Science, vol. 313, no. 5786, pp. 504-507, 2006.
2. Y. Bengio, "Learning deep architectures for ai, " Foundations and trends R in Machine Learning, vol. 2, no. 1, pp. 1-127, 2009.
3. A. Krizhevsky, I. Sutskever, and G. E. Hinton, "Imagenet classification with deep convolutional neural networks, " in Advances in neural information processing systems, 2012, pp. 1097-1105.
4. P. Vincent, H. Larochelle, I. Lajoie, Y. Bengio, and P.-A. Manzagol, "Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion, " The Journal of Machine Learning Research, vol. 11, pp. 3371-3408, 2010.
5. R. Salakhutdinov and G. E. Hinton, "Deep boltzmann machines, " in International Conference on Artificial Intelligence and Statistics, 2009, pp. 448-455.
6. G. E. Hinton, "Training products of experts by minimizing contrastive divergence, " Neural computation, vol. 14, no. 8, pp. 1771-1800, 2002.
7. L. Deng, "Three classes of deep learning architectures and their applications: a tutorial survey, " APSIPA transactions on signal and information processing, 2012.
8. Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, "Gradientbased learning applied to document recognition, " Proceedings of the IEEE, vol. 86, no. 11, pp. 2278-2324, 1998.
9. Y. LeCun and Y. Bengio, "Convolutional networks for images, speech, and time series, " The handbook of brain theory and neural networks, vol. 3361, no. 10, 1995.
10. T. Joachims, "Making large scale svm learning practical, " Universität Dortmund, Tech. Rep., 1999.
11. J. Narasimhan, A. Vishnu, L. Holder, and A. Hoisie, "Fast support vector machines using parallel adaptive shrinking on distributed systems, " arXiv preprint arXiv: 1406. 5161, 2014.
12. J. Wang, J. Zhou, P. Wonka, and J. Ye, "Lasso screening rules via dual polytope projection, " in Advances in Neural Information Processing Systems, 2013, pp. 1070-1078.
13. A. Bonnefoy, V. Emiya, L. Ralaivola, and R. Gribonval, "A dynamic screening principle for the lasso, " in Signal Processing Conference (EUSIPCO), 2014 Proceedings of the 22nd European. IEEE, 2014, pp. 6-10.
14. J. Manyika, M. Chui, B. Brown, J. Bughin, R. Dobbs, C. Roxburgh, and A. H. Byers, "Big data: The next frontier for innovation, competition, and productivity, " 2011.
15. S. Zheng, X. Cai, C. H. Ding, F. Nie, and H. Huang, "A closed form solution to multi-view low-rank regression. " in AAAI, 2015, pp. 1973-1979.
16. S. Zheng and C. Ding, "Kernel alignment inspired linear discriminant analysis, " in Joint European Conference on Machine Learning and Knowledge Discovery in Databases. Springer Berlin Heidelberg, 2014, pp. 401-416.
17. D. Williams, S. Zheng, X. Zhang, and H. Jamjoom, "Tidewatch: Fingerprinting the cyclicality of big data workloads, " in IEEE INFOCOM 2014-IEEE Conference on Computer Communications. IEEE, 2014, pp. 2031-2039.
18. X. Zhang, Z.-Y. Shae, S. Zheng, and H. Jamjoom, "Virtual machine migration in an over-committed cloud, " in 2012 IEEE Network Operations and Management Symposium. IEEE, 2012, pp. 196-203.
19. S. Zheng, Z.-Y. Shae, X. Zhang, H. Jamjoom, and L. Fong, "Analysis and modeling of social influence in high performance computing workloads, " in European Conference on Parallel Processing. Springer Berlin Heidelberg, 2011, pp. 193-204.
20. J. A. Suykens and J. Vandewalle, "Least squares support vector machine classifiers, " Neural processing letters, vol. 9, no. 3, pp. 293-300, 1999.
21. R. Tibshirani, "Regression shrinkage and selection via the lasso, " Journal of the Royal Statistical Society. Series B (Methodological), pp. 267-288, 1996.
22. E. S. Gardner, "Exponential smoothing: The state of the art, " Journal of forecasting, vol. 4, no. 1, pp. 1-28, 1985.
23. R. B. Palm, "Prediction as a candidate for learning deep hierarchical models of data, " Technical University of Denmark, 2012.
24. A. Vedaldi and K. Lenc, "Matconvnet-convolutional neural networks for matlab, " arXiv preprint arXiv: 1412. 4564, 2014.
25. A. Krizhevsky and G. Hinton, "Learning multiple layers of features from tiny images, " 2009.
26. P. Baldi, P. Sadowski, and D. Whiteson, "Searching for exotic particles in high-energy physics with deep learning, " Nature communications, vol. 5, 2014.
27. H. Y. Xiong, Y. Barash, and B. J. Frey, "Bayesian prediction of tissue-regulated splicing using rna sequence and cellular context, " Bioinformatics, vol. 27, no. 18, pp. 2554-2562, 2011.