Large-scale deep unsupervised learning using graphics processors

Proceedings of the 26th International Conference On Machine Learning, ICML 2009

Volumn , Issue , 2009, Pages 873-880

  Raina, Rajat ^a   Madhavan, Anand ^a   Ng, Andrew Y ^a  

^a Stanford University   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

BELIEF NETWORKS; COMPUTATIONAL CAPABILITY; DUAL-CORE; FREE PARAMETERS; GRAPHICS PROCESSOR; HIGHLY NONLINEAR; LARGE-SCALE APPLICATIONS; MACHINE LEARNING APPLICATIONS; MULTI CORE; PARALLEL METHOD; PARALLELIZING; SCALE MODELS; SCALING-UP; SPARSE CODING; TRAINING EXAMPLE; UNLABELED DATA; UNSUPERVISED LEARNING METHOD;

EDUCATION; PARALLEL ALGORITHMS; PROGRAM PROCESSORS; ROBOT LEARNING; UNSUPERVISED LEARNING;

LEARNING ALGORITHMS;

EID: 71149105669     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (33)

1. 1-regularized log-linear models. International Conference on Machine Learning (pp. 33-40).
2. Banko, M., & Brill, E. (2001). Scaling to very very large corpora for natural language disambiguation. Annual Meeting of the Association for Computational Linguistics (pp. 26-33).
3. Bengio, Y. (2007). Speeding up stochastic gradient descent. Neural Information Processing Systems Workshop on Efficient Machine Learning.
4. Bengio, Y., Lamblin, P., Popovici, D., & Larochelle, H. (2006). Greedy layer-wise training of deep networks. Neural Information Processing Systems (pp. 153-160).
5. Bradley, D., & Bagnell, J. A. (2008). Differentiable sparse coding. Neural Information Processing Systems (pp. 113-120).
6. Brants, T., Popat, A. C., Xu, P., Och, F. J., & Dean, J. (2007). Large language models in machine translation. Conference on Empirical Methods in Natural Language Processing (EMNLP-CoNLL).
7. Catanzaro, B. C., Sundaram, N., & Keutzer, K. (2008). Fast support vector machine training and classification on graphics processors. International Conference on Machine Learning (pp. 104-111).
8. Chellapilla, K., Puri, S., & Simard, P. (2006). High performance convolutional neural networks for document processing. International Workshop on Frontiers in Handwriting Recognition.
9. Chu, C. T., Kim, S. K., Lin, Y. A., Yu, Y., Bradski, G. R., Ng, A. Y., & Olukotun, K. (2006). Map-reduce for machine learning on multicore. Neural Information Processing Systems (pp. 281-288).
10. Dean, J., & Ghemawat, S. (2004). Mapreduce: Simplified data processing on large clusters. Operating System Design and Implementation (pp. 137-150).
11. Desjardins, G., & Bengio, Y. (2008). Empirical evaluation of convolutional RBMs for vision. Tech Report.
12. Efron, B., Hastie, T., Johnstone, I., & Tibshirani, R. (2004). Least angle regression. Ann. Stat., 32, 407.
13. Frank, D. (2002). Power-constrained CMOS scaling limits. IBM Jour. of Res. and Devel., 46, 235-244.
14. Friedman, J., Hastie, T., Hfling, H., & Tibshirani, R. (2007). Pathwise coordinate optimization. Ann. App. Stat., 2, 302-332.
15. Gelsinger, P. (2001). Microprocessors for the new millennium: Challenges, opportunities and new frontiers. ISSCC Tech. Digest, 22-25.
16. Goto, K., & Van De Geijn, R. (2008). High-performance implementation of the level-3 BLAS. ACM Trans. Math. Softw., 35, 1-14.
17. Harris, M. (2008). Many-core GPU computing with NVIDIA CUDA. Int. Conf. Supercomputing (p. 1).
18. Hinton, G. E. (2002). Training products of experts by minimizing contrastive divergence. Neural Computation, 14, 1771-1800.
19. Hinton, G. E., Osindero, S., & Teh, Y.-W. (2006). A fast learning algorithm for deep belief nets. Neural Computation, 18, 1527-1554.
20. Hinton, G. E., & Salakhutdinov, R. R. (2006). Reducing the dimensionality of data with neural networks. Science, 313, 504-507.
21. Kavukcuoglu, K., Ranzato, M., & LeCun, Y. (2008). Fast inference in sparse coding algorithms with applications to object recognition. NYU Tech Report.
22. Lee, H., Battle, A., Raina, R., & Ng, A. Y. (2006). Efficient sparse coding algorithms. Neural Information Processing Systems (pp. 801-808).
23. Lee, H., Chaitanya, E., & Ng, A. Y. (2007). Sparse deep belief net model for visual area V2. Neural Information Processing Systems (pp. 873-880).
24. Lee, H., Grosse, R., Ranganath, R., & Ng, A. Y. (2009). Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations. International Conference on Machine Learning (to appear).
25. Murray, J. F., & Kreutz-Delgado, K. (2006). Learning sparse overcomplete codes for images. J. VLSI Signal Processing Systems, 45, 97-110.
26. 2 regu-larization, and rotational invariance. International Conference on Machine Learning (pp. 78-85).
27. Olshausen, B. A., & Field, D. J. (1996). Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature, 381, 607-609.
28. Raina, R., Battle, A., Lee, H., Packer, B., & Ng, A. Y. (2007). Self-taught learning: Transfer learning from unlabeled data. International Conference on Machine Learning (pp. 759-766).
29. Ranzato, M. A., & Szummer, M. (2008). Semi-supervised learning of compact document representations with deep networks. International Conference on Machine Learning (pp. 792-799).
30. Salakhutdinov, R., & Hinton, G. (2007). Semantic Hashing. SIGIR Workshop on Information Retrieval and Applications of Graphical Models.
31. Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. J. R. Stat. Soc. B., 58, 267-288.
32. van Hateren, J. H., & van der Schaaff, A. (1997). Independent component filters of natural images compared with simple cells in primary visual cortex. Royal Soc. Lond. B, 265, 359-366.
33. Whaley, R. C., Petitet, A., & Dongarra, J. J. (2001). Automated empirical optimization of software and the ATLAS project. Parallel Computing, 27, 3-35.