Quasi-Monte Carlo feature maps for shift-invariant Kernels

Journal of Machine Learning Research

Volumn 17, Issue , 2016, Pages 1-38

  Avron, Haim ^a   Sindhwani, Vikas ^b   Yang, Jiyan ^c   Mahoney, Michael W ^d  

^a TEL AVIV UNIVERSITY   (Israel)

^b GOOGLE INC   (United States)

^c Stanford University   (United States)

^d UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE; SOFTWARE ENGINEERING;

APPROXIMATE FEATURES; DISCREPANCY MEASURES; INTEGRAL REPRESENTATION; INTEGRATION ERROR; LOW-DISCREPANCY SEQUENCES; MONTE CARLO APPROACH; MONTE-CARLO APPROXIMATIONS; TRAINING AND TESTING;

MONTE CARLO METHODS;

EID: 84989191960     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (55)

1. H. Avron, H. Nguyen, and D. Woodruff. Subspace embeddings for the polynomial kernel. In Advances in Neural Information Processing Systems (NIPS). 2014.
2. F. Bach. Sharp analysis of low-rank kernel matrix approximations. In Conference on Learning Theory (COLT), 2013.
3. F. Bach, S. Lacoste-Julien, and G. Obozinski. On the equivalence between herding and conditional gradient algorithms. In International Conference in Machine Learning (ICML), 2012.
4. M. Belkin, P. Niyogi, and V. Sindhwani. Manifold regularization: A geometric framework for learning from labeled and unlabeled examples. J. Mach. Learn. Res., 7:2399-2434, 2006.
5. A. Berlinet and C. Thomas-Agnan. Reproducing Kernel Hilbert Spaces in Probability and Statistics. Kluwer Academic Publishers, 2004.
6. S. Bochner. Monotone funktionen, Stieltjes integrale und harmonische analyse. Math. Ann., 108:378-410, 1933.
7. B. Boots, A. Gretton, and G. J. Gordon. Hilbert space embeddings of predictive state representations. In Conference Uncertainty in Artificial Intelligence (UAI), 2013.
8. L. Bottou, O. Chapelle, D. DeCoste, and J. Weston (Editors). Large-scale Kernel Machines. MIT Press, 2007.
9. R. E. Caflisch. Monte Carlo and Quasi-Monte Carlo methods. Acta Numerica, 7:1-49, 1 1998.
10. Y. Chen, M. Welling, and A. Smola. Super-samples from kernel herding. In Conference on Uncertainty in Artificial Intelligence (UAI), 2010.
11. F. Cucker and S. Smale. On the mathematical foundations of learning. Bull. Amer. Math. Soc., 39:1-49, 2001.
12. B. Dai, B. Xie, N. He, Y. Liang, A. Raj, M.-F. Balcan, and L. Song. Scalable kernel methods via doubly stochastic gradients. In Advances in Neural Information Processing Systems (NIPS). 2014.
13. J. Dick, F. Y. Kuo, and I. H. Sloan. High-dimensional integration: The Quasi-Monte Carlo way. Acta Numerica, 22:133-288, 2013.
14. A. El Alaoui and M. W. Mahoney. Fast Randomized Kernel Methods With Statistical Guarantees. ArXiv e-prints, November 2014.
15. Z. Ghahramani and C. E. Rasmussen. Bayesian Monte Carlo. In Advances in Neural Information Processing Systems (NIPS). 2003.
16. A. Gittens and M. W. Mahoney. Revisiting the Nyström method for improved large-scale machine learning. In International Conference on Machine Learning (ICML), 2013. To appear in the Journal of Machine Learning Research.
17. M. Grant and S. Boyd. CVX: Matlab software for disciplined convex programming, version 2.1. http://cvxr.com/cvx, March 2014.
18. M. Grant and S. Boyd. Graph implementations for nonsmooth convex programs. In V. Blondel, S. Boyd, and H. Kimura, editors, Recent Advances in Learning and Control, Lecture Notes in Control and Information Sciences, pages 95-110. Springer-Verlag Limited, 2008.
19. R. Hamid, A. Gittens, Y. Xiao, and D. DeCoste. Compact random feature maps. In International Conference on Machine Learning (ICML), 2014.
20. Z. Harchaoui, F. Bach, O. Cappe, and E. Moulines. Kernel-based methods for hypothesis testing: A unified view. IEEE Signal Processing Magazine, 30(4):87-97, July 2013.
21. P. Huang, H. Avron, T. Sainath, V. Sindhwani, and B. Ramabhadran. Kernel methods match Deep Neural Networks on TIMIT. In International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2014.
22. F. Huszár and D. Duvenaud. Optimally-weighted herding is Bayesian quadrature. In Conference on Uncertainty in Artificial Intelligence (UAI), 2012.
23. P. Kar and H. Karnick. Random feature maps for dot product kernels. In International Conference on Artificial Intelligence and Statistics (AISTATS), 2012.
24. S. S. Keerthi, O. Chapelle, and D. DeCoste. Building support vector machines with reduced classifier complexity. J. Mach. Learn. Res., 7:1493-1515, 2006.
25. Q. Le, T. Sarlós, and A. Smola. Fastfood - Approximating kernel expansions in loglinear time. In International Conference on Machine Learning (ICML), 2013.
26. G. Leobacher and F. Pillichschammer. Introduction to Quasi-Monte Carlo Integration and Applications. Springer International Publishing, 2014.
27. F. Li, C. Ionescu, and C. Sminchisescu. Random Fourier approximations for skewed multiplicative histogram kernels. Pattern Recognition, 6376:262-271, 2010.
28. Z. Lu, A. May, K. Liu, A. Bagheri Garakani, D. Guo, A. Bellet, L. Fan, M. Collins, B. Kingsbury, M. Picheny, and F. Sha. How to scale up kernel methods to be as good as deep neural net. ArXiv e-prints, November 2014.
29. S. Maji and A. C. Berg. Max-margin additive classifiers for detection. In International Conference on Computer Vision (ICCV), 2009.
30. M. Mori. A method for evaluation of the error function of real and complex variable with high relative accuracy. Publ. RIMS, Kyoto Univ., 19:1081-1094, 1983.
31. H. Niederreiter. Random number generation and Quasi-Monte Carlo methods. Society for Industrial and Applied Mathematics, Philadelphia, PA, USA, 1992.
32. E. Parzen. Statistical inference on time series by RKHS methods. In Biennial Seminar Canadian Mathematical Congress on Time Series and Stochastic Processes: convexity and combinatorics, 1970.
33. M. M. Peloso. Classical spaces of holomorphic functions. Technical report, Universit' di Milano, 2011.
34. N. Pham and R. Pagh. Fast and scalable polynomial kernels via explicit feature maps. In International Conference on Knowledge Discovery and Data Mining (KDD), 2013.
35. A. Rahimi and B. Recht. Random features for large-scale kernel machines. In Advances in Neural Information Processing Systems (NIPS). 2008.
36. V. C. Raykar and R. Duraiswami. Fast large scale gaussian process regression using approximate matrix-vector products, 2007.
37. I. J. Schoenberg. Positive definite functions on spheres. Duke Mathematical Journal, 9(1):96-108, 03 1942.
38. B. Schölkopf and A. Smola, editors. Learning with Kernels: Support Vector Machines, Regularization, Optimization and Beyond. MIT Press, 2002.
39. V. Sindhwani and H. Avron. High-performance kernel machines with implicit distributed optimization and randomization. In JSM Proceedings, Tradeoffs in Big Data Modeling - Section on Statistical Computing, 2014. To appear in Technometrics.
40. I. H. Sloan and H. Wozniakowski. When are Quasi-Monte Carlo algorithms efficient for high dimensional integrals. Journal of Complexity, 14(1):1-33, 1998.
41. A. Smola, A. Gretton, L. Song, and B. Schlkopf. A Hilbert space embedding for distributions. In Algorithmic Learning Theory, volume 4754 of Lecture Notes in Computer Science, pages 13-31. Springer Berlin Heidelberg, 2007. ISBN 978-3-540-75224-0.
42. L. Song, B. Boots, S. Siddiqi, G. Gordon, and A. Smola. Hilbert space embeddings of Hidden Markov Models. In International Conference in Machine Learning (ICML), 2010.
43. V. Sreekanth, A. Vedaldi, C. V. Jawahar, and A. Zisserman. Generalized RBF feature maps for efficient detection. In British Machine Vision Conference (BMVC), 2010.
44. B. Sriperumbudur, A. Gretton, K. Fukumizu, B. Scholkopf, and G. Lanckriet. Hilbert space embeddings and metrics on probability measures. J. Mach. Learn. Res., 11:1517-1561, 2010.
45. J. F. Traub and H. Wozniakowski. Breaking intractability. Scientific American, pages 102-107, 1994.
46. I. W. Tsang, J. T. Kwok, and P. Cheung. Core vector machines: Fast svm training on very large data sets. J. Mach. Learn. Res., 6:363-392, December 2005.
47. A. Vedaldi and A. Zisserman. Efficient additive kernels via explicit feature maps. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(3):480-492, March 2012.
48. G. Wahba, editor. Spline Models for Observational Data. Society for Industrial and Applied Mathematics, Philadelphia, PA, USA, 1990.
49. J. A. C. Weideman. Computation of the complex error function. SIAM Journal of Numerical Analysis, 31(5):1497-1518, 10 1994.
50. M. Welling. Herding dynamical weights to learn. In International Conference on Machine Learning (ICML), 2009.
51. C. K. I. Williams and M. Seeger. Using the Nyström method to speed up kernel machines. In Advances in Neural Information Processing Systems (NIPS). 2001.
52. H. Wozniakowski. Average case complexity of multivariate integration. Bull. Amer. Math. Soc., 24:185-194, 1991.
53. J. Yang, V. Sindhwani, Q. Fan, H. Avron, and M. Mahoney. Random Laplace feature maps for semigroup kernels on histograms. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014.
54. K. Yao. Applications of Reproducing Kernel Hilbert Spaces - bandlimited signal models. Inform. Control, 11:429-444, 1967.
55. K. Zhang, J. Peters, D. Janzing, and B. Scholkopf. Kernel based conditional independence test and application in causal discovery. In Confernece on Uncertainty in Artificial Intelligence (UAI), 2011.