Learning non-parametric basis independent models from point queries via low-rank methods

Applied and Computational Harmonic Analysis

Volumn 37, Issue 3, 2014, Pages 389-412

  Tyagi, Hemant ^a   Cevher, Volkan ^b  

^a ETH ZURICH   (Switzerland)

^b EPFL   (Switzerland)

Author keywords

High dimensional function approximation; Low rank matrix recovery; Multi ridge functions; Non linear approximation; Oracle based learning

Indexed keywords

HARMONIC ANALYSIS; MATHEMATICAL TECHNIQUES;

HIGH DIMENSIONAL FUNCTION APPROXIMATIONS; LOW-RANK MATRIX RECOVERIES; NONLINEAR APPROXIMATION; ORACLE-BASED LEARNING; RIDGE FUNCTIONS;

POLYNOMIAL APPROXIMATION;

EID: 84908556298     PISSN: 10635203     EISSN: 1096603X     Source Type: Journal    
DOI: 10.1016/j.acha.2014.01.002     Document Type: Article

References (41)

1. C.M. Bishop, Neural Networks for Pattern Recognition, Oxford University Press, USA, 1995.
2. E.J. Candès, Harmonic analysis of neural networks, Appl. Comput. Harmon. Anal. 6 (2) (1999) 197-218.
3. E.J. Candès, Ridgelets: Estimating with ridge functions, Ann. Statist. 31 (5) (2003) 1561-1599.
4. E.J. Candès, D.L. Donoho, Ridgelets: a key to higher dimensional intermittency? Philos. Trans. R. Soc. Lond. Ser. A Math. Phys. Eng. Sci. 357 (1760) (1999) 2495-2509.
5. E.J. Candès, X. Li, Y. Ma, J. Wright, Robust principal component analysis? J. ACM 58 (3) (2011) 1-37.
6. E.J. Candès, Y. Plan, Tight oracle bounds for low-rank matrix recovery from a minimal number of random measurements, CoRR abs/1001.0339, 2010.
7. E.J. Candès, B. Recht, Exact matrix completion via convex optimization, Found. Comput. Math. 9 (6) (2009) 717-772.
8. E.J. Candès, T. Tao, The power of convex relaxation: near-optimal matrix completion, IEEE Trans. Inform. Theory 56 (May 2010) 2053-2080.
9. A. Cohen, I. Daubechies, R.A. DeVore, G. Kerkyacharian, D. Picard, Capturing ridge functions in high dimensions from point queries, Constr. Approx. 35 (2) (2012) 225-243.
10. R. DeVore, G.G. Lorentz, Constructive Approximation, 1993.
11. D.L. Donoho, I.M. Johnstone, Projection based regression and a duality with kernel methods, Ann. Statist. 17 (1989) 58-106.
12. M. Fornasier, K. Schnass, J. Vybíral, Learning functions of few arbitrary linear parameters in high dimensions, Found. Comput. Math. 12 (2) (2012) 229-262.
13. J.H. Friedman, W. Stuetzel, Projection pursuit regression, J. Amer. Statist. Assoc. 76 (1981) 817-823.
14. P. Hall, K.C. Li, On almost linearity of low dimensional projections from high dimensional data, Ann. Statist. (1993) 867-889.
15. W. Hardle, Applied Nonparametric Regression, vol. 26, Cambridge Univ. Press, 1990.
16. P.J. Huber, Projection pursuit, Ann. Statist. 13 (1985) 435-475.
17. V. Koltchinskii, M. Yuan, Sparsity in multiple kernel learning, Ann. Statist. 38 (6) (2010) 3660-3695.
18. B. Laurent, P. Massart, Adaptive estimation of a quadratic functional by model selection, Ann. Statist. 28 (5) (2000) 1302-1338.
19. K. Lee, Y. Bresler, ADMiRA: Atomic decomposition for minimum rank approximation, IEEE Trans. Inform. Theory 56 (9) (2010) 4402-4416.
20. K.C. Li, Sliced inverse regression for dimension reduction, J. Amer. Statist. Assoc. (1991) 316-327.
21. Q. Li, J. Racine, Nonparametric Econometrics: Theory and Practice, Princeton University Press, Princeton, NJ, 2007.
22. Y. Lin, H.H. Zhang, Component selection and smoothing in multivariate nonparametric regression, Ann. Statist. 34 (5) (2006) 2272-2297.
23. B.F. Logan, L.A. Shepp, Optimal reconstruction of a function from its projections, Duke Math. J. 42 (1975) 645-659.
24. L. Meier, S. Van De Geer, P. Bühlmann, High-dimensional additive modeling, Ann. Statist. 37 (6B) (2009) 3779-3821.
25. R. Meka, P. Jain, I.S. Dhillon, Guaranteed rank minimization via singular value projection, CoRR abs/0909.5457, 2009.
26. Th. Muller-Gronbach, K. Ritter, Minimal errors for strong and weak approximation of stochastic differential equations, in: Monte Carlo and Quasi-Monte Carlo Methods, 2008, pp. 53-82.
27. E. Novak, H. Wozniakowski, Approximation of infinitely differentiable multivariate functions is intractable, J. Complexity 25 (August 2009) 398-404.
28. A. Pinkus, Approximation theory of the MLP model in neural networks, Acta Numer. 8 (1999) 143-195.
29. G. Raskutti, M.J. Wainwright, B. Yu, Minimax-optimal rates for sparse additive models over kernel classes via convex programming, Technical report, 2010.
30. P. Ravikumar, J. Lafferty, H. Liu, L. Wasserman, Sparse additive models, J. R. Stat. Soc. Ser. B Stat. Methodol. 71 (5) (2009) 1009-1030.
31. B. Recht, M. Fazel, P.A. Parrilo, Guaranteed minimum-rank solutions of linear matrix equations via nuclear norm minimization, SIAM Rev. 52 (2010) 471-501.
32. J. Rohn, A handbook of results on interval linear problems, Technical report, Czech Academy of Sciences, Prague, Czech Republic, 2005.
33. n, Springer Verlag, New York/Berlin, 1980.
34. J.F. Traub, G.W. Wasilkowski, H. Wozniakowski, Information-Based Complexity, Academic Press, New York, 1988.
35. J.A. Tropp, User friendly tail bounds for matrix martingales. ArXiv e-prints, 2010.
36. Andrew E. Waters, Aswin C. Sankaranarayanan, Richard G. Baraniuk, SPARCS: Recovering low-rank and sparse matrices from compressive measurements, in: Neural Information Processing Systems (NIPS), 2011.
37. P.A. Wedin, Perturbation bounds in connection with singular value decomposition, BIT 12 (1972) 99-111.
38. H. Weyl, Das asymptotische verteilungsgesetz der eigenwerte linearer partieller differentialgleichungen (mit einer anwendung auf die theorie der hohlraumstrahlung), Math. Ann. 71 (1912) 441-479.
39. W. So, Rank one perturbation and its application to the Laplacian spectrum of a graph, Linear Multilinear Algebra 46 (1999) 193-198.
40. Y. Xia, A multiple-index model and dimension reduction, J. Amer. Statist. Assoc. 103 (484) (2008) 1631-1640.
41. Y. Xia, H. Tong, W.K. Li, L.X. Zhu, An adaptive estimation of dimension reduction space, J. R. Stat. Soc. Ser. B Stat. Methodol. 64 (3) (2002) 363-410.