Near-optimal design of experiments via regret minimization

34th International Conference on Machine Learning, ICML 2017

Volumn 1, Issue , 2017, Pages 169-178

  Allen Zhu, Zeyuan ^a   Li, Yuanzhi ^b   Singh, Aarti ^c   Wang, Yining ^c  

^a MICROSOFT RESEARCH   (United States)

^b PRINCETON UNIVERSITY   (United States)

^c CARNEGIE MELLON UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE; LEARNING SYSTEMS; OPTIMAL SYSTEMS;

APPROXIMATE OPTIMAL DESIGNS; LINEAR FUNCTIONS; N-DESIGN; NUMERICAL RESULTS; OPTIMALITY CRITERIA; REAL-WORLD DESIGNS; REGRET MINIMIZATION;

DESIGN OF EXPERIMENTS;

EID: 85048685137     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (39)

1. Ageev, Alexander A and Sviridenko, Maxim I. Pipage rounding: A new method of constructing algorithms with proven performance guarantee. Journal of Combinatorial Optimization, 8(3):307-328, 2004.
2. Allen-Zhu, Zeyuan, Liao, Zhenyu, and Orecchia, Lorenzo. Spectral sparsification and regret minimization beyond matrix multiplicative updates. In Proceedings of Annual Symposium on the Theory of Computing (STOC), 2015.
3. Avron, Haim and Boutsidis, Christos. Faster subset selection for matrices and applications. SIAM Journal on Matrix Analysis and Applications, 34(4):1464-1499, 2013.
4. Balean, Maria-Florina and Long, Philip M. Active and passive learning of linear separators under log-concave distributions. In Proceedings of Annual Conference on Learning Theory (COLT), 2013.
5. Bouhtou, Mustapha, Gaubert, Stephane, and Sagnol, Guillaume. Submodularity and randomized rounding techniques for optimal experimental design. Electronic Notes in Discrete Mathematics, 36:679-686, 2010.
6. Boutsidis, Christos and Woodruff, David P. Optimal CUR matrix decompositions. In Proceedings of Annual Symposium on the Theory of Computing (STOC), 2014.
7. Cerny, Michal and Hladik, Milan. Two complexity results on C-optimality in experimental design. Computational Optimization and Applications, 51(3):1397-1408, 2012.
8. Chaudhuri, Kamalika, Kakade, Sham, Netrapalli, Praneeth, and Sanghavi, Sujay. Convergence rates of active learning for maximum likelihood estimation. In Proceedings of Advances in Neural Information Processing Systems (NIPS), 2015.
9. Chen, Siheng, Varma, Rohan, Singh, Aarti, and Kovacevié, Jelena. Signal representations on graphs: Tools and applications. arXiv preprint arXiv:1512.05406, 2015.
10. Çivril, AH and Magdon-Ismail, Malik. On selecting a maximum volume sub-matrix of a matrix and related problems. Theoretical Computer Science, 410(47-49):4801-4811, 2009.
11. Deshpande, Amit and Rademacher, Luis. Efficient volume sampling for row/column subset selection. In Proceedings of Annual Conference on Foundations of Computer Science (FOCS), 2010.
12. Deshpande, Yash and Montanari, Andrea. Linear bandits in high dimension and recommendation systems. In Proceedings of Annual Allerton Conference on Communication, Control, and Computing (Allerton), 2012.
13. Dhillon, Paramveer, Lu, Yichao, Foster, Dean P, and Ungar, Lyle. New subsampling. algorithms for fast least squares regression. In Proceedings of Advances in Neural Information Processing Systems (NIPS), 2013.
14. Drineas, Petros and Mahoney, Michael W. On the Nyström method for approximating a gram matrix for improved kernel-based learning. Journal of Machine Learning Research, 6(12):2153-2175, 2005.
15. Drineas, Petros, Mahoney, Michael W, and Muthukrishnan, S. Relative-error CUR matrix decompositions. SIAM Journal on Matrix Analysis and Applications, 30(2):844-881, 2008.
16. Fedorov, Valerii Vadimovich. Theory of optimal experiments. Elsevier, 1972.
17. Hazan, Elad and Karnin, Zohar. Hard-margin active linear regression. In Proceedings of International Conference on Machine Learning (ICML), 2015.
18. Horel, Thibaut, Ioannidis, Stratis, and Muthukrishnan, S. Budget feasible mechanisms for experimental design. In Proceedings of Latin American Symposium on Theoretical Informatics (LATIN), 2014.
19. Huang, Ruitong, Lattimore, Tor, György, András, and Szepesvári, Csaba. Following the leader and fast rates in linear prediction: Curved constraint sets and other regularities. In Proceedings of Advances in Neural Information Processing Systems (NIPS), 2016.
20. Khuri, Andre, Mukherjee, Bhramar, Sinha, Bikas, and Ghosh, Malay. Design issues for generalized linear models: a review. Statistical Science, 21(3):376-399, 2006.
21. Kulesza, Alex and Taskar, Ben. Determinantal point processes for machine learning. Foundations and Trends® in Machine Learning, 5(2-3):123-286, 2012.
22. Li, Chengtao, Jegelka, Stefanie, and Sra, Suvrit. Polynomial time algorithms for dual volume sampling. arXiv preprint arXiv:1703.02674, 2017.
23. Ma, Yifei, Gamett, Roman, and Schneider, Jeff. ω-optimality for active learning on Gaussian random fields. In Proceedings of Advances in Neural Information Processing Systems (NIPS), 2013.
24. McMahan, H Brendan. Follow-the-regularized-leader and mirror descent: Equivalence theorems and LI régularisation. In Procedings of International Conference on Artificial Intelligence and Statistics (A1STATS), 2011.
25. Miller, Alan and Nguyen, Nam-Ky. A Fedorov exchange algorithm for d-optimal design. Journal of the Royal Statistical Society, Series C (Applied Statistics), 43(4):669-677, 1994.
26. Pilanci, Mert and Wainwright, Martin J. Iterative Hessian sketch: Fast and accurate solution approximation for constrained least-squares. Journal of Machine Learning Research, 17(53):1-38, 2016.
27. Pukelsheim, Friedrich. Optimal design of experiments. SIAM, 2006.
28. Raskutti, Garvesh and Mahoney, Michael. A statistical perspective on randomized sketching for ordinary leastsquares. arXiv preprint arXiv:1406.5986, 2014.
29. Ravi, Sathya N, Ithapu, Vamsi K, Johnson, Sterling C, and Singh, Vikas. Experimental design on a budget for sparse linear models and applications. In Proceedings of International Conference on Machine Learning (ICML), 2016.
30. Silva, Marcel K, Harvey, Nicholas JA, and Sato, Cristiane M. Sparse sums of positive semidefinite matrices. ACM Transactions on Algorithms, 12(1):9, 2016.
31. Spielman, Daniel A and Srivastava, Nikhil. Graph sparsification by effective resistances. SIAM Journal on Computing, 40(6):1913-1926, 2011.
32. Su, Hao, Yu, Adams Wei, and Li, Fei-Fei. Efficient euclidean projections onto the intersection of norm balls. In Proceedings of International Conference on Machine Learning (ICML), 2012.
33. Van der Vaart, Aad W. Asymptotic statistics, volume 3. Cambridge university press, 2000.
34. Wang, Shusen and Zhang, Zhihua. Improving CUR matrix decomposition and the Nyström approximation via adaptive sampling. Journal of Machine Learning Research, 14(1):2729-2769, 2013.
35. Wang, Yining and Singh, Aarti. An empirical comparison of sampling techniques for matrix column subset selection. In Proceedings of Annual Allerton Conference on Communication, Control, and Computing (Allerton), 2015a.
36. Wang, Yining and Singh, Aarti. Provably correct active sampling algorithms for matrix column subset selection with missing data. arXiv preprint arXiv:1505.04343, 2015b.
37. Wang, Yining and Singh, Aarti. Noise-adaptive marginbased active learning and lower bounds under tsybakov noise condition. In Proceedings of AAAI Conference on Artificial Intelligence (AAAI), 2016.
38. Wang, Yining, Yu, Wei Adams, and Singh, Aarti. On computationally tractable selection of experiments in regression models. arXiv preprints: arXiv:1601.02068, 2016.
39. Woodruff, David P. Sketching as a tool for numerical linear algebra. Foundations and Trends® in Theoretical Computer Science, 10(1-2):1-157, 2014.