Sequential information maximization: When is greedy near-optimal?

Journal of Machine Learning Research

Volumn 40, Issue 2015, 2015, Pages

  Chen, Yuxin ^a,b   Hassani, S Hamed ^a,b   Karbasi, Amin ^a,b   Krause, Andreas ^a,b  

^a ETH ZURICH   (Switzerland)

^b YALE UNIVERSITY   (United States)

Author keywords

Active learning; Information theory; Optimization

Indexed keywords

BAYESIAN NETWORKS; INFORMATION THEORY; LEARNING SYSTEMS; OPTIMIZATION;

ACTIVE LEARNING; AUTOMATED DIAGNOSIS; BAYESIAN EXPERIMENTAL DESIGNS; INFORMATION GATHERING; MUTUAL INFORMATIONS; NOISY OBSERVATIONS; PROBABILISTIC MODELING; SEQUENTIAL INFORMATION;

ARTIFICIAL INTELLIGENCE;

EID: 84984698544     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Conference Paper

References (38)

1. Maria-Florina Balcan and Ruth Urner. Active learning-modern learning theory. 2015.
2. Maria-Florina Balcan, Alina Beygelzimer, and John Langford. Agnostic active learning. In Proceedings of the 23rd international conference on Machine learning, pages 65-72. ACM, 2006.
3. Maria-Florina Balcan, Andrei Broder, and Tong Zhang. Margin based active learning. In Learning Theory, pages 35-50. Springer, 2007.
4. Scott M Berry, Bradley P Carlin, J Jack Lee, and Peter Muller. Bayesian adaptive methods for clinical trials. CRC press, 2010.
5. Marat Valievich Burnashev. Data transmission over a discrete channel with feedback. random transmission time. Problemy peredachi informatsii, 12 (4): 10-30, 1976.
6. Venkatesan T Chakaravarthy, Vinayaka Pandit, Sambuddha Roy, Pranjal Awasthi, and Mukesh Mohania. Decision trees for entity identification: Approximation algorithms and hardness results. In Proceedings of the twenty-sixth ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems, pages 53-62. ACM, 2007.
7. Yuxin Chen, Shervin Javdani, Amin Karbasi, James Andrew Bagnell, Siddhartha Srinivasa, and Andreas Krause. Submodular surrogates for value of information. In Proc. Conference on Artificial Intelligence (AAAI), 2015.
8. Herman Chernoff. Sequential design of experiments. The Annals of Mathematical Statistics, pages 755-770, 1959.
9. Imre Csiszar and János Körner. Information theory: coding theorems for discrete memoryless systems. Cambridge University Press, 2011.
10. Sanjoy Dasgupta. Analysis of a greedy active learning strategy. In Advances in Neural Information Processing Systems 17. 2005.
11. Sanjoy Dasgupta and J Langford. Active learning. Encyclopedia of Machine Learning, 2011.
12. Amol Deshpande, Lisa Hellerstein, and Devorah Kletenik. Approximation algorithms for stochastic boolean function evaluation and stochastic submodular set cover. In Proceedings of the Twenty-Fifth Annual ACM-SIAM Symposium on Discrete Algorithms, pages 1453-1467. SIAM, 2014.
13. Valerii Vadimovich Fedorov. Theory of optimal experiments. Elsevier, 1972.
14. Yoav Freund, H Sebastian Seung, Eli Shamir, and Naftali Tishby. Selective sampling using the query by committee algorithm. Machine learning, 28 (2-3): 133-168, 1997.
15. Daniel Golovin and Andreas Krause. Adaptive submodularity: Theory and applications in active learning and stochastic optimization. JAIR, 2011.
16. Daniel Golovin, Andreas Krause, and Debajyoti Ray. Near-optimal bayesian active learning with noisy observations. In Proc. Neural Information Processing Systems (NIPS), December 2010.
17. Alon Gonen, Sivan Sabato, and Shai Shalev-Shwartz. Efficient active learning of halfspaces: an aggressive approach. The Journal of Machine Learning Research, 14 (1): 2583-2615, 2013.
18. Steve Hanneke. A bound on the label complexity of agnostic active learning. In Proceedings of the 24th international conference on Machine learning, pages 353-360. ACM, 2007.
19. Steve Hanneke. Statistical Theory of Active Learning. Now Publishers Incorporated, 2014.
20. Steve Hanneke and Liu Yang. Minimax analysis of active learning. CoRR, abs/1410.0996, 2014.
21. Michael Horstein. Sequential transmission using noiseless feedback. Information Theory, IEEE Transactions on, 9 (3): 136-143, 1963.
22. Matti Kääriäinen. Active learning in the non-realizable case. In Algorithmic Learning Theory, pages 63-77. Springer, 2006.
23. Haim Kaplan, Eyal Kushilevitz, and Yishay Mansour. Learning with attribute costs. In Proceedings of the thirty-seventh annual ACM symposium on Theory of computing, pages 356-365. ACM, 2005.
24. Chun-Wa Ko, Jon Lee, and Maurice Queyranne. An exact algorithm for maximum entropy sampling. Operations Research, 43 (4): 684-691, 1995.
25. S Rao Kosaraju, Teresa M Przytycka, and Ryan Borgstrom. On an optimal split tree problem. In Algorithms and Data Structures, pages 157-168. Springer, 1999.
26. Andreas Krause and Carlos Guestrin. Near-optimal nonmyopic value of information in graphical models. In Conference on Uncertainty in Artificial Intelligence (UAI), July 2005.
27. Dennis V Lindley. On a measure of the information provided by an experiment. The Annals of Mathematical Statistics, pages 986-1005, 1956.
28. SP Luttrell. The use of transinformation in the design of data sampling schemes for inverse problems. Inverse Problems, 1 (3): 199, 1985.
29. David MacKay. Information-based objective functions for active data selection. Neural computation, 4 (4): 590-604, 1992.
30. Mohammad Naghshvar, Tara Javidi, and Kamalika Chaudhuri. Noisy bayesian active learning. CoRR, abs/1312.2315, 2013a.
31. Mohammad Naghshvar, Tara Javidi, et al. Active sequential hypothesis testing. The Annals of Statistics, 41 (6): 2703-2738, 2013b.
32. George L Nemhauser, Laurence A Wolsey, and Marshall L Fisher. An analysis of approximations for maximizing submodular set functionsi. Mathematical Programming, 14 (1): 265-294, 1978.
33. Robert Nowak. Noisy generalized binary search. In Advances in neural information processing systems, pages 1366-1374, 2009.
34. Burr Settles. Active learning. Synthesis Lectures on Artificial Intelligence and Machine Learning, 6 (1): 1-114, 2012.
35. C. E. Shannon. A mathematical theory of communication. Bell System Technical Journal, 27 (3): 379-423, 1948.
36. Alexandre B Tsybakov. Optimal aggregation of classifiers in statistical learning. Annals of Statistics, pages 135-166, 2004.
37. Jason L Williams, John Iii, and Alan S Willsky. Performance guarantees for information theoretic active inference. In International Conference on Artificial Intelligence and Statistics, pages 620-627, 2007.
38. Alice X. Zheng, Irina Rish, and Alina Beygelzimer. Efficient test selection in active diagnosis via entropy approximation. In In Proceedings of UAI-05, 2005.