Learning noisy linear classifiers via adaptive and selective sampling

Machine Learning

Volumn 83, Issue 1, 2011, Pages 71-102

  Cavallanti, Giovanni ^a   Cesa Bianchi, Nicolò ^a   Gentile, Claudio ^b  

^a UNIVERSITY OF MILAN   (Italy)

^b UNIVERSITY OF INSUBRIA   (Italy)

Author keywords

Active learning; Adaptive sampling; Linear classification; Low noise; Selective sampling

Indexed keywords

ACTIVE LEARNING; ADAPTIVE SAMPLING; LINEAR CLASSIFICATION; LOW NOISE; SELECTIVE SAMPLING;

ALGORITHMS; CONVERGENCE OF NUMERICAL METHODS;

SAMPLING;

EID: 79953223661     PISSN: 08856125     EISSN: 15730565     Source Type: Journal    
DOI: 10.1007/s10994-010-5191-x     Document Type: Article

References (37)

1. D. Angluin 2004 Queries revisited Theoretical Computer Science 313 2 175 194 10.1016/j.tcs.2003.11.004 1069.68086 2051785
2. K. S. Azoury M. K. Warmuth 2001 Relative loss bounds for on-line density estimation with the exponential family of distributions Machine Learning 43 3 211 246 10.1023/A:1010896012157 0988.68173 (Pubitemid 32471833)
3. Balcan, M. F., Beygelzimer, A., & Langford, J. (2006). Agnostic active learning. In Proceedings of the 23rd international conference on machine learning (ICML) (pp. 65-72). Omnipress.
4. Balcan, M. F., Broder, A., & Zhang, T. (2007). Margin-based active learning. In Proceedings of the 20th annual conference on learning theory (COLT) (pp. 35-50). Berlin: Springer.
5. P. L. Bartlett M. I. Jordan J. D. McAuliffe 2006 Convexity, classification, and risk bounds Journal of the American Statistical Association 101 473 138 156 10.1198/016214505000000907 1118.62330 2268032 (Pubitemid 43500031)
6. G. Blanchard O. Bousquet L. Zwald 2007 Statistical properties of kernel principal component analysis Machine Learning 66 2-3 259 294 10.1007/s10994-006-6895-9 (Pubitemid 46360614)
7. S. Boucheron O. Bousquet G. Lugosi 2005 Theory of classification: a survey of some recent advances ESAIM Probability and Statistics 9 323 375 10.1051/ps:2005018 1136.62355 2182250
8. M. L. Braun 2006 Accurate error bounds for the eigenvalues of the kernel matrix Journal of Machine Learning Research 7 2303 2328 2274441 (Pubitemid 44708008)
9. Campbell, C., Cristianini, N., & Smola, A. (2000). Query learning with large margin classifiers. In Proceedings of the 17th international conference on machine learning (ICML) (pp. 111-118). San Mateo: Morgan Kaufmann.
10. R. Castro R. D. Nowak 2008 Minimax bounds for active learning IEEE Transactions on Information Theory 54 5 2339 2353 10.1109/TIT.2008.920189 2450865 (Pubitemid 351689563)
11. Cesa-Bianchi, N., & Lugosi, G. (2006). Prediction, learning, and games. Cambridge: Cambridge University Press.
12. N. Cesa-Bianchi A. Conconi C. Gentile 2005 A second-order Perceptron algorithm SIAM Journal on Computing 43 3 640 668 10.1137/S0097539703432542 2137076 (Pubitemid 40844236)
13. N. Cesa-Bianchi C. Gentile L. Zaniboni 2006 Incremental algorithms for hierarchical classification Journal of Machine Learning Research 7 31 54 2274361 (Pubitemid 43022941)
14. N. Cesa-Bianchi C. Gentile L. Zaniboni 2006 Worst-case analysis of selective sampling for linear-threshold algorithms Journal of Machine Learning Research 7 1205 1230 2274403 (Pubitemid 44015299)
15. Cohn, R., Atlas, L., & Ladner, R. (1990). Training connectionist networks with queries and selective sampling. In Advances in neural information processing systems (NIPS), 1989. New York: MIT Press.
16. R. Cohn L. Atlas R. Ladner 1994 Improving generalization with active learning Machine Learning 15 2 201 221
17. Dasgupta, S., Kalai, A. T., & Monteleoni, C. (2005). Analysis of Perceptron-based active learning. In Proceedings of the 18th conference on learning theory (COLT 2005) (pp. 249-263). Berlin: Springer.
18. Dasgupta, S., Hsu, D., & Monteleoni, C. (2008). A general agnostic active learning algorithm. In Advances in neural information processing systems (NIPS) (Vol. 21, pp. 353-360). New York: MIT Press.
19. Y. Freund S. Seung E. Shamir N. Tishby 1997 Selective sampling using the query by committee algorithm Machine Learning 28 2/3 133 168 10.1023/A:1007330508534 0881.68093 (Pubitemid 127506338)
20. Gilad-Bachrach, R., Navot, A., & Tishby, N. (2005). Query by committee made real. In Advances in neural information processing systems (NIPS) (Vol. 19). New York: MIT Press.
21. Hanneke, S. (2007). A bound on the label complexity of agnostic active learning. In Proceedings of the 24th international conference on machine learning (ICML) (pp. 353-360). Omnipress.
22. Hanneke, S. (2009). Adaptive rates of convergence in active learning. In Proceedings of the 22nd conference on learning theory (COLT 2009). Omnipress.
23. D. Helmbold N. Littlestone P. Long 2000 Apple tasting Information and Computation 161 2 85 139 10.1006/inco.2000.2870 1045.68573 1781564
24. Joachims, T. (1999). Making large-scale support vector machine learning practical. In B. Schölkopf, C. Burges, & A. Smola (Eds.), Advances in kernel methods: support vector learning. New York: MIT Press.
25. Kääriäinen, M. (2006). Active learning in the non-realizable case. In Proceedings of the 17th international conference on algorithmic learning theory (ALT 2006) (pp. 63-77). Berlin: Springer. (Pubitemid 44705620)
26. Lewis, D. D., & Gale, W. A. (1994). A sequential algorithm for training text classifiers. In Proceedings of the 17th annual international ACM-SIGIR conference on research and development in information retrieval (pp. 3-12). Berlin: Springer.
27. Monteleoni, C., & Kääriäinen, M. (2007). Practical online active learning for classification. In IEEE computer society conference on computer vision and pattern recognition (pp. 249-263). New York: IEEE Computer Society.
28. Muslea, I., Minton, S., & Knoblock, C. A. (2000). Selective sampling with redundant views. In Proceedings of the national conference on artificial intelligence (AAAI 2000) (pp. 621-626). New York: MIT Press.
29. NIST (2004). trec.nist.gov/data/reuters/reuters.html.
30. Sculley, D. (2008). Advances in online learning-based spam filtering. PhD Thesis in Computer Science, Tufts University. August.
31. J. Shawe-Taylor C. K. I. Williams N. Cristianini J. Kandola 2005 On the eigenspectrum of the Gram matrix and the generalization error of kernel-PCA IEEE Transactions on Information Theory 51 7 2510 2522 10.1109/TIT.2005.850052 2246374 (Pubitemid 41136405)
32. I. Steinwart C. Scovel 2007 Fast rates for support vector machines using Gaussian kernels Annals of Statistics 35 575 560 10.1214/009053606000001226 1127.68091 2336860
33. Strehl, A. L., & Littman, M. L. (2008). Online linear regression and its application to model-based reinforcement learning. In Advances in neural information processing systems (NIPS) (Vol. 21, pp. 1417-1424). New York: MIT Press.
34. Tong, S., & Koller, D. (2000). Support vector machine active learning with applications to text classification. In Proceedings of the 17th international conference on machine learning (ICML) (pp. 999-1006). San Mateo: Morgan Kaufmann.
35. A. Tsybakov 2004 Optimal aggregation of classifiers in statistical learning The Annals of Statistics 32 1 135 166 10.1214/aos/1079120131 1105.62353 2051002 (Pubitemid 41449306)
36. V. Vovk 2001 Competitive on-line statistics International Statistical Review 69 213 248 10.1111/j.1751-5823.2001.tb00457.x 02124683
37. Y. Ying D. X. Zhou 2006 Online regularized classification algorithms IEEE Transactions on Information Theory 52 4775 4788 10.1109/TIT.2006.883632 2302601 (Pubitemid 44680574)