Learning halfspaces with malicious noise

Journal of Machine Learning Research

Volumn 10, Issue , 2009, Pages 2715-2740

  Klivans, Adam R ^a   Long, Philip M ^b   Servedio, Rocco A ^c  

^a University of Texas at Austin   (United States)

^b GOOGLE INC   (United States)

^c Columbia University ^*  (United States)

Author keywords

Agnostic learning; Halfspace learning; Label noise; Linear classifiers; Malicious noise; Noise tolerance; Pac learning

Indexed keywords

AGNOSTIC LEARNING; HALF-SPACE; HALFSPACE LEARNING; LINEAR CLASSIFIERS; NOISE TOLERANCE; PAC LEARNING;

CLASSIFIERS; EXPONENTIAL FUNCTIONS; LABELS; PRINCIPAL COMPONENT ANALYSIS;

LEARNING ALGORITHMS;

EID: 75249095624     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (32)

1. S. Arora, L. Babai, J. Stern, and Z. Sweedyk. The hardness of approximate optima in lattices, codes, and systems of linear equations. Proceedings of the 34th Annual Symposium on Foundations of Computer Science, pages 724-733, 1993.
2. A. Blum, A. Frieze, R. Kannan, and S. Vempala. A polynomial time algorithm for learning noisy linear threshold functions. Algorithmica, 22(l/2):35-52, 1997.
3. A. Blumer, A. Ehrenfeucht, D. Haussler, and M. Warmuth. Learnability and the VapnikChervonenkis dimension. Journal of the ACM, 36(4):929-965, 1989.
4. S. C. Brubaker. Extensions of Principle Components Analysis. PhD thesis, Georgia Institute of Technology, 2009.
5. N. H. Bshouty, Y. Li, and P. M. Long. Using the doubling dimension to analyze the generalization of learning algorithms. Journal of Computer & System Sciences, 75(6):323-335, 2009.
6. J. Dunagan and S. Vempala. Optimal outlier removal in high-dimensional spaces. J. Computer & System Sciences, 68(2):335-373, 2004.
7. V. Feldman, P. Gopalan, S. Khot, and A. Ponnuswami. New results for learning noisy parities and halfspaces. In Proc. 47th IEEE Symposium on Foundations of Computer Science (FOCS), pages 563-576, 2006.
8. Y. Freund and R. E. Schapire. Large margin classification using the perceptron algorithm. Machine Learning, 37(3):277-296, 1999. (Pubitemid 32210619)
9. D. Gavinsky. Optimally-smooth adaptive boosting and application to agnostic learning. Journal of Machine Learning Research, 4:101-117, 2003.
10. V. Guruswami and P. Raghavendra. Hardness of learning halfspaces with noise. In Proc. 47th IEEE Symposium on Foundations of Computer Science (FOCS), pages 543-552. IEEE Computer Society, 2006.
11. D. Haussler, M. Kearns, N. Littlestone, and M. Warmuth. Equivalence of models for polynomial learnability. Information and Computation, 95(2):.129-161, 1991.
12. I.T. Jolliffe. Principal Component Analysis. Springer Series in Statistics, 2002.
13. A. Kalai, A. Klivans, Y. Mansour, and R. Servedio. Agnostically learning halfspaces. SIAM Journal on Computing, 37(6): 1777 - 1805, 2008.
14. M. Kearns and M. Li. Learning in the presence of malicious errors. SIAM Journal on Computing, 22(4): 807-837, 1993.
15. M. Kearns, R. Schapire, and L. Sellie. Toward Efficient Agnostic Learning. Machine Learning, 17 (2/3): 115-141, 1994.
16. A. Klivans and A. Sherstov. Cryptographic hardness for learning intersections of halfspaces. In Proc. 47th IEEE Symposium on Foundations of Computer Science (FOCS), pages 553-562, 2006. (Pubitemid 351175543)
17. N. Littlestone. Learning quickly when irrelevant attributes abound: a new linear-threshold algorithm. Machine Learning, 2:285-318, 1987.
18. N. Littlestone. Redundant noisy attributes, attribute errors, and linear-threshold learning using Winnow. In Proceedings of the Fourth Annual Workshop on Computational Learning Theory, pages 147-156, 1991.
19. L. Lovász and S. Vempala. The geometry of logconcave functions and sampling algorithms. Random Structures and Algorithms, 30(3):307-358, 2007.
20. W. Maass and G Turan. How fast can a threshold gate learn? In Computational Learning Theory and Natural Learning Systems: Volume I: Constraints and Prospects, pages 381-414. MIT Press, 1994.
21. Y. Mansour and M. Parnas. Learning conjunctions with noise under product distributions. Information Processing Letters, 68(4): 189-196, 1998.
22. A, Novikoff. On convergence proofs on perceptrons. In Proceedings of the Symposium on Mathematical Theory of Automata, volume XII, pages 615-622, 1962.
23. D. Pollard. Convergence of Stochastic Processes. Springer Verlag, 1984.
24. F. Rosenblatt. The Perceptron: a probabilistic model for information storage and organization in the brain. Psychological Review, 65:386-407, 1958.
25. R. Schapire and Y Singer. Improved boosting algorithms using confidence-rated predictions. Machine Learning, 37:297-336, 1999. (Pubitemid 32210620)
26. R. Servedio. Efficient Algorithms in Computational Learning Theory. PhD thesis, Harvard University, 2001.
27. R. Servedio. PAC analogues of Perceptron and Winnow via boosting the margin. Machine Learning, 47(2/3): 133-151, 2002.
28. R. Servedio. Smooth boosting and learning with malicious noise. Journal of Machine Learning Research, 4:633-648, 2003.
29. J. Shawe-Taylor and N. Cristianini, An Introduction to Support Vector Machines. Cambridge University Press, 2000.
30. M. Talagrand. Sharper bounds for Gaussian and empirical processes. Annals of Probability, 22: 28-76, 1994.
31. L. Valiant. Learning disjunctions of conjunctions. In Proceedings of the Ninth International Joint Conference on Artificial Intelligence, pages 560-566, 1985.
32. H. Xu, C. Caramanis, and S. Mannor. Principal component analysis with contaminated data: The high dimensional case. Journal of Machine Learning Research, 2009. To appear.