Robustness and generalization

Machine Learning

Volumn 86, Issue 3, 2012, Pages 391-423

  Xu, Huan ^a   Mannor, Shie ^b  

^a NATIONAL UNIVERSITY OF SINGAPORE   (Singapore)

^b TECHNION ISRAEL INSTITUTE OF TECHNOLOGY   (Israel)

Author keywords

Generalization; Non IID sample; Quantile loss; Robust

Indexed keywords

FUNDAMENTAL PROPERTIES; GENERALIZATION; GENERALIZATION BOUND; MARKOVIAN; NON-IID SAMPLE; ROBUST; TESTING ERRORS; TESTING SAMPLES; TRAINING ERRORS; TRAINING SAMPLE;

ARTIFICIAL INTELLIGENCE; SOFTWARE ENGINEERING;

LEARNING ALGORITHMS;

EID: 84867738590     PISSN: 08856125     EISSN: 15730565     Source Type: Journal    
DOI: 10.1007/s10994-011-5268-1     Document Type: Article

References (56)

1. Alon, N., Ben-David, S., Cesa-Bianchi, N., & Haussler, D. (1997). Scale-sensitive dimension, uniform convergence, and learnability. Journal of the ACM, 44(4), 615-631. (Pubitemid 127617707)
2. Bartlett, P. L. (1998). The sample complexity of pattern classification with neural networks: the size of the weight is more important than the size of the network. IEEE Transactions on Information Theory, 44(2), 525-536. (Pubitemid 128737620)
3. Bartlett, P. L., & Mendelson, S. (2002). Rademacher and Gaussian complexities: risk bounds and structural results. Journal of Machine Learning Research, 3, 463-482.
4. Bartlett, P. L., Bousquet, O., & Mendelson, S. (2005). Local Rademacher complexities. The Annals of Statistics, 33(4), 1497-1537. (Pubitemid 41423979)
5. Ben-David, S., Blitzer, J., Crammer, K., & Pereira, F. (2007). Analysis of representations for domain adaptation. In Advances in neural information processing systems (Vol. 19).
6. Ben-Tal, A., & Nemirovski, A. (1998). Robust convex optimization. Mathematics of Operations Research, 23, 769-805. (Pubitemid 128508557)
7. Ben-Tal, A., & Nemirovski, A. (1999). Robust solutions of uncertain linear programs. Operations Research Letters, 25(1), 1-13.
8. Bertsimas, D., & Sim, M. (2004). The price of robustness. Operations Research, 52(1), 35-53. (Pubitemid 40000812)
9. Bhattacharyya, C., Pannagadatta, K. S., & Smola, A. J. (2004). A second order cone programming formulation for classifying missing data. In L. K. Saul, Y. Weiss, & L. Bottou (Eds.), Advances in neural information processing systems (NIPS17). Cambridge: MIT Press.
10. Bousquet, O., & Elisseeff, A. (2002). Stability and generalization. Journal of Machine Learning Research, 2, 499-526. (Pubitemid 135712570)
11. Bousquet, O., Boucheron, S., & Lugosi, G. (2005). Theory of classification: a survey of recent advances. ESAIM Probability and Statistics, 9, 323-375.
12. Cade, B., & Noon, B. (2003). A gentle introduction to quantile regression for ecologists. Frontiers in Ecology and the Environment, 1, 412-420.
13. Caramanis, C., Mannor, S., & Xu, H. (2011). Robust optimization and machine learning. In S. Sra, S. Nowozin, & S. Wright (Eds.), Optimization for machine learning. Cambridge: MIT Press.
14. Cole, T. (1988). Fitting smoothed centile curves to reference data. Journal of the Royal Statistical Society, Ser. A, 151, 385-418.
15. Cortes, C., & Vapnik, V. N. (1995). Support vector networks. Machine Learning, 20, 1-25.
16. Devroye, L., & Wagner, T. (1979a). Distribution-free inequalities for the deleted and holdout error estimates. IEEE Transactions of Information Theory, 25(2), 202-207. (Pubitemid 9443301)
17. Devroye, L., & Wagner, T. (1979b). Distribution-free performance bounds for potential function rules. IEEE Transactions of Information Theory, 25(2), 601-604. (Pubitemid 10426456)
18. Devroye, L., Györfi, L., & Lugosi, G. (1996). A probabilistic theory of pattern recognition. New York: Springer.
19. Doob, J. L. (1953). Stochastic processes. New York: Wiley.
20. Evgeniou, T., Pontil, M., & Poggio, T. (2000). Regularization networks and support vector machines. In A. J. Smola, P. L. Bartlett, B. Schölkopf, & D. Schuurmans (Eds.), Advances in large margin classifiers (pp. 171-203). Cambridge: MIT Press.
21. Gamarnik, D. (2003). Extension of the PAC framework to finite and countable Markov chains. IEEE Transaction on Information Theory, 49(1), 338-345.
22. Globerson, A., & Roweis, S. (2006). Nightmare at test time: robust learning by feature deletion. In Proceedings of the 23rd international conference on machine learning (pp. 353-360). New York: ACM Press.
23. Glynn, P. W., & Ormoneit, D. (2002). Hoeffding's inequality for uniformly ergodic Markov chains. Statistics and Probability Letters, 56, 143-146.
24. Huber, P. J. (1981). Robust statistics. New York: Wiley.
25. Kakade, S., Sridharan, K., & Tewari, A. (2009). On the complexity of linear predictions: risk bounds, margin bounds, and regularization. In Advances in neural information processing systems (Vol. 21, pp. 793-800).
26. Kearns, M., & Schapire, R. (1994). Efficient distribution-free learning of probabilistic concepts. Journal of Computer and System Sciences, 48(3), 464-497. (Pubitemid 124013300)
27. Klivans, A., Long, P., & Servedio, R. (2009). Learning halfspaces with malicious noise. Journal of Machine Learning Research, 10, 2715-2740.
28. Koenker, R., & Bassett, G. (1978). Regression quantiles. Econometrica, 46, 33-50.
29. Kolmogorov, A. N., & Tihomirov, V. (2002). ε-entropy and ε-capaciity of sets in functional spaces. American Mathematical Society Translations (2), 17, 227-364.
30. Koltchinskii, V. (2002). Rademacher penalties and structural risk minimization. IEEE Transactions on Information Theory, 47(5), 1902-1914.
31. Koltchinskii, V., & Panchenko, D. (2002). Empirical margin distributions and bounding the generalization error of combined classifiers. The Annals of Statistics, 30(1), 1-50. (Pubitemid 37095367)
32. Lanckriet, G. R., El Ghaoui, L., Bhattacharyya, C., & Jordan, M. I. (2003). A robust minimax approach to classification. Journal of Machine Learning Research, 3, 555-582.
33. Lozano, A. C., Kulkarni, S. R., & Schapire, R. E. (2006). Convergence and consistency of regularized boosting algorithms with stationary β-mixing observations. In Advances in neural information processing systems (Vol. 18).
34. Mansour, Y., Mohri, M., & Rostamizadeh, A. (2009). Domain adaptation: learning bounds and algorithms. In Proceedings of the 22nd annual conference on learning theory.
35. McDiarmid, C. (1989). On the method of bounded differences. In Surveys in combinatorics (pp. 148-188).
36. Meyn, S. P., & Tweedie, R. L. (1993). Markov chains and stochastic stability. New York: Springer.
37. Mukherjee, S., Niyogi, P., Poggio, T., & Rifkin, R. (2006). Learning theory: stability is sufficient for generalization and necessary and sufficient for consistency of empirical risk minimization. Advances in Computational Mathematics, 25(1-3), 161-193. (Pubitemid 43974373)
38. Poggio, T., Rifkin, R., Mukherjee, S., & Niyogi, P. (2004). General conditions for predictivity in learning theory. Nature, 428(6981), 419-422. (Pubitemid 38419686)
39. Rousseeuw, P. J., & Leroy, A. M. (1987). Robust regression and outlier detection. New York: Wiley.
40. Schölkopf, B., & Smola, A. J. (2002). Learning with kernels. Cambridge: MIT Press.
41. Shalev-Shwartz, S., Shamir, O., Srebro, N., & Sridharan, K. (2009). Learnability and stability in the general learning setting. In Proceedings of 22nd annual conference of learning theory.
42. Shivaswamy, P. K., Bhattacharyya, C., & Smola, A. J. (2006). Second order cone programming approaches for handling missing and uncertain data. Journal of Machine Learning Research, 7, 1283-1314. (Pubitemid 44024599)
43. Steinwart, I. (2006). Consistency of support vector machines and other regularized kernel classifiers. IEEE Transactions on Information Theory, 51(1), 128-142.
44. Sutton, R. S., & Barto, A. G. (1998). Reinforcement learning: an introduction. Cambridge: MIT Press.
45. Tibshirani, R. (1996). Regression shrinkage and selection via the Lasso. Journal of the Royal Statistical Society, Series B, 58(1), 267-288.
46. van der Vaart, A. W., & Wellner, J. A. (2000). Weak convergence and empirical processes. New York: Springer.
47. Vapnik, V. N., & Chervonenkis, A. (1974). Theory of pattern recognition. Moscow: Nauka.
48. Vapnik, V. N., & Chervonenkis, A. (1991). The necessary and sufficient conditions for consistency in the empirical risk minimization method. Pattern Recognition and Image Analysis, 1(3), 260-284.
49. Xu, H., & Mannor, S. (2010). Robustness and generalization. In Proceedings of the twenty-third annual conference on learning theory (pp. 503-515).
50. Xu, H., Caramanis, C., & Mannor, S. (2009a). Robustness and regularization of support vector machines. Journal of Machine Learning Research, 10, 1485-1510.
51. Xu, H., Caramanis, C., Mannor, S., & Yun, S. (2009b). Risk sensitive robust support vector machines. In Proceedings of the forty-eighth IEEE conference on decision and control (pp. 4655-4661).
52. Xu, H., Caramanis, C., & Mannor, S. (2010a). Principal component analysis with contaminated data: the high dimensional case. In Proceeding of the twenty-third annual conference on learning theory (pp. 490-502).
53. Xu, H., Caramanis, C.,&Mannor, S. (2010b). Robust regression and lasso. IEEE Transactions on Information Theory, 56(7), 3561-3574.
54. Yu, Y., Yang, M., Xu, L., White, M., & Schuurmans, D. (2011). Relaxed clipping: a global training method for robust regression and classification. In Advances in neural information processing systems (Vol. 23).
55. Zakai, A., & Ritov, R. (2009). Consistency and localizability. Journal of Machine Learning Research, 10, 827-856.
56. Zou, B., Li, L. Q., & Xu, Z. B. (2009). The generalization performance of ERM algorithm with strongly mixing observations. Machine Learning, 75, 275-295.