Feature selection, L1 vs. L2 regularization, and rotational invariance

Proceedings, Twenty-First International Conference on Machine Learning, ICML 2004

Volumn , Issue , 2004, Pages 615-622

  Ng, Andrew Y ^a  

^a Stanford University ^*  (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHMS; LEARNING SYSTEMS; LOGISTICS; MAXIMUM LIKELIHOOD ESTIMATION; OPTIMIZATION; PARAMETER ESTIMATION; PERSONNEL TRAINING; REGRESSION ANALYSIS; VECTORS;

FEATURE SELECTION; LOGARITHMIC RATES; LOGISTIC REGRESSION; REGULARIZATION;

FEATURE EXTRACTION;

EID: 14344249889     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (21)

1. Anthony, M., & Bartlett, P. (1999). Neural network learning: Theoretical foundations. Cambridge University Press.
2. Bartlett, P. (1998). The sample complexity of pattern classification with neural networks: The size of the weights is more important than the size of the network. IEEE Transactions on Information Theory, 2, 525-536.
3. Blum, A., & Langley, P. (1997). Selection of relevant features and examples in machine learning. Artificial Intelligence, 97, 245-271.
4. Ehrenfeucht, A., Haussler, D., Kearns, M., & Valiant, L. (1989). A general lower bound on the number of examples needed for learning. Information and Computation, 82, 247-261.
5. Haussler, D. (1992). Decision-theoretic generalizations of the PAC model for neural networks and other applications. Information and Computation, 100, 78-150.
6. Kivinen, J., Warmuth, M., & Auer, P. (1995). The percep tron vs. winnow: Linear vs. logarithmic mistake bounds when few input variables are relevant. Proc. 8th Annual Conference on Computational Learning Theory (pp. 289-296).
7. Kivinen, J., & Warmuth, M. K. (1994). Exponentiated gradient versus gradient descent for linear predictors (Technical Report UCSC-CRL-94-16). Univ. of California Santa Cruz, Computer Research Laboratory.
8. Kohavi, R., & John, G. (1997). Wrappers for feature subset selection. Artificial Intelligence, 97, 273-324.
9. Littlestone, N. (1988). Learning quickly when irrelevant attributes abound: A new linear-threshold algorithm. Machine Learning, 2, 285-318.
10. McCullagh, P., & Nelder, J. A. (1989). Generalized linear models (second edition). Chapman and Hall.
11. Ng, A. Y. (1998). On feature selection: Learning with exponentially many irrelevant features as training examples. Proceedings of the Fifteenth International Conference on Machine Learning (pp. 404-412). Morgan Kaufmann.
12. Ng, A. Y., & Jordan, M. I. (2001). Convergence rates of the voting gibbs classifier, with application to bayesian feature selection. Proceedings of the Eighteenth International Conference on Machine Learning. Morgan Kaufmann.
13. Nigam, K., Lafferty, J., & McCallum, A. (1999). Using maximum entropy for text classification. IJCAI-99 Workshop on ML for Information Filtering.
14. Pollard, D. (1984). Empirical processes: Theory and applications. Springer-Verlag.
15. Rockafellar, R. (1970). Convex analysis. Princeton Univ. Press.
16. Strang, G. (1988). Linear algebra and its applications, 3rd ed. International Thomas Publishing.
17. Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. J. Royal. Statist. Soc B., 58, 267-288.
18. Vapnik, V. (1982). Estimation of dependences based on empirical data. Springer-Verlag.
19. Vapnik, V. N. (1998). Statistical learning theory. John Wiley & Sons.
20. Zhang, T. (2002). Covering number bounds of certain regularized linear function classes. Journal of Machine Learning Research, 527-550.
21. Zheng, A. X., Jordan, M. I., Liblit, B., & Aiken, A. (2004). Statistical debugging of sampled programs. Neural Information Processing Systems 16.