Algorithms for sparse linear classifiers in the massive data setting

Journal of Machine Learning Research

Volumn 9, Issue , 2008, Pages 313-337

  Balakrishnan, Suhrid ^a   Madigan, David ^b  

^a RUTGERS UNIVERSITY   (United States)

^b Columbia University ^*  (United States)

Author keywords

Expectation propagation; Laplace approximation; LASSO

Indexed keywords

APPROXIMATION THEORY; CLASSIFICATION (OF INFORMATION); COMPUTATIONAL COMPLEXITY; PROBLEM SOLVING;

EXPECTATION PROPAGATION; LAPLACE APPROXIMATION; LASSO;

ALGORITHMS;

EID: 41549108812     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (27)

1. J. M. Bernardo and A. F. M. Smith. Bayesian Theory. John Wiley and Sons, Inc., 1994.
2. S. S. Chen, D. L. Donoho, and M. A. Saunders. Atomic decomposition by Basis Pursuit. SIAM Journal on Scientific Computing, 20(1):33-61, January 1999.
3. B. Efron, T. Hastie, I. Johnstone, and R. Tibshirani. Least angle regression. The Annals of Statistics, 32(2):407-499, 2004.
4. E. Eskin, A. J. Smola, and S.V.N. Vishwanathan. Laplace Propagation. In Neural Information Processing Systems, 16. MIT Press, 2003.
5. M. A. T. Figueiredo and A. K. Jain. Bayesian learning of sparse classifiers. In Proceedings of the Computer Vision and Pattern Recognition Conference, volume 1, pages 35-41, 2001.
6. W. J. Fu. Penalized regressions: The Bridge versus the Lasso. Journal of Computational and Graphical Statistics, 7(3):397-416, 1998.
7. A. Genkin, D. D. Lewis, and D. Madigan. Large-scale Bayesian logisitic regression for text categorization. Technometrics, 49:291-304, 2007.
8. T. Hastie, S. Rosset, R. Tibshirani, and J. Zhu. The entire regularization path for the Support Vector Machine. Journal of Machine Learning Research, 5:1391-1415, 2004.
9. T. Jaakkola and M. Jordan. Bayesian parameter estimation via variational methods. Statistics and Computing, 10:25-37, 2000.
10. R. E. Kass and A. E. Raftery. Bayes factors. Journal of the American Statistical Association, 90:773-795, 1995.
11. 1-regularized logistic regression. Journal of Machine Learning Research, 8:1519-1555, 2007.
12. P. Komarek and A. Moore. Making logistic regression a core data mining tool: A practical investigation of accuracy, speed, and simplicity. Technical Report TR-05-27, Robotics Institute, Carnegie Mellon University, Pittsburgh, PA, May 2005.
13. B. Krishnapuram, L. Carin, M. A. T. Figueiredo, and A. J. Hartemink. Sparse multinomial logistic regression: Fast algorithms and generalization bounds. IEEE Transactions on Pattern Analalysis and Machine Intelligence, 2005.
14. D. D. Lewis. Reuters-21578 text categorization test collection: Distribution 1.0 readme file (v 1.3)., 2004. URL http://www.daviddlewis.com/ resources/testcollections/reuters21578/readme.txt.
15. D. D. Lewis, Y. Yang, T. G. Rose, and F. Li. RCV1: A new benchmark collection for text categorization research. Journal of Machine Learning Research, 5:361-397, 2004.
16. D. J. C. MacKay. Probable networks and plausible predictions: a review of practical Bayesian methods for supervised neural networks. Network: Computation in Neural Systems, 6:469-505, 1995.
17. T. P. Minka. Expectation Propagation for approximate Bayesian inference. In Jack Breese and Daphne Koller, editors, Proceedings of the Seventeenth Conference on Uncertainty in Artificial Intelligence (UAI-01), pages 362-369, San Francisco, CA, August 2-5 2001a. Morgan Kaufmann Publishers.
18. T. P. Minka. A Family of Algorithms for Approximate Bayesian Inference. PhD thesis, Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2001b.
19. M. Opper. A Bayesian approach to on-line learning. In D. Saad, editor, Online Learning in Neural Networks, pages 363-378. Cambridge University Press, 1998.
20. M. R. Osborne, B. Presnell, and B. A. Turlach. A new approach to variable selection in least squares problems. IMA Journal of Numerical Analysis, 20(3):389-403, July 2000.
21. Y. Qi, T. R Minka, R. W. Picard, and Z. Ghahramani. Predictive automatic relevance determination by Expectation Propagation. In Proceedings of Twenty-first International Conference on Machine Learning, Banff, Alberta, Canada, July 4-8 2004.
22. R. T. Rockafellar. Convex Analysis. Princeton University Press, Princeton, N.J, 1970.
23. S. K. Shevade and S. S. Keerthi. A simple and efficient algorithm for gene selection using sparse logistic regression. Bioinformatics., 19(17):2246-2253, 2003.
24. R. J. Tibshirani. Regression shrinkage and selection via the Lasso. Journal of the Royal Statistical Society, Series B, 58(1):267-288, 1996.
25. T. Zhang. On the dual formulation of regularized linear systems. Machine Learning, 46:91-129, 2002.
26. T. Zhang and F. J. Oles. Text categorization based on regularized linear classification methods. Information Retrieval, 4(1):5-31, 2001.
27. H. Zou. The adaptive Lasso and its oracle properties. Journal of the American Statistical Association, 101:1418-1429, 2006.