Adaptive sparseness for supervised learning

IEEE Transactions on Pattern Analysis and Machine Intelligence

Volumn 25, Issue 9, 2003, Pages 1050-1159

  Figueiredo, Mário A T ^a  

^a INSTITUTO SUPERIOR TÉCNICO   (Portugal)

Author keywords

Classification; Expectation maximization algorithm; Feature selection; Kernel methods; Regression; Sparseness; Supervised learning

Indexed keywords

ALGORITHMS; COMPUTATIONAL COMPLEXITY; COMPUTER SIMULATION; NEURAL NETWORKS; OPTIMIZATION; PROBABILITY DENSITY FUNCTION; REGRESSION ANALYSIS;

ADAPTIVE SPARSENESS; EXPECTATION-MAXIMIZATION ALGORITHM; FEATURE SELECTION; SUPERVISED LEARNING;

LEARNING SYSTEMS;

EID: 0141836275     PISSN: 01628828     EISSN: None     Source Type: Journal    
DOI: 10.1109/TPAMI.2003.1227989     Document Type: Article

References (32)

1. M. Figueiredo, "Adaptive Sparseness Using Jeffreys Prior," Proc. Advances in Neural Information Processing Systems 14, T. Dietterich, S. Becker, and Z. Ghahramani, eds. Cambridge, Mass.: MIT Press, pp. 697-704, 2002.
2. M. Figueiredo and A.K. Jain, "Bayesian Learning of Sparse Classifiers," Proc. IEEE Computer Soc. Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 35-41, 2001.
3. Z. Chen and S. Haykin, "On Different Facets of Regularization Theory," Neural Computation, vol. 14, no. 12, pp. 2791-2846, 2002.
4. N. Cristianini and J. Shawe-Taylor, Support Vector Machines and Other Kernel-Based Learning Methods. Cambridge, U.K.: Cambridge Univ. Press, 2000.
5. R. Neal, Bayesian Learning for Neural Networks. New York: Springer Verlag, 1996.
6. B. Ripley, Pattern Recognition and Neural Networks. Cambridge, U.K.: Cambridge Univ. Press, 1996.
7. V. Vapnik, Statistical Learning Theory. New York: John Wiley, 1998.
8. M. Figueiredo, "On Gaussian Radial Basis Function Approximations: Interpretation, Extensions, and Learning Strategies," Proc. Int'l Conf. Pattern Recognition, vol. 2, pp. 618-621, 2000.
9. L. Breiman, J. Friedman, R. Olshen, and C. Stone, Classification and Regression Trees. CRC Press, 1983.
10. C. Williams, "Prediction with Gaussian Processes: From Linear Regression to Linear Prediction and Beyond," Learning in Graphical Models, M.I. Jordan, ed., The Netherlands: Kluwer, pp. 599-621, 1998.
11. A. Hoerl and R. Kennard, "Ridge Regression: Biased Estimation for Nonorthogonal Problems," Technometrics, vol. 12, pp. 55-67, 1970.
12. C. Williams and D. Barber, "Bayesian Classification with Gaussian Priors," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 20, no. 12, pp. 1342-1351, Dec. 1998.
13. G. Wahba, Spline Models for Observational Data. Soc. for Industrial and Applied Math. (SIAM), Philadelphia, 1990.
14. T. Poggio and F. Girosi, "Networks for Approximation and Learning," Proc. IEEE, vol. 78, pp. 1481-1497, 1990.
15. S. Chen, D. Donoho, and M. Saunders, "Atomic Decomposition by Basis Pursuit," SIAM J. Scientific Computation, vol. 20, no. 1, pp. 33-61, 1998.
16. R. Tibshirani, "Regression Shrinkage and Selection via the Lasso," J. Royal Statistical Soc. (B), vol. 58, pp. 267-288, 1996.
17. P. Williams, "Bayesian Regularization and Pruning Using a Laplace Prior," Neural Computation, vol. 7, pp. 117-143, 1995.
18. K. Lange and J. Sinsheimer, "Normal/Independent Distributions and Their Applications in Robust Regression," J. Computational and Graphical Statistics, vol. 2, pp. 175-198, 1993.
19. M. Figueiredo and R. Nowak, "Wavelet-Based Image Estimation: An Empirical Bayes Approach Using Jeffreys' Noninformative Prior," IEEE Trans. Image Processing, vol. 10, no. 9, pp. 1322-1331, 2001.
20. J. Berger, Statistical Decision Theory and Bayesian Analysis. New York: Springer-Verlag, 1980.
21. D. MacKay, "Bayesian Non-Linear Modelling for the 1993 Energy Prediction Competition," Maximum Entropy and Bayesian Methods, G. Heidbreder, ed. Kluwer, pp. 221-234, 1996.
22. C. Bishop and M. Tipping, "Variational Relevance Vector Machines," Proc. 16th Uncertainty in Artificial Intelligence, pp. 46-53, 2000.
23. M. Tipping, "The Relevance Vector Machine," Proc. Advances in Neural Information Processing Systems S. Solla, T. Leen, and K.-R. Müller, eds. Cambridge, Mass.: MIT Press, pp. 652-658, 2000.
24. D. Donoho and I. Johnstone, "Ideal Spatial Adaptation via Wavelet Shrinkage," Biometrika, vol. 81, pp. 425-455, 1994.
25. P. Moulin and J. Liu, "Analysis of Multiresolution Image Denoising Schemes Using Generalized-Gaussian and Complexity Priors," IEEE Trans. Information Theory, vol. 45, pp. 909-919, 1999.
26. Y. Grandvalet, "Least Absolute Shrinkage is Equivalent to Quadratic Penalization," Perspectives in Neural Computing, L. Niklasson, M. Boden, and T. Ziemske, eds. Springer Verlag, pp. 201-206, 1998.
27. M. Osborne, B. Presnell, and B. Turlach, "A New Approach to Variable Selection in Least Squares Problems," IMA J. Numerical Analysis, vol. 20, pp. 389-404, 2000.
28. P. McCullagh and J. Nelder, Generalized Linear Models. London: Chapman and Hall, 1989.
29. L. Fahrmeir and G. Tutz, Multivariate Statistical Modeling Based on Generalized Linear Models. New York: Springer Verlag, 1994.
30. J. Albert and S. Chib, "Bayesian Analysis of Binary and Polychotomous Response Data," J. Am. Statistical Assoc., vol. 88, pp. 669-679, 1993.
31. M. Seeger, "Bayesian Model Selection for Support Vector Machines, Gaussian Processes, and Other Kernel Classifiers," Proc. Advances in Neural Information Processing Systems 12, S. Solla, T. Leen, and K.-R. Müller, eds. Cambridge, Mass.: MIT press, 2000.
32. C. Williams and M. Seeger, "Using the Nystrom Method to Speedup Kernel Machines," Proc. Advances in Neural Information Processing Systems 13, T. Leen, T. Dietterich, and V. Tresp, eds. Cambridge, Mass.: MIT Press, pp. 682-688, 2001.