Active learning and basis selection for kernel-based linear models: A bayesian perspective

IEEE Transactions on Signal Processing

Volumn 58, Issue 5, 2010, Pages 2686-2700

  Paisley, John ^a   Liao, Xuejun ^a   Carin, Lawrence ^a  

^a Duke University   (United States)

Author keywords

Active learning; Bayesian models; Kernel methods; Linear regression and classification; Optimal experiments

Indexed keywords

ACTIVE LEARNING; ACTIVE LEARNING METHODS; ACTIVE-LEARNING ALGORITHM; APRIORI; BASIS SELECTION; BAYESIAN; BAYESIAN MODEL; BAYESIAN PERSPECTIVE; DIFFERENTIAL ENTROPY; ENTROPY CRITERIA; GREEDY ALGORITHMS; KERNEL FUNCTION; KERNEL METHODS; LINEAR MODEL; MATRIX; MODEL PARAMETERS; OBSERVED DATA; OPTIMAL EXPERIMENTS; PARAMETER VECTORS; POSTERIOR DISTRIBUTIONS; RIDGE REGRESSION; SUBMATRIX;

BAYESIAN NETWORKS; OPTIMIZATION; REGRESSION ANALYSIS;

LEARNING ALGORITHMS;

EID: 77951218308     PISSN: 1053587X     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSP.2010.2042491     Document Type: Article

References (33)

1. R. G. Baraniuk and M. B.Wakin, "Random projections of smooth manifolds," Found. Computat. Math., vol.9, no.1, pp. 51-77, 2009.
2. C. Bishop, Pattern Recognition and Machine Learning. New York: Springer, 2006.
3. S. Boyd and L. Vandenberghe, Convex Optimization. Cambridge, U.K.: Cambridge Univ. Press, 2003.
4. C. J. C. Burges, "A tutorial on support vector machines for pattern recognition," Data Mining Knowledge Discov., vol.2, pp. 121-167, 1998.
5. K. Chaloner and I. Verdinelli, "Bayesian experimental design: A review," Statist. Sci., vol.10, pp. 237-304, 1995.
6. D. A. Cohn, Z. Ghahramani, and M. I. Jordan, "Active learning with statistical models," J. Artif. Intell. Res., vol.4, pp. 129-145, 1996.
7. T. M. Cover and J. A. Thomas, Elements of Information Theory, 2nd ed. New York: Wiley, 2006.
8. D. L. Donoho, "Compressed sensing," IEEE Trans. Inf. Theory, vol.52, pp. 1289-1306, 2006.
9. V. V. Federov, Theory of Optimal Experiments. New York: Academic , 1972.
10. C. S. Foo, C. B. Do, and A. Y. Ng, "A majorization-minimization algorithm for (multiple) hyperparameter learning," in Proc. 26th Int. Conf. Mach. Learn. (ICML), 2009.
11. P. Grassberger and I. Procaccia, "Measuring the strangeness of strange attractors," Physica D: Nonlin. Phenom., vol.9, pp. 189-208, 1983.
12. C. Hegde, M. B. Wakin, and R. G. Baraniuk, "Random projections for manifold learning," NIPS, 2007.
13. A. E. Hoerl and R. W. Kennard, "Ridge regression: Biased estimation for nonorthogonal problems," Technometrics, Special Ed., Feb. 2000, vol. 42, no. 1, pp. 80-86, 1970.
14. I. Jolliffe, Principal Component Analysis. New York: Wiley, 2005.
15. N. D. Lawrence, M. Seeger, and R. Herbrich, "Fast sparse Gaussian process methods: The informative vector machine," Adv. Neural Inf. Process. Syst., vol.15, pp. 609-616, 2002.
16. X. Liao, Y. Zhang, and L. Carin, "Plan-in-advance active learning of classifiers," in Foundations and Applications of Sensor Management, A. Hero, Ed. New York: Springer, 2007, pp. 201-220.
17. X. Liao and L. Carin, "Application of the theory of optimal experiments to adaptive electromagnetic-induction sensing of buried targets," IEEE Trans. Pattern Anal. Mach. Intell., vol.26, no.8, pp. 961-972, 2004.
18. D. J. MacKay, "Information-based objective functions for active data selection," Neural Computat., vol.4, pp. 590-604, 1992.
19. D. J. MacKay, "The evidence framework applied to classification networks," Neural Computat., vol.4, pp. 720-736, 1992.
20. P. McCullagh and J. A. Nelder, Generalized Linear Models, 2nd ed. London, U.K.: Chapman and Hall, 1989.
21. C. E. Rasmussen and C. K. I. Williams, Gaussian Processes for Machine Learning. Cambridge: MIT press, 2006.
22. N. Roy and A. McCallum, "Toward optimal active learning through sampling estimation of error reduction," in Proc. 18th Int. Conf. Mach. Learn. (ICML), 2001.
23. M. W. Seeger and H. Nickisch, "Compressed sensing and Bayesian experimental design," in Proc. 25th Int. Conf. Mach. Learn. (ICML), 2008.
24. R. Tibshirani, "Regression shrinkage and selection via the lasso," J. Royal Statist. Soc. Ser. B, vol.1-58, pp. 267-288, 1996.
25. M. E. Tipping, "Sparse Bayesian learning and the relevance vector machine," J. Mach. Learn. Res. , vol.1, pp. 211-244, 2001.
26. J. A. Tropp and A. C. Gilbert, Signal Recovery From Partial Information via Orthogonal Matching Pursuit 2005, Preprint Univ. Mich..
27. P. Vincent and Y. Bengio, "Kernel matching pursuit," Mach. Learn., vol.48, pp. 165-187, 2002.
28. M. B.Wakin and R. G. Baraniuk, "Random projections of signal manifolds," in Proc. ICASSP 2006, 2006.
29. Y. Zhang, X. Liao, E. Dura, and L. Carin, "Active selection of labeled data for target detection," in Proc. ICASSP, 2004, vol.5, pp. 465-468.
30. Y. Zhang, X. Liao, and L. Carin, "Detection of buried targets via active selection of labeled data: Application to sensing subsurface UXO," IEEE Trans. Geosci. Remote Sens., vol.42, no.11, pp. 2535-2543, 2004.
31. X. Zhu, Z. Ghahramani, and J. Lafferty, "Semi-supervised learning using Gaussian fields and harmonic functions," in Proc. 20th Int. Conf. Mach. Learn. (ICML), 2003.
32. X. Zhu, J. Lafferty, and Z. Ghahramani, "Combining active learning and semi-supervised learning using Gaussian fields and harmonic functions," in Proc. 20th Int. Conf. Mach. Learn.(ICML), 2003.
33. H. Zou and T. Hastie, "Regularization and variable selection via the elastic net," J. Royal Statist. Soc.: Series B, vol.67, no.2, pp. 301-320, 2005.