Iterative learning algorithms for linear Gaussian observation models

IEEE Transactions on Signal Processing

Volumn 52, Issue 8, 2004, Pages 2286-2297

  Deng, Guang ^a  

^a LA TROBE UNIVERSITY   (Australia)

Author keywords

[No Author keywords available]

Indexed keywords

INTEGRATION; ITERATIVE METHODS; LEARNING SYSTEMS; MATHEMATICAL MODELS; OPTIMIZATION; PARAMETER ESTIMATION; SIGNAL PROCESSING; SPURIOUS SIGNAL NOISE; STATISTICAL METHODS; WAVELET TRANSFORMS;

ADAPTIVE PREDICTION; HYPERPARAMETERS; ITERATIVE ALGORITHM; SUPERVISED LEARNING; WAVELET DENOISING;

LEARNING ALGORITHMS;

EID: 3543111425     PISSN: 1053587X     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSP.2004.830984     Document Type: Article

References (41)

1. C. M. Bishop, Neural Networks for Pattern Recognition. London, U.K.: Oxford Univ. Press, 1995.
2. T. Hastie, R. Tibshirani, and J. Friedman, The Elements of Statistical Learning: Data Mining, Inference, and Prediction. New York: Springer-Verlag, 2001.
3. X. Li and M. Orchard, "Edge-directed prediction for lossless compression of natural images," IEEE Trans. Image Processing, Vol. 10, pp. 813-817, June 2001.
4. Lego - an object oriented image modeling framework," in Proc. IEEE Data Compression Conf., Snowbird, UT, Mar. 2001, p. 504.
5. D. J. C. Mackay, "Bayesian interpolation," Neural Comput., vol. 4, no. 3, pp. 415-447, 1992.
6. M. Nikolova, "Regularization functions and estimators," in Proc. IEEE Int. Conf. Image Processing, vol. 2, Sept. 1996, pp. 457-460.
7. J. Q. Fan and R. Z. Li, "Variable selection via nonconcave penalized likelihood and its oracle properties," J. Amer. Statist. Assoc., vol. 96, pp. 939-955, Sept. 2001.
8. W. J. Fu, "Penalized regressions: The bridge versus the LASSO," J. Comput. Graphical Statist., vol. 7, pp. 397-416, 1998.
9. J. E. Besag, "Digital image processing: toward Bayesian image analysis," J. Applied Statist., vol. 16, pp. 395-407, Feb. 1989.
10. R. Tibshirani, "Regression shrinkage and selection via lasso," J. R. Statist. Soc., ser. B, vol. 57, pp. 257-288, Sept. 1995.
11. R. Gonin and A. H. Money, Nonlinear Lp-Norm Estimation. New York: Marcel Dekker, 1989.
12. M. Nikolova, "Local strong homogeneity of a regularized estimator," SIAM J. Applied Math., vol. 61, pp. 633-658, 2000.
13. M. A. T. Figueiredo and R. B. Nowak, "An EM algorithm for wavelet-based image restoration," IEEE Trans. Image Processing, vol. 12, pp. 906-916, Aug. 2003.
14. M. A. T. Figueiredo, "Adaptive sparseness for supervised learning," IEEE Trans. Pattern Anal. Machine Intell., vol. 25, pp. 1150-1159, Sept. 2003.
15. E. P. Simoncelli, "Bayesian denoising of visual images in the wavelet domain," in Bayesian Inference in Wavelet Based Models. New York: Springer-Verlag, 1999, vol. 141, Lecture Notes in Statistics, pp. 291-308.
16. M. S. Lewicki and B. A. Olshausen, "Probabilistic framework for the adaptation and comparison of image codes," J. Opt. Soc. Amer. A: Opt., Image Sci., Vision, vol. 16, pp. 1587-1601, July 1999.
17. P. Moulin and J. Liu, "Analysis of multiresolution image denoising schemes using generalized Gaussian and complexity priors," IEEE Trans. Inform. Theory, vol. 45, pp. 909-919, Apr. 1999.
18. B. A. Olshausen, P. Sallee, and M. S. Lewicki, "Learning sparse image codes using a wavelet pyramid architecture," in Proc. Neural Inform. Process. Syst. Conf., Sept. 2000, pp. 887-893.
19. S. G. Chang, B. Yu, and M. Vetterli, "Adaptive wavelet thresholding for image denoising and compression," IEEE Trans. Image Processing, vol. 9, pp. 1532-1546, Sept. 2000.
20. B. A. Olshausen and K. J. Millman, "Learning sparse codes with a mixture-of-Gaussians prior," in Advances in Neural Information Processing Systems. Cambridge, MA: MIT Press, 2002, vol. 12, pp. 841-847.
21. P. M. Williams, "Bayesian regularization and pruning using a Laplace prior," Neural Comput., vol. 7, no. 1, pp. 117-143, 1995.
22. A. Antoniadis and J. Q. Fan, "Regularization of wavelet approximations," J. Amer. Statist. Assoc., vol. 96, pp. 939-955, Sept. 2001.
23. B. Vidakovic, "Discussion," J. Amer. Statist. Assoc., vol. 96, pp. 956-958, Sept. 2001.
24. P. Moulin, "Discussion," J. Amer. Statist. Assoc., vol. 96, pp. 959-960, Sept. 2001.
25. N. P. Galasanos and A. K. Katsaggelos, "Methods for choosing the regularization parameter and estimating the noise variance in image restoration and their relation," IEEE Trans. Image Processing, vol. 1, pp. 322-336, July 1992.
26. G. Archer and D. M. Titterington, "On some Bayesian/regularization methods for image restoration," IEEE Trans. Image Processing, vol. 4, pp. 989-995, July 1995.
27. D. J. C. MacKay, "Comparison of approximate methods for handling hyperparameters," Neural Comput., vol. 11, pp. 1035-1068, 1999.
28. S. S. Chen, "Basis Pursuit," Ph.D. dissertation, Dept. Statist., Stanford Univ., Stanford, CA, 1995.
29. Y. Grandvalet, "Least absolute shrinkage is equivalent to quadratic penalization," in Proc. Int. Conf. Artificial Neural Networks, Sweden, Sept. 1998, pp. 201-206.
30. D. J. C. MacKay, "Bayesian nonlinear modeling for the energy prediction competition," ASHRAE Trans., vol. 100, pp. 1053-1062, 1994.
31. B. A. Olshausen and D. J. Field, "Emergence of simple-cell receptive field properties by learning a spare code for natural images," Nature, vol. 381, pp. 607-609, 1996.
32. J. Dias, "Fast GEM wavelet-based deconvolution algorithm," in Proc. IEEE Int. Conf. Image Processing, vol. 2, Sept. 2003, pp. 961-964.
33. P. Bloomfield and W. L. Steiger, Least Absolute Deviations Theory, Applications and Algorithms. Boston, MA: Birkhauser, 1983.
34. P. Combettes and J. Lou, "An adaptive level set method for nondifferentiable constrained image recovery," IEEE Trans. Image Processing, vol. 11, pp. 1295-1304, Nov. 2002.
35. D. J. C. MacKay, Ensemble learning and evidence maximization, [Online] Available: wol.ra.phy.cam.ac.uk/mackay/nips.ps.gz, 1995.
36. R. E. Kass and L. Wasserman, "Formal rules for selecting prior distributions: A review and annotated bibliography," Carnegie Mellon Univ., Pittsburgh, PA, Tech. Rep. 583, 1994.
37. R. Y. Yang and J. O. Berger, A catalog of noninformative priors, [Online] Available: ftp.isds.duke.edulWorkingPapers/97-42.ps, 1996.
38. S. M. Kay, Fundamentals of Statistical Signal Processing Estimation Theory. Englewood Cliffs, NJ: Prentice-Hall, 1993.
39. M. A. T. Figueiredo and R. D. Nowak, "Wavelet-based image estimation: an empirical Bayes approach using Jeifreys' noninformative prior," IEEE Trans. Image Processing, vol. 10, pp. 1322-1331, Sept. 2001.
40. J. Portilla, V. Strela, M. J. Wainwright, and E. P. Simincelli, "Adaptive Wiener denoising using a Gaussian scale mixture model in the wavelet domain," in Proc. IEEE Int. Conf. Image Process., vol. 2, Sept. 2001, pp. 37-40.
41. D. S. Sivia, Data Analysis A Bayesian Tutorial. Oxford, U.K.: Clarendon, 1996.