Noise benefits in the expectation-maximization algorithm: Nem theorems and models

Proceedings of the International Joint Conference on Neural Networks

Volumn , Issue , 2011, Pages 3178-3183

  Osoba, Osonde ^a   Mitaim, Sanya ^b   Kosko, Bart ^a  

^a University of Southern California   (United States)

^b THAMMASAT UNIVERSITY   (Thailand)

Author keywords

[No Author keywords available]

Indexed keywords

EM ALGORITHMS; EXPECTATION-MAXIMIZATION ALGORITHMS; KULLBACK RELATIVE ENTROPIES; LOCAL MAXIMUM; NOISE CONDITIONS; PERFORMANCE MEASURE; SUFFICIENT CONDITIONS;

ADDITIVE NOISE; CONVERGENCE OF NUMERICAL METHODS; MAXIMUM LIKELIHOOD; MAXIMUM PRINCIPLE; NEURAL NETWORKS;

ALGORITHMS;

EID: 80054716868     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/IJCNN.2011.6033642     Document Type: Conference Paper

References (39)

1. P. Hänggi, "Stochastic resonance in biology," ChemPhysChem, vol. 3, no. 3, pp. 285-290, 2002.
2. R. Benzi, G. Parisi, A. Sutera, and A. Vulpiani, "A Theory of Stochastic Resonance in Climatic Change," SIAM Journal on Applied Mathematics, vol. 43, no. 3, pp. 565-578, June 1983.
3. J. J. Brey and A. Prados, "Stochastic Resonance in a One-Dimension Ising Model," Physics Letters A, vol. 216, pp. 240-246, June 1996.
4. A. R. Bulsara, R. D. Boss, and E. W. Jacobs, "Noise Effects in an Electronic Model of a Single Neuron," Biological Cybernetics, vol. 61, pp. 211-222, 1989.
5. A. R. Bulsara and L. Gammaitoni, "Tuning in to Noise," Physics Today, pp. 39-45, March 1996.
6. A. R. Bulsara and A. Zador, "Threshold Detection of Wideband Signals: A Noise-Induced Maximum in the Mutual Information," Physical Review E, vol. 54, no. 3, pp. R2185-R2188, September 1996.
7. R. Castro and T. Sauer, "Chaotic Stochastic Resonance: Noise- Enhanced Reconstruction of Attractors," Physical Review Letters, vol. 79, no. 6, pp. 1030-1033, August 1997.
8. F. Chapeau-Blondeau, "Noise-Enhanced Capacity via Stochastic Resonance in an Asymmetric Binary Channel," Physical Review E, vol. 55, no. 2, pp. 2016-2019, February 1997.
9. F. Chapeau-Blondeau and D. Rousseau, "Noise-enhanced performance for an optimal Bayesian estimator," IEEE Transactions on Signal Processing, vol. 52, no. 5, pp. 1327-1334, 2004.
10. H. Chen, P. Varshney, S. Kay, and J. Michels, "Noise enhanced nonparametric detection," IEEE Transactions on Information Theory, vol. 55, no. 2, pp. 499-506, 2009.
11. J. J. Collins, C. C. Chow, and T. T. Imhoff, "Stochastic Resonance without Tuning," Nature, vol. 376, pp. 236-238, July 1995.
12. P. Cordo, J. T. Inglis, S. Vershueren, J. J. Collins, D. M. Merfeld, S. Rosenblum, S. Buckley, and F. Moss, "Noise in Human Muscle Spindles," Nature, vol. 383, pp. 769-770, October 1996.
13. L. G. M. Löcher, A. Bulsara, P. Hänggi, J. Neff, K. Wiesenfeld, W. Ditto, and M. E. Inchiosa, "Controlling Stochastic Resonance," Physical Review Letters, vol. 82, no. 23, pp. 4574-4577, June 1999.
14. M. Grifoni and P. Hänggi, "Nonlinear Quantum Stochastic Resonance," Physical Review E, vol. 54, no. 2, pp. 1390-1401, August 1996.
15. D. Guerra, A. Bulsara, W. Ditto, S. Sinha, K. Murali, and P. Mohanty, "A Noise-Assisted Reprogrammable Nanomechanical Logic Gate," Nano letters, vol. 10, no. 4, pp. 1168-1171, 2010.
16. B. Kosko and S. Mitaim, "Stochastic Resonance in Noisy Threshold Neurons," Neural Networks, vol. 16, no. 5-6, pp. 755-761, June-July 2003.
17. B. Kosko, Noise. Viking, 2006.
18. I. Lee, X. Liu, C. Zhou, and B. Kosko, "Noise-enhanced detection of subthreshold signals with carbon nanotubes," IEEE Transactions on Nanotechnology, vol. 5, no. 6, pp. 613 -627, Nov. 2006.
19. M. McDonnell, N. Stocks, C. Pearce, and D. Abbott, Stochastic resonance: from suprathreshold stochastic resonance to stochastic signal quantization. Cambridge University Press, 2008.
20. S. Mitaim and B. Kosko, "Adaptive Stochastic Resonance," Proceedings of the IEEE: Special Issue on Intelligent Signal Processing, vol. 86, no. 11, pp. 2152-2183, November 1998.
21. F. Moss, A. Bulsara, and M. Shlesinger, Eds., Journal of Statistical Physics, Special Issue on Stochastic Resonance in Physics and Biology (Proceedings of the NATO Advanced Research Workshop). Plenum Press, January 1993, vol. 70, no. 1/2.
22. A. Patel and B. Kosko, "Stochastic Resonance in Continuous and Spiking Neurons with Levy Noise," IEEE Transactions on Neural Networks, vol. 19, no. 12, December 2008.
23. A. Patel and B. Kosko, "Optimal Mean-Square Noise Benefits in Quantizer-Array Linear Estimation," IEEE Signal Processing Letters, vol. 17, no. 12, pp. 1005 -1009, Dec. 2010.
24. A. Patel and B. Kosko, "Noise benefits in quantizer-array correlation detection and watermark decoding," IEEE Transactions on Signal Processing, vol. 59, no. 2, pp. 488 -505, Feb. 2011.
25. M. Wilde and B. Kosko, "Quantum forbidden-interval theorems for stochastic resonance," J. Phys. A: Math. Theor., no. 46, 2009.
26. T. Cover and J. Thomas, Elements of Information Theory, 1st ed. Wiley & Sons, New York, 1991.
27. A. P. Dempster, N. M. Laird, and D. B. Rubin, "Maximum Likelihood from Incomplete Data via the EM Algorithm," Journal of the Royal Statistical Society, Series B, vol. 39, no. 1, pp. 1-38, January 1977.
28. C. F. J. Wu, "On the Convergence Properties of the EM Algorithm," The Annals of Statistics, vol. 11, no. 1, pp. 95-103, 1983.
29. T. Orchard and M.Woodbury, "A missing information principle: theory and applications," in Proceedings of the 6th Berkeley Symposium on Mathematical Statistics and Probability, vol. 1, 1972, pp. 697-715.
30. G. Celeux and J. Diebolt, "The SEM algorithm: a probabilistic teacher algorithm derived from the EM algorithm for the mixture problem," Computational Statistics Quarterly, vol. 2, no. 1, pp. 73-82, 1985.
31. M. T. Tan, G. Tian, and K. W. Ng, Bayesian Missing Data Problems: EM, Data Augmentation and Noniterative Computation. CRC Press, 2010.
32. M. Gupta and Y. Chen, "Theory and Use of the EM Algorithm," Foundations and Trends in Signal Processing, vol. 4, no. 3, pp. 223-296, 2010.
33. M. Á. Carreira-Perpiñán, "Gaussian mean shift is an EM algorithm," IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 29, p. 2007, 2005.
34. R. A. Redner and H. F. Walker, "Mixture Densities, Maximum Likelihood and the EM algorithm," SIAM Review, vol. 26, no. 2, pp. 195-239, 1984.
35. V. Hasselblad, "Estimation of Parameters for a Mixture of Normal Distributions," Technometrics, vol. 8, no. 3, pp. 431-444, 1966.
36. P. Billingsley, Probability and Measure, 3rd ed. John Wiley & Sons, 1995.
37. R. Durrett, Probability: Theory and Examples, 4th ed. Cambridge U Press, 2010.
38. W. Feller, An Introduction to Probability Theory and Its Applications. John Wiley & Sons, 1966, vol. II.
39. G. B. Folland, Real Analysis: Modern Techniques and Their Applications, 2nd ed. Wiley-Interscience, 1999.