Markov and semi-Markov switching of source appearances for nonstationary independent component analysis

IEEE Transactions on Neural Networks

Volumn 18, Issue 5, 2007, Pages 1326-1342

  Hirayama, Jun Ichiro ^a   Maeda, Shin Ichi ^a   Ishii, Shin ^a  

^a NARA INSTITUTE OF SCIENCE AND TECHNOLOGY   (Japan)

Author keywords

Blind source separation (BSS); Hidden Markov model (HMM); Hidden semi Markov model (HSMM); Independent component analysis (ICA); Variational Bayes (VB) method

Indexed keywords

ALGORITHMS; BAYESIAN NETWORKS; BLIND SOURCE SEPARATION; HIDDEN MARKOV MODELS; TIME SERIES ANALYSIS;

HIDDEN SEMI-MARKOV MODEL; SOURCE SIGNALS; VARIATIONAL BAYES (VB) METHOD;

INDEPENDENT COMPONENT ANALYSIS;

ALGORITHM; ARTICLE; ARTIFICIAL INTELLIGENCE; AUTOMATED PATTERN RECOGNITION; BAYES THEOREM; COMPUTER SIMULATION; METHODOLOGY; PRINCIPAL COMPONENT ANALYSIS; PROBABILITY; REPRODUCIBILITY; SENSITIVITY AND SPECIFICITY; STATISTICAL MODEL; STATISTICS;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; BAYES THEOREM; COMPUTER SIMULATION; MARKOV CHAINS; MODELS, STATISTICAL; PATTERN RECOGNITION, AUTOMATED; PRINCIPAL COMPONENT ANALYSIS; REPRODUCIBILITY OF RESULTS; SENSITIVITY AND SPECIFICITY; STOCHASTIC PROCESSES;

EID: 34548623374     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/TNN.2007.895829     Document Type: Article

References (49)

1. A. Hyvärinen, J. Karhunen, and E. Oja, Independent Component Analysis. New York: Wiley, 2001.
2. E. Moulines, J.-F. Cardoso, and E. Gassiat, "Maximum likelihood for blind separation and deconvolution of noisy signals using mixture models," in Proc. Int. Conf. Acoustics Speech Signal Process., 1997, vol. 5, pp. 3617-3620.
3. A. Cichocki, S. C. Douglas, and S. Amari, "Robust techniques for independent component analysis (ICA) with noisy data," Neurocomputing, vol. 22, no. 1-3, pp. 113-129, 1998.
4. A. Hyvärinen, "Gaussian moments for noisy independent component analysis," IEEE Signal Process. Lett., vol. 6, no. 6, pp. 145-147, Jun. 1999.
5. H. Attias, "Independent factor analysis," Neural Comput., vol. 11, no. 4, pp. 803-851, 1999.
6. M. S. Lewicki and T. J. Sejnowski, "Learning overcomplete representations," Neural Comput., vol. 12, no. 2, pp. 337-365, 2000.
7. S. Maeda, W. Song, and S. Ishii, "Nonlinear and noisy extension of independent component analysis: Theory and its application to a pitch sensation model," Neural Comput., vol. 17, pp. 115-144, 2005.
8. T.-W. Lee, B. Koehler, and R. Orglmeister, "Blind separation of nonlinear mixing models," in Proc. IEEE Int. Workshop Neural Netw. Signal Process., 1997, pp. 406-415.
9. A. Taleb and C. Jutten, "Source separation in post-nonlinear mixtures," IEEE Trans. Signal Process., vol. 47, no. 10, pp. 2807-2820, Oct. 1999.
10. L. B. Almeida, "MISEP-linear and nonlinear ICA based on mutual information," Signal Process., vol. 84, no. 2, pp. 231-245, 2004.
11. P. Gao, W. L. Woo, and S. S. Dlay, "Nonlinear signal separation for multinonlinearity constrained mixing model," IEEE Trans. Neural Netw., vol. 17, no. 3, pp. 796-802, May 2006.
12. S. Amari, T. P. Chen, and A. Cichocki, "Nonholonomic orthogonal learning algorithms for blind source separation," Neural Comput., vol. 12, no. 6, pp. 1463-1484, 2000.
13. A. Hyvärinen, "Independent component analysis in the presence of Gaussian noise by maximizing joint likelihood," Neurocomputing, vol. 22, pp.49-67, 1998.
14. M. Welling and M. Weber, "A constrainted EM algorithm for independent component analysis," Neural Comput., vol. 13, pp. 677-689, 2001.
15. W. D. Penny, R. M. Everson, and S. J. Roberts, "Hidden Markov independent component analysis," in Advances in Independent Component Analysis, M. Girolami, Ed. Berlin, Germany: Springer-Verlag, 2000, pp. 3-22.
16. H. Attias, "Independent factor analysis with temporally structured sources," in Advances in Neural Information Processing Systems. Berlin, Germany: Springer-Verlag, 2000, vol. 12, pp. 386-392.
17. D. T. Pham and J. F. Cardoso, "Blind separation of instantaneous mixtures of nonstationary sources," IEEE Trans. Signal Process., vol. 49, no. 9, pp. 1837-1848, Sep. 2001.
18. R. A. Choudrey and S. J. Roberts, "Bayesian ICA with hidden Markov sources," in Proc. Int. Conf. Independent Component Anal., 2003, pp. 809-814.
19. H. Valpola, M. Harva, and J. Karhunen, "Hierarchical models of variance sources," Signal Process., vol. 84, pp. 267-282, 2004.
20. H. Snoussi and A. Mohammad-Djafari, "Bayesian unsupervised learning for source separation with mixture of Gaussians prior," J. VLSI Signal Process., vol. 3 7, no. 2-3, pp. 263-279, 2004.
21. L. R. Rabiner, "A tutorial on hidden Markov models and selected applications in speech recognition," Proc. IEEE, vol. 77, no. 2, pp. 257-286, Feb. 1989.
22. J. D. Freguson, "Variable duration models for speech," in Proc. Symp. Appl. Hidden Markov Models to Text and Speech, 1980, pp. 143-179.
23. M. Russell and R. Moore, "Explicit modeling of state occupancy in hidden Markov models for automatic speech recognition," in Proc. IEEE Conf. Acoustics Speech Signal Process., 1985, pp. 5-8.
24. S. E. Levinson, "Continuously variable duration hidden Markov models for automatic speech recognition," Comput. Speech Language, vol. 1, no. 1, pp. 29-45, 1986.
25. M. T. Johnson, "Capacity and complexity of HMM duration modeling techniques," IEEE Signal Process. Lett., vol. 12, no. 5, pp. 407-410, May 2005.
26. J. Miskin and D. MacKay, "Ensemble learning for blind source separation," in Independent Component Analysis: Principles and Practice, S. Roberts and R. Everson, Eds. Cambridge, U.K.: Cambridge Univ. Press, 2001, pp. 209-233.
27. R. Choudrey and S. Roberts, "Flexible Bayesian independent component analysis for blind source separation," in Proc. Int. Conf. Independent Component Anal. Signal Separation, 2001, pp. 90-95.
28. K. Chan, T.-W. Lee, and T. J. Sejnowski, "Variational Bayesian learning of ICA with missing data," Neural Comput., vol. 15, pp. 1991-2011, 2003.
29. J. Särelä and R. Vigário, "A Bayesian approach to overlearning in ICA: A comparison study," Lab. Comput. Inf. Science, Helsinki Univ. Technol., Tech. Rep. A70, 2003.
30. J. Särelä and R. Vigário, "Overlearning in marginal distribution-based ICA: Analysis and solutions," J. Mach. Learn. Res., vol. 4, pp. 1447-1469, 2003.
31. H. Attias, "A variational Bayesian framework for graphical models," in Advances in Neural Information Processing Systems 12, S. A. Solla, T. K. Leen, and K. R. Müller, Eds. Cambridge, MA: MIT Press, 2000, pp. 206-212.
32. D. F. Andrews and C. L. Mallows, "Scale mixtures of normal distributions," J. Roy. Statist. Soc. B, vol. 36, pp. 99-102, 1974.
33. H. Attias, "Inferring parameters and structure of latent variable models by variational Bayes," in Proc. 15th Conf. Uncertainty Artif. Intell., 1999, pp. 21-30 [Online]. Available: Citeseer.nj.nec.com/attias99inferring.html
34. A. Dempster, N. Laird, and D. Rubin, "Maximum likelihood from incomplete data via the EM algorithm," J. Roy. Statist. Soc. B, vol. 39, no. 1, pp. 1-38, 1977.
35. R. Neal and G. Hinton, A new view of the EM algorithm that justifies incremental and other variants, in Learning in Graphical Models, M. Jordan, Ed. Cambridge, MA: MIT Press, 1999, pp. 355-368.
36. S. Amari, A. Cichocki, and H. H. Yang, "A new learning algorithm for blind source separation," in Advances in Neural Information Processing Systems. Cambridge, MA: MIT press, 1996, vol. 8, pp. 753-763.
37. R. Gribonval, L. Benaroya, E. Vincent, and C. Févotte, "Proposals for performance measurement in source separation," in Proc. 5th Int. Conf. Independent Component Analysis Blind Signal Separation, 2003, pp. 763-768.
38. E. Wan, A. Nelson, and R. Peterson, "Speech enhancement assessment resource (SPEAR) database," Oregon Grad. Inst. Sci. Technol., Beaverton, OR [Online]. Available: http://ee.ogi.edu/NSEL/
39. C. Févotte, "Bayesian blind separation of audio mixtures with structured priors," in Proc. 14th Euro. Signal Process. Conf., 2006.
40. E. I. George and R. E. McCulloch, "Variable selection via Gibbs sampling," J. Amer. Statist. Assoc., vol. 88, pp. 881-889, 1993.
41. J. Geweke, "Variable selection and model comparison in regression," in Bayesian Statistics 5, J. M. Bernardo, J. O. Berger, A. P. Dawid, and A. F. M. Smith, Eds. Oxford, U.K.: Clarendon, 1996, pp. 609-620.
42. E. I. George and R. E. McCulloch, "Approaches for Bayesian variable selection," Statistica Sinica, vol. 7, no. 2, pp. 339-373, 1997.
43. P. J. Brown, M. Vannucci, and T. Fearn, "Multivariate Bayesian variable selection and prediction," J. Roy. Statist. Soc. B, vol. 60, no. 3, pp. 627-641, 1998.
44. B. A. Olshausen and D. J. Field, "Sparse coding with an overcomplete basis set: A strategy employed by V1?," Vis. Res., vol. 37, pp. 3311-3325, 1997.
45. B. A. Olshausen and K. J. Millman, "Learning sparse codes with a mixture-of-Gaussians prior," in Advances in Neural Information Processing Systems. Cambridge, MA: MIT Press, 2000, vol. 12, pp. 841-884.
46. P. Georgiev, F. Theis, and A. Cichocki, "Sparse component analysis and blind source separation of underdetermined mixtures," IEEE Trans. Neural Netw., vol. 16, no. 4, pp. 992-996, Jul. 2005.
47. P. A. Højen-Sørensen, O. Winther, and L. K. Hansen, "Mean-field approaches to independent component analysis," Neural Comput., vol. 14, no. 4, pp. 889-918, 2002.
48. A. Doucet, N. de Freitas, and N. Gordon, Eds., Sequential Monte Carlo in Practice. New York: Springer-Verlag, 2001.
49. T. Minka, "Bayesian linear Regression," MIT, Cambridge, MA, Tech. Rep., 2000.