Structure learning by pruning in independent component analysis

Neurocomputing

Volumn 71, Issue 10-12, 2008, Pages 2281-2290

  Nielsen, Andreas Brinch ^a   Hansen, Lars Kai ^a  

^a TECHNICAL UNIVERSITY OF DENMARK   (Denmark)

Author keywords

Independent component analysis; Pruning; Sparsity; Structure

Indexed keywords

ABSTRACTING; BAYESIAN NETWORKS; DATA REDUCTION; MATHEMATICAL MODELS; SIGNAL PROCESSING;

BAYESIAN INFORMATION CRITERION (BIC); BAYESIAN SPARSIFICATION; PRUNING;

INDEPENDENT COMPONENT ANALYSIS;

ANALYTIC METHOD; ARTICLE; BAYESIAN LEARNING; INDEPENDENT COMPONENT ANALYSIS; MATHEMATICAL COMPUTING; PRIORITY JOURNAL; SIGNAL PROCESSING; SIMULATION;

EID: 44649169676     PISSN: 09252312     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.neucom.2007.09.016     Document Type: Article

References (32)

1. Akaike H. Information theory and the extension of the maximum likelihood principle. Breakthroughs in Statistics vol. 1 (1993), Springer-Verlag 610-624
2. Bell A., and Sejnowski T. An information-maximisation approach to blind separation and blind deconvolution. Neural Comput. 7 6 (1995) 1129-1159
3. Bishop C.M. Neural Networks for Pattern Recognition (1996), Oxford University Press, Oxford, UK
4. Bronstein A.M., Bronstein M.M., Zibulevsky M., and Zeevi Y.Y. Sparse ICA for blind separation of transmitted and reflected images. Int. J. Imaging Sci. Technol. (IJIST) 15 1 (2005) 84-91
5. Cardoso J.-F. Infomax and maximum likelihood for blind source separation. IEEE Signal Process. Lett. 4 4 (1997) 112-114
6. Gorodkin J., Hansen L.K., Krogh A., Svarer C., and Winther O. A quantitative study of pruning by optimal brain damage. Int. J. of Neural Systems 4 (1993) 159-169
7. Gorodkin J., Hansen L.K., Lautrup B., and Solla S.A. Universal distribution of saliencies for pruning in layered neural networks. Int. J. of Neural Systems 8 5-6 (1997) 489-498
8. Goutte C., and Hansen L.K. Regularization with a pruning prior. Neural Networks 10 6 (1997) 1053-1059
9. C. Goutte. On the use of a purning prior for neural networks, in: Neural Networks for Signal Processing VI, 1996, pp. 52-61.
10. Hansen L.K., Larsen J., and Kolenda T. Blind detection of independent dynamic components. IEEE International Conference on Acoustics, Speech, and Signal Processing vol. 5 (2001) 3197-3200
11. Hansen L.K., and Rasmussen C.E. Pruning from adaptive regularization. Neural Computation 6 6 (1994) 1222-1231
12. B. Hassibi, D.G. Stork. Second order derivatives for network prunning: Optimal brain surgeon, in: Advances in Neural Information Processing Systems, vol. 5, Morgan Kaufmann, 1993, pp. 164-171.
13. Z. He, A. Cichocki, K-EVD clustering and Its Applications to Sparse Component Analysis, in: Independent Component Analysis and Blind Signal Separation (ICA), 2006, pp. 164-171.
14. Højen-Sørensen P.A., Winther O., and Hansen L.K. Mean field approaches to independent component analysis. Neural Comput. 14 (2002) 889-918
15. P.O. Hoyer, S. Shimizu, A. Hyvärinen, Y. Kano, A.J. Kerminen, New permutation algorithms for causal discovery using ICA, in: Independent Component Analysis and Blind Source Separation (ICA), 2006, pp. 115-122.
16. Hyvärinen A., Karhunen J., and Oja E. Independent Component Analysis. first ed. (2001), Wiley-Interscience
17. Kaashoek J.F., and van Dijk H.K. Neural network pruning applied to real exchange rate analysis. J. Forecasting 21 8 (2002) 559-577
18. Y. Le Cun, J. Denker, S. Solla. Optimal brain damage, in: Advances in Neural Information Processing Systems, vol. 2, Morgan Kaufmann, 1990. pp. 598-605.
19. MacKay D.J.C. Probable networks and plausible predictions-a review of practical Bayesian methods for supervised neural networks. Network: Comput. Neural Syst. 6 (1995) 469-505
20. H.B. Nielsen, A matlab toolbox for optimization and data fitting, Web, 2006. URL 〈http://www2.imm.dtu.dk/∼hbn/immoptibox/〉.
21. H.-M. Park, J.-H. Lee, S.-H. Oh, S.-Y. Lee, Blind deconvolution with sparse priors on the deconvolution filters, in: Independent Component Analysis and Blind Signal Separation (ICA), 2006, pp. 658-665.
22. M.S. Pedersen, J. Larsen, U. Kjems, On the difference between updating the mixing matrix and updating the separation matrix, in: IEEE International Conference on Acoustics, Speech and Signal Processing, vol. V, Philadelphia, PA, USA, March 2005, pp. 297-300.
23. M.W. Pedersen, L.K. Hansen, J. Larsen, Pruning with generalization based weight saliencies: γ OBD, γ OBS, in: Advances in Neural Information Processing Systems, vol. 8, MIT Press, 1996, pp. 521-527.
24. Ragg T., Braun H., and Landsberg H. A comparative study of neural network optimization techniques. International Conference on Artificial Neural Networks and Genetic Algorithms (1997), Springer, Berlin
25. J.P. Rosca, D. Erdogmus, J.C. Príncipe, S. Haykin (Eds.), Independent Component Analysis and Blind Signal Separation, 6th International Conference, ICA 2006, Charleston, SC, USA, Proceedings, Lecture Notes in Computer Science, vol. 3889, Springer, Berlin, vol. 3889, 2006.
26. Schwarz G. Estimating the dimension of a model. Ann. Statist. 6 2 (1978) 461-464
27. Shimizu S., Hyvarinen A., Kano Y., and Hoyer P.O. Discovery of non-Gaussian linear causal models using ICA. Proceedings of the 21st Annual Conference on Uncertainty in Artificial Intelligence. Arlington, Virginia (2005), AUAI Press 525-553
28. J.B. Tenenbaum, T.L. Griffits, Structure learning in human causal induction, in: Advances in Neural Information Processing Systems, MIT Press, 2000, pp. 59-65.
29. Thodberg H.H. Improving generalization of neural networks through pruning. Int. J. Neural Syst. 1 4 (1991) 317-326
30. Thodberg H.H. Review of Bayesian neural networks with an application to near infrared spectroscopy. IEEE Trans. Neural Networks 7 1 (1996) 56-72
31. Tibshirani R. Regression shrinkage and selection via the Lasso. J. Roy. Statist. Soc. B 58 1 (1996) 267-288
32. Tipping M.E. Sparse Bayesian learning and the relevance vector machine. J. Mach. Res. 1 (2001) 211-244