Whitening as a tool for estimating mutual information in spatiotemporal data sets

Journal of Statistical Physics

Volumn 124, Issue 5, 2006, Pages 1275-1315

  Galka, Andreas ^a,b   Ozaki, Tohru ^a   Bayard, Jorge Bosch ^c   Yamashita, Okito ^d  

^a INSTITUTE OF STATISTICAL MATHEMATICS   (Japan)

^b UNIVERSITY OF KIEL   (Germany)

^c CUBAN NEUROSCIENCE CENTER   (Cuba)

^d ADVANCED TELECOMMUNICATIONS RESEARCH INSTITUTE INTERNATIONAL   (Japan)

Author keywords

Autoregressive modelling; Independent component analysis; Innovations; Maximum likelihood; Mutual information; Spatiotemporal modelling; Time series analysis; Whitening

Indexed keywords

EID: 33750064260     PISSN: 00224715     EISSN: None     Source Type: Journal    
DOI: 10.1007/s10955-006-9131-x     Document Type: Article

References (52)

1. P. E. Cladis and P. Palffy-Muhoray (eds.), Spatio-Temporal Patterns in Nonequilibrium Complex Systems. SFI Studies in the Sciences of Complexity (Addison Wesley Longman, Reading, MA, 1995).
2. L. B. Almeida, F. Lopes da Silva and J. C. Principe (eds.), Spatiotemporal Models in Biological and Artificial Systems, volume 37 of Frontiers in Artificial Intelligence and Applications (IOS Press, Amsterdam, 1997).
3. D. Walgraef, Spatio-temporal Pattern Formation (Springer, Berlin, Heidelberg, New York, 1997).
4. E. Gehrig and O. Hess, Spatio-Temporal Dynamics and Quantum Fluctuations in Semiconductor Lasers, Springer Tracts in Modern Physics, Vol. 189 (Springer, Berlin, Heidelberg, New York, 2003).
5. G. Christakos, Modern Spatiotemporal Geostatistics. Studies in Mathematical Geology, Vol. 6 (Oxford University Press, Oxford, 2000).
6. J. Bascompte and R. V. Solé (eds.), Modelling Spatiotemporal Dynamics in Ecology (Springer, Berlin, Heidelberg, New York, 1998).
7. E. Yago, C. Escera, K. Alho, M. H. Giard and J. M. Serra-Grabulosa, Spatiotemporal dynamics of the auditory novelty-P3 event-related brain potential. Cogn. Brain Res. 16:383-390, 2003.
8. C. Lukas, J. Falck, J. Bartkova, J. Bartek and J. Lukas, Distinct spatiotemporal dynamics of mammalian checkpoint regulators induced by DNA damage. Nature Cell Biol. 5:255-260, 2003.
9. M. Mori, M. Kaino, S. Kanemoto, M. Enomoto, S. Ebata and S. Tsunoyama, Development of advanced core noise monitoring system for BWRS. Prog. Nucl. Energy 43:201-207, 2003.
10. W. Li, Mutual information functions versus correlation functions. J. Stat. Phys. 60:823-837, 1990.
11. P. Hall and S. C. Morton, On the estimation of entropy. Ann. Inst. Statist. Math. 45:69-88, 1993.
12. D. R. Brillinger, Second-order moments and mutual information in the analysis of time series. In Y. P. Chaubey (ed.), Recent Advances in Statistical Methods, pp. 64-76 (Imperial College Press, London, 2002).
13. D. R. Brillinger, Some data analyses using mutual information. Brazilian J. Prob. Statist. 18:163-183, 2004.
14. K. Kaneko, Spatiotemporal chaos in one- and two-dimensional coupled map lattices. Physica D 37:60-82, 1989.
15. C. E. Shannon, A mathematical theory of communication. Bell Syst. Tech. J. 27:379-423, 623-656, 1948.
16. L. Boltzmann, Über die Beziehung zwischen dem zweiten Hauptsatze der mechanischen Wärmetheorie und der Wahrscheinlichkeitsrechnung respektive den Sätzen über das Wärmegleichgewicht. Wiener Berichte 76:373-435, 1877.
17. H. Akaike, On the likelihood of a time series model. The Statistician 27:217-235, 1978.
18. T. M. Cover and J. A. Thomas, Elements of Information Theory (Wiley, New York, 1991).
19. Y. Moon, B. Rajagopalan and U. Lall, Estimation of mutual information using kernel density estimators. Phys. Rev. E 52:2318-2321, 1995.
20. D. Prichard and J. Theiler, Generalized redundancies for time series analysis. Physica D 84:476493, 1995.
21. A. Kraskov, Harald Stögbauer and P. Grassberger, Estimating mutual information. Phys. Rev. E 69:066138, 2004.
22. R. Moddemeijer, A statistic to estimate the variance of the histogram based mutual information estimator based on dependent pairs of observations. Signal Proc. 75:51-63, 1999.
23. H. Stögbauer, A. Kraskov, S. A. Astakhov and P. Grassberger, Least-dependent-component analysis based on mutual information. Phys. Rev. E 70:066123, 2004.
24. K. Ito, On stochastic differential equations. Mem. Am. Math. Soc. 4:1-51, 1951.
25. J. L. Doob, Stochastic Processes (Wiley, New York, 1953).
26. A. H. Jazwinski, Stochastic Processes and Filtering Theory (Academic Press, San Diego, 1970).
27. T. Kailath, A view of three decades of linear filtering theory. IEEE Trans. Inf. Theory 20:146-181, 1974.
28. P. Levy, Sur une classe de courbes de l'espace de Hilbert et sur une équation intégrale non linéaire. Ann. Sci. École Norm. Sup. 73:121-156, 1956.
29. P. Protter, Stochastic Integration and Differential Equations (Springer-Verlag, Berlin, 1990).
30. P. A. Frost and T. Kailath, An innovation approach to least squares estimation - part III: Nonlinear estimation in white gaussian noise. IEEE Trans. Autom. Contr. 16:217-226, 1971.
31. B. V. Gnedenko, The Theory of Probability (Mir Publishers, Moscow, 1969).
32. G. E. P. Box and G. M. Jenkins, Time Series Analysis, Forecasting and Control 2nd edition (Holden-Day, San Francisco, 1976).
33. R. Moddemeijer, On estimation of entropy and mutual information of continuous distributions. Signal Proc. 16:233-248, 1989.
34. A. Galka and T. Ozaki, Testing for nonlinearity in high-dimensional time series from continuous dynamics. Physica D 158:32-44, 2001.
35. H. Akaike, Prediction and entropy. In A. C. Atkinson and S. E. Fienberg (eds.), A Celebration of Statistics, pp. 1-24 (Springer, Berlin, Heidelberg, New York, 1985).
36. J. Kuha, AIC and BIC: Comparisons of assumptions and performance. Sociol. Methods Res. 33:188-229, 2004.
37. M. Stone, An asymptotic equivalence of choice of model by cross-validation and Akaike's criterion. J. Roy. Statist. Soc. B 39:44-47, 1977.
38. N. Levinson, The Wiener rms error criterion in filter design and prediction. J. Math. Phys. 25:261-278, 1947.
39. P. Whittle, On the fitting of multivariate autoregressions, and the approximate canonical factorization of a spectral density matrix. Biometrika 50:129-134, 1963.
40. A. Neumaier and T. Schneider, Estimation of parameters and eigenmodes of multivariate autoregressive models. ACM Trans. Math. Softw. 27:27-57, 2001.
41. C. Penland, Random forcing and forecasting using principal oscillation pattern analysis. Monthly Weather Rev. 117:2165-2185, 1989.
42. A. Galka, O. Yamashita, T. Ozaki, R. Biscay and P. A. Valdés-Sosa, A solution to the dynamical inverse problem of EEC generation using spatiotemporal Kalman filtering. NeuroImage 23:435-453, 2004.
43. A. Galka, O. Yamashita and T. Ozaki, GARCH modelling of covariance in dynamical estimation of inverse solutions. Phys. Lett. A 333:261-268, 2004.
44. J. Geweke, Inference and causality in economic time series models. In Z. Grilliches and M. Intriligator (eds.), Handbook of Econometrics, pp. 1101-1144 (North-Holland, Amsterdam, 1984).
45. A. Buse, The likelihood ratio, Wald, and Lagrange multiplier test: An expository note. Am. Stat. 36:153-157, 1982.
46. A. Hyvärinen, J. Karhunen and E. Oja, Independent Component Analysis (Wiley, New York, 2001).
47. L. Molgedey and H. G. Schuster, Separation of a mixture of independent signals using time delayed correlations. Phys. Rev. Lett. 72:3634-3637, 1994.
48. T. Schreiber, Spatio-temporal structure in coupled map lattices: two-point correlations versus mutual information. J. Phys. A: Math. Gen. 23:L393-L398, 1990.
49. H. Wu and J. Principe, Generalized anti-Hebbian learning for source separation. In Proceedings of ICASSP'99, pp. 1073-1076 (IEEE Press, Piscataway, NJ, 1999).
50. S. Choi, A. Cichocki and S. Amari, Flexible independent component analysis. J. VLSI Signal Process 26:25-38, 2000.
51. K. G. Jöreskog, Some contributions to maximum likelihood Factor Analysis. Psychometrika 32:443-482, 1967.
52. B. Flury, A First Course in Multivariate Statistics (Springer, New York, Berlin, Heidelberg, 1997).