Causal network inference by optimal causation entropy

SIAM Journal on Applied Dynamical Systems

Volumn 14, Issue 1, 2015, Pages 73-106

  Sun, Jie ^a   Taylor, Dane ^b,c   Bollt, Erik M ^a  

^a CLARKSON UNIVERSITY   (United States)

^b Statistical and Applied Mathematical Sciences Institute USA   (United States)

^c University of North Carolina at Chapel Hill   (United States)

Author keywords

Causal network inference; Optimal causation entropy; Stochastic network dynamics

Indexed keywords

ALGORITHMS; COMPUTATION THEORY; COMPUTATIONAL EFFICIENCY; DYNAMICAL SYSTEMS; ENTROPY; INFORMATION THEORY; MARKOV PROCESSES; NUMERICAL METHODS; STOCHASTIC SYSTEMS; TIME SERIES;

CAUSAL NETWORK; COUPLED DYNAMICAL SYSTEMS; INFORMATION DIFFUSION; MATHEMATICAL THEORY; NETWORK CHARACTERISTICS; STATIONARY MARKOV PROCESS; STOCHASTIC NETWORKS; UNDERLYING NETWORKS;

INFERENCE ENGINES;

EID: 84925293776     PISSN: None     EISSN: 15360040     Source Type: Journal    
DOI: 10.1137/140956166     Document Type: Article

References (79)

1. N. A. AHMED AND D. V. GOKHALE, Entropy expressions and their estimators for multivariate distributions, IEEE Trans. Inform. Theory, 35 (1989), pp. 688-692.
2. N. ANCONA, D. MARINAZZO, AND S. STRAMAGLIA, Radial basis function approach to nonlinear Granger causality of time series, Phys. Rev. E, 70 (2004), 056221.
3. L. BARNETT, A. B. BARRETT, AND A. K. SETH, Granger causality and transfer entropy are equivalent for Gaussian variables, Phys. Rev. Lett., 103 (2009), 238701.
4. M. S. BARTLETT, The frequency goodness of fit for probability chains, Math. Proc. Cambridge Philos. Soc., 47 (1951), pp. 86-95.
5. A. Y. BARRAUD, A numerical algorithm to solve AXA - X = Q, IEEE Trans. Automat. Control, 22 (1977), pp. 883-885.
6. A. BARRAT, M. BARTHELEMY, AND A. VESPIGNANI, Dynamical Processes on Complex Networks, Cambridge University Press, Cambridge, UK, 2008.
7. A. J. BELL, The co-information lattice, in Proceedings of the 4th International Symposium on Independent Component Analysis and Blind Signal Separation (ICA 03), Nara, Japan, 2003, pp. 921-926.
8. B. BOLLOBÁS, Random Graphs, 2nd ed., Academic Press, New York, 2001.
9. E. BOLLT, Synchronization as a process of sharing and transferring information, Internat. J. Bifur. Chaos Appl. Sci. Engrg., 22 (2012), 1250261.
10. P. J. BROCKWELL, Time Series Analysis: Encyclopedia of Statistics in Behavioral Science, John Wiley & Sons, Hoboken, NJ, 2005.
11. D. S. BASSETT AND E. BULLMORE, Small-world brain networks, Neuroscientist, 6 (2006), pp. 512-523.
12. E. BULLMORE AND O. SPORNS, Complex brain networks: Graph theoretical analysis of structural and functional systems, Nat. Rev. Neurosci., 10 (2009), pp. 186-198.
13. N. CHEN, On the approximability of influence in social networks, SIAM J. Discrete Math., 23 (2009), pp. 1400-1415.
14. Y. CHEN, G. RANGARAJAN, J. FENG, AND M. DING, Analyzing multiple nonlinear time series with extended Granger causality, Phys. Lett. A, 324 (2004), pp. 26-35.
15. S. P. CORNELIUS, W. L. KATH, AND A. E. MOTTER, Realistic control of network dynamics, Nat. Commun., 4 (2013), 1942.
16. T. M. COVER AND J. A. THOMAS, Elements of Information Theory, 2nd ed., John Wiley & Sons, Hoboken, NJ, 2006.
17. G. CRACIUN AND M. FEINBERG, Multiple equilibria in complex chemical reaction networks: Semiopen mass action systems, SIAM J. Appl. Math., 70 (2010), pp. 1859-1877.
18. F. DÖRFLER AND F. BULLO, Synchronization and transient stability in power networks and nonuniform Kuramoto oscillators, SIAM J. Control Optim., 50 (2012), pp. 1616-1642.
19. S. N. DOROGOVTSEV, A. V. GOLTSEV, AND J. F. F. MENDES, Critical phenomena in complex networks, Rev. Modern Phys., 80 (2008), 1275.
20. M. L. EATON, Multivariate Statistics: A Vector Space Approach, John Wiley & Sons, New York, 1983.
21. S. FRENZEL AND B. POMPE, Partial mutual information for coupling analysis of multivariate time series, Phys. Rev. Lett., 99 (2007), 204101.
22. J. FRIEDMAN, T. HASTIE, AND R. TIBSHIRANI, Sparse inverse covariance estimation with the graphical lasso, Biostatistics, 9 (2008), pp. 432-441.
23. Q. GAO, X. DUAN, AND H. CHEN, Evaluation of effective connectivity of motor areas during motor imagery and execution using conditional Granger causality, NeuroImage, 54 (2011), pp. 1280-1288.
24. W. R. GARNER, Uncertainty and Structure as Psychological Concepts, John Wiley & Sons, New York, 1962.
25. I. GELFAND, Normierte Ringe, Rech. Math. [Mat. Sbornik] N.S., 9 (51) (1941), pp. 3-24.
26. J. W. GIBBS, Elementary Principles in Statistical Mechanics, Dover, New York, 1960.
27. M. GOLUBITSKY, I. STEWART, AND A. TÖRÖK, Patterns of synchrony in coupled cell networks with multiple arrows, SIAM J. Appl. Dyn. Syst., 4 (2005), pp. 78-100.
28. P. GOOD, Permutation, Parametric and Bootstrap Tests of Hypotheses, Springer, New York, 2005.
29. C. W. J. GRANGER, Investigating causal relations by econometric models and cross-spectral methods, Econometrica, 37 (1969), pp. 425-438.
30. C. W. J. GRANGER, Some recent developments in a concept of causality, J. Econometrics, 39 (1988), pp. 199-211.
31. G. R. GRIMMETT AND D. R. STIRZAKER, Probability and Random Process, 3rd ed., Oxford University Press, Oxford, UK, 2001.
32. S. GUO, A. K. SETH, K. M. KENDRICK, C. ZHOU, AND J. FENG, Partial Granger causality - eliminating exogenous inputs and latent variables, J. Neurosci. Methods, 172 (2008), pp. 79-93.
33. D. W. HAHS AND S. D. PETHEL, Distinguishing anticipation from causality: Anticipatory bias in the estimation of information flow, Phys. Rev. Lett., 107 (2011), 128701.
34. J. J. HECKMAN, Econometric causality, Int. Stat. Rev., 76 (2008), pp. 1-27.
35. F. HEIDER, Social perception and phenomenal causality, Psychol. Rev., 51 (1944), pp. 358-374.
36. R. A. HORN AND C. R. JOHNSON, Matrix Analysis, 2nd ed., Cambridge University Press, Cambridge, UK, 2013.
37. A. KAISER AND T. SCHREIBER, Information transfer in continuous processes, Phys. D, 166 (2002), pp. 43-62.
38. J. KLEINBERG, The small-world phenomenon: An algorithmic perspective, in Proceedings of the 32nd ACM Symposium on Theory of Computing, ACM, New York, 2000, pp. 163-170.
39. O. KINOUCHI AND M. COPELLI, Optimal dynamical range of excitable networks at criticality, Nat. Phys., 2 (2006), 348.
40. A. KRASKOV, H. STÖGBAUER, AND P. GRASSBERGER, Estimating mutual information, Phys. Rev. E, 69 (2004), 066138.
41. O. KUCHAIEV, M. RAŠAJSKI, D. J. HIGHAM, AND N. PRŽULJ, Geometric de-noising of protein-protein interaction networks, PLoS Comput. Biol., 5 (2009), e1000454.
42. D. B. LARREMORE, W. L. SHEW, AND J. G. RESTREPO, Predicting criticality and dynamic range in complex networks: Effects of topology, Phys. Rev. Lett., 106 (2011), 058101.
43. A. LASOTA AND M. C. MACKEY, Chaos, Fractals, and Noise: Stochastic Aspects of Dynamics, 2nd ed., Springer-Verlag, New York, 1994.
44. S. L. LAURITZEN, Graphical Models, Oxford University Press, Oxford, UK, 1996.
45. A. V. MANTZARIS, D. S. BASSETT, N. F. WYMBS, E. ESTRADA, M. A. PORTER, P. J. MUCHA, S. T. GRAFTON, AND D. J. HIGHAM, Dynamic network centrality summarizes learning in the human brain, J. Complex Networks, 1 (2013), pp. 83-92.
46. A. A. MARGOLIN, I. NEMENMAN, K. BASSO, C. WIGGINS, G. STOLOVITZKY, R. D. FAVERA, AND A. CALIFANO, ARACNE: An algorithm for the reconstruction of gene regulatory networks in a mammalian cellular context, BMC Bioinformatics, 7 (Suppl 1) (2006), S7.
47. W. J. MCGILL, Multivariate information transmission, Psychometrika, 19 (1954), pp. 97-116.
48. M. E. J. NEWMAN, The structure and function of complex networks, SIAM Rev., 45 (2003), pp. 167-256.
49. M. E. J. NEWMAN, Networks: An Introduction, Oxford University Press, Oxford, UK, 2010.
50. T. NISHIKAWA AND A. E. MOTTER, Network synchronization landscape reveals compensatory structures, quantization, and the positive effect of negative interactions, Proc. Natl. Acad. Sci. USA, 107 (2010), pp. 10342-10347.
51. M. PALUŠ, V. KOMÁREK, Z. HRNČÍŘ, AND K. ŠTĚRBOVÁ, Synchronization as adjustment of information rates: Detection from bivariate time series, Phys. Rev. E, 63 (2001), 046211.
52. L. PANINSKI, Estimation of entropy and mutual information, Neural Comput., 15 (2003), pp. 1191-1253.
53. J. PEARL, Causality: Models, Reasoning and Inference, 2nd ed., Cambridge University Press, Cambridge, UK, 2009.
54. S. D. PETHEL AND D. W. HAHS, Exact significance test for Markov order, Phys. D, 269 (2014), pp. 42-47.
55. A. POMERANCE, E. OTT, M. GIRVAN, AND W. LOSERT, The effect of network topology on the stability of discrete state models of genetic control, Proc. Natl. Acad. Sci. USA, 106 (2009), pp. 8209-8214.
56. B. RAVOORI, A. B. COHEN, J. SUN, A. E. MOTTER, T. E. MURPHY, AND R. ROY, Robustness of optimal synchronization in real networks, Phys. Rev. Lett., 107 (2011), 034102.
57. K. J. ROTHMAN AND S. GREENLAND, Causation and causal inference in epidemiology, Am. J. Public Health, 95 (2005), pp. S144-S150.
58. H. L. ROYDEN, Real Analysis, 3rd ed., Prentice-Hall, Englewood Cliffs, NJ, 1988.
59. W. J. RUGH, Linear System Theory, Prentice-Hall, Englewood Cliffs, NJ, 1993.
60. J. RUNGE, J. HEITZIG, V. PETOUKHOV, AND J. KURTHS, Quantifying causal coupling strength: A lagspecific measure for multivariate time series related to transfer entropy, Phys. Rev. Lett., 108 (2012), 258701.
61. J. RUNGE, J. HEITZIG, N. MARWAN, AND J. KURTHS, Escaping the curse of dimensionality in estimating multivariate transfer entropy, Phys. Rev. E, 86 (2012), 061121.
62. K. HLAVÁČKOVÁ-SCHINDLERA, M. PALUŠ, M. VEJMELKA, AND J. BHATTACHARYA, Causality detection based on information-theoretic approaches in time series analysis, Phys. Rep., 441 (2007), pp. 1-46.
63. T. SCHREIBER, Measuring information transfer, Phys. Rev. Lett., 85 (2000), 461.
64. C. E. SHANNON, A mathematical theory of communication, Bell System Tech. J., 27 (1948), pp. 379-423, 623-656.
65. D. A. SMIRNOV, Spurious causalities with transfer entropy, Phys. Rev. E, 87 (2013), 042917.
66. P. SPIRTES, C. N. GLYMOUR, AND R. SCHEINES, Causation, Prediction, and Search, 2nd ed., MIT Press, Cambridge, MA, 2000.
67. D. J. STILWELL, E. M. BOLLT, AND D. G. ROBERSON, Sufficient conditions for fast switching synchronization in time-varying network topologies, SIAM J. Appl. Dyn. Syst., 5 (2006), pp. 140-156.
68. G. STOLOVITZKY, D. MONROE, AND A. CALIFANO, Dialogue on Reverse-Engineering Assessment and Methods: The DREAM of high-throughput pathway inference, Ann. New York Acad. Sci., 1115 (2007), pp. 1-22.
69. M. STUDENÝ AND J. VEJNAROVÁ, The multiinformation function as a tool for measuring stochastic dependence, in Learning in Graphical Models, M. I. Jordan, ed., MIT Press, Cambridge, MA, 1998, pp. 261-297.
70. J. SUN AND A. E. MOTTER, Controllability transition and nonlocality in network control, Phys. Rev. Lett., 110 (2013), 208701.
71. J. SUN AND E. M. BOLLT, Causation entropy identifies indirect influences, dominance of neighbors and anticipatory couplings, Phys. D, 267 (2014), pp. 49-57.
72. D. TAYLOR AND J. G. RESTREPO, Network connectivity during mergers and growth: Optimizing the addition of a module, Phys. Rev. E, 83 (2011), 066112.
73. R. TIBSHIRANI, Regression shrinkage and selection via the lasso, J. R. Stat. Soc. Ser. B Stat. Methodol., 58 (1996), pp. 267-288.
74. A. L. TRAUD, E. D. KELSIC, P. J. MUCHA, AND M. A. PORTER, Comparing community structure to characteristics in online collegiate social networks, SIAM Rev., 53 (2011), pp. 526-543.
75. M. VEJMELKA AND M. PALUS, Inferring the directionality of coupling with conditional mutual information, Phys. Rev. E, 77 (2008), 026214.
76. R. VICENTE, M. WIBRAL, M. LINDNER, AND G. PIPA, Transfer entropy - a model-free measure of effective connectivity for the neurosciences, J. Comput. Neurosci., 30 (2011), pp. 45-67.
77. B. WANG, J. SUN, AND A. E. MOTTER, Detecting structural breaks in seasonal time series by regularized optimization, in Safety, Reliability, Risk and Life-Cycle Performance of Structures and Infrastructures, G. Deodatis, B. R. Ellingwood, and D. M. Frangopol, eds., CRC Press, Boca Raton, FL, 2014, pp. 3621-3628.
78. S. WATANABE, Information theoretical analysis of multivariate correlation, IBM J. Res. Develop., 4 (1960), pp. 66-82.
79. D. J. WATTS AND S. H. STROGATZ, Collective dynamics of 'small-world' networks, Nature, 393 (2000), pp. 440-442.