Information theory and adaptation

Quantitative Biology: From Molecular to Cellular Systems

Volumn , Issue , 2012, Pages 73-92

  Nemenman, Ilya ^a  

^a EMORY UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84989293004     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.1201/b12676     Document Type: Chapter

References (106)

1. S Kakade and P Dayan. Acquisition in autoshaping. Advances in Neural Information Processing Systems, 12:24, 2000.
2. CR Gallistel and J Gibbon. Time, rate, and conditioning. Psychol Rev, 107(2):289-344, 2000.
3. CR Gallistel, T Mark, A King, and P Latham. The rat approximates an ideal detector of changes in rates of reward: Implications for the law of effect. J Exp Psychol: Anim Behav Process, 27(4):354-72, 2001.
4. CR Gallistel, S Fairhurst, and P Balsam. The learning curve: Implications of a quantitative analysis. Proc Natl Acad Sci USA, 101(36):13124-31, 2004.
5. L. Sugrue, G Corrado, and WT Newsome. Matching behavior and the representation of value in the parietal cortex. Science, 304(5678):1782-7, 2004.
6. P Balsam and CR Gallistel. Temporal maps and informativeness in associative learning. Trends Neurosci, 32(2):73-8, 2009.
7. P. Balsam, M Drew, and CR Gallistel. Time and associative learning. Compar Cognition Behavior Rev, 5:1-22, 2010.
8. S Laughlin. A simple coding procedure enhances a neuron’s information capacity. Z Naturforsch, C, Biosci, 36:910-2, 1981.
9. S Laughlin, R de Ruyter van Steveninck, and Anderson J. The metabolic cost of neural information. Nat Neurosci, 1:36-41, 1998.
10. F. Rieke, D. Warland, R. de Ruyter van Steveninck, and W Bialek. Spikes: Exploring the Neural Code. MIT Press, 1999.
11. N Brenner, W Bialek, and R de Ruyter van Steveninck. Adaptive rescaling optimizes information transmission. Neuron, 26:695, 2000.
12. P Reinagel and R Reid. Temporal coding of visual information in the thalamus. J Neurosci, 20:5392-400, 2000.
13. A. Fairhall, G Lewen, W Bialek, and R de Ruyter van Steveninck. Efficiency and ambiguity in an adaptive neural code. Nature, 412:787, 2001.
14. R. Liu, S Tzonev, S Rebrik, and K Miller. Variability and information in a neural code of the cat lateral geniculate nucleus. J Neurophysiol, 86:2789-806, 2001.
15. J Victor. Binless strategies for estimation of information from neural data. Phys Rev E, 66:51903, 2002.
16. I Nemenman, GD Lewen, W Bialek, and R de Ruyter van Steveninck. Neural coding of natural stimuli: Information at sub-millisecond resolution. PLoS Comput Biol, 4(3):e1000025, 2008.
17. S Forrest and S Hofmeyr. Immunology as information processing. In L Segel and I Cohen, editors, Design Principles for the Immune System and Other Distributed Autonomous Systems. New York: Oxford University Press, 2000.
18. A Arkin. Signal processing by biochemical reaction networks. In J Walleczek, editor, Self-Organized Biological Dynamics and Nonlinear Control: Toward Understanding Complexity, Chaos and Emergent Function in Living Systems. Cambridge, UK: Cambridge University Press, 2000.
19. M Samoilov, A Arkin, and J Ross. Signal processing by simple chemical systems. J Phys Chem A, 106:10205-21, 2002.
20. B Andrews and P Iglesias. An information-theoretic characterization of the optimal gradient sensing response of cells. PLoS Comput Biol, 3(8):e153, 2007.
21. E Ziv, I Nemenman, and CH Wiggins. Optimal signal processing in small stochastic biochemical networks. PLoS One, 2(10):e1077, 2007.
22. G. Tkacik, C Callan, and W Bialek. Information flow and optimization in transcriptional regulation. Proc Natl Acad Sci USA, 105:12265-70, 2008.
23. F Tostevin and PR ten Wolde. Mutual information between input and output trajectories of biochemical networks. Phys Rev Lett, 102:218101, 2009.
24. A Celani and M Vergassola. Bacterial strategies for chemotaxis response. Proc Natl Acad Sci USA, 107:1391-6, 2010.
25. M Skoge, Y Meir, and N Wingreen. Dynamics of cooperativity in chemical sensing among cell-surface receptors. Phys Rev Lett, 107:178101, 2011.
26. R Cheong, A Rhee, CJ Wang, I Nemenman, and A Levchenko. Information transduction capacity of noisy biochemical signaling networks. Science, 334:354-358, 2011.
27. S Still. Information-theoretic approach to interactive learning. Europhys Lett (EPL), 85:28005, 2009.
28. J Paulsson. Summing up the noise in gene networks. Nature, 427:415-8, 2004.
29. N van Kampen. Stochastic Processes in Physics and Chemistry, 3d ed. Amsterdam: North-Holland, 2007.
30. P Dayan and L Abbott. Theoretical Neuroscience: Computational and Mathematical Modeling of Neural Systems. Cambridge, MA: MIT Press, 2005.
31. H Berg. E. coli in motion. New York: Springer, 2004.
32. CE Shannon and W Weaver. A mathematical theory of communication. Urbana: University of Illinois Press, 1949.
33. T Cover and J Thomas. Elements of Information Theory, 2d ed. New York: Wiley-Interscience, 2006.
34. JW Gibbs. The Scientific Papers of J. Willard Gibbs, vol. 1. Woodbridge, CT: Ox Bow Press, 1993.
35. S. Strong, R Koberle, R de Ruyter van Steveninck, and W Bialek. Entropy and information in neural spike trains. Phys Rev Lett, 80(1):197-200, 1998.
36. J Kelly. A new interpretation of information rate. IRE Transactions on Information Theory, 2:185-9, 1956.
37. C Bergstrom and M Lachmann. Shannon information and biological fitness. In Information Theory Workshop, 2004, pages 50-4. IEEE, 2005.
38. E Kussell and S Leibler. Phenotypic diversity, population growth, and information in fluctuating environments. Science, 309:2075-8, 2005.
39. M Donaldson-Matasci, M Lachmann, and C Bergstrom. Phenotypic diversity as an adaptation to environmental uncertainty. Evo Ecology Res, 10:493-515, 2008.
40. O Rivoire and S Leibler. The value of information for populations in varying environments. J Stat Phys, 142:1124-1166, 2011.
41. J Seger and H Brockmann. What Is Bet Hedging?, vol. 4. New York: Oxford University Press, 1987.
42. H Beaumont, J Gallie, C Kost, G Ferguson, and P Rainey. Experimental evolution of bet hedging. Nature, 462:90-93, 2009.
43. S Frank. Natural selection maximizes Fisher information. J Evol Biol, 22:231-44, 2009.
44. N Balaban, J Merrin, R Chait, L Kowalik, and S Leibler. Bacterial persistence as a phenotypic switch. Science, 305:1622-5, 2004.
45. E Kussell, R Kishony, N Balaban, and S Leibler. Bacterial persistence: A model of survival in changing environments. Genetics, 169:1807-14, 2005.
46. H Maamar, A Raj, and D Dubnau. Noise in gene expression determines cell fate in Bacillus subtilis. Science, 317:526-9, 2007.
47. T Cağatay, M Turcotte, M Elowitz, J Garcia-Ojalvo, and G Süel. Architecture-dependent noise discriminates functionally analogous differentiation circuits. Cell, 139:512-22, 2009.
48. CS Wylie, A Trout, D Kessler, and H Levine. Optimal strategy for competence differentiation in bacteria. PLoS Genet, 6:e1001108, 2010.
49. N Bowen, L Walker, L Matyunina, S Logani, K Totten, B Benigno, and J McDonald. Gene expression profiling supports the hypothesis that human ovarian surface epithelia are multipotent and capable of serving as ovarian cancer initiating cells. BMC Med Genomics, 2:71, 2009.
50. A Arkin and J Ross. Statistical construction of chemical reaction mechanisms from measured timeseries. J Phys Chem, 99:9709, 1995.
51. W Bialek, I Nemenman, and N Tishby. Predictability, complexity, and learning. Neural Comput, 13:2409-63, 2001.
52. F Creutzig, A Globerson, and N Tishby. Past-future information bottleneck in dynamical systems. Phys Rev E, 79:041925, 2009.
53. M Srinivasan, S Laughlin, and A Dubs. Predictive coding: A fresh view of inhibiton in retina. Proc Roy Soc B, 216:427-59, 1982.
54. T Hosoya, S Baccus, and M Meister. Dynamic predictive coding by the retina. Nature, 436:71-77, 2005.
55. M Vergassola, E Villermaux, and B Shraiman. “Infotaxis” as a strategy for searching without gradients. Nature, 445:406-9, 2007.
56. G Schwartz, R Harris, D Shrom, and MJ Berry. Detection and prediction of periodic patterns by the retina. Nature Neurosci, 10:552-4, 2007.
57. I Tagkopoulos, Y-C Liu, and S Tavazoie. Predictive behavior within microbial genetic networks. Science, 320:1313-7, 2008.
58. A Mitchell, G Romano, B Groisman, A Yona, E Dekel, M Kupiec, O Dahan, and Y Pilpel. Adaptive prediction of environmental changes by microorganisms. Nature, 460(7252):220-4, 2009.
59. I Nemenman, F Shafee, and W Bialek. Entropy and inference, revisited. In TG Dietterich, S Becker, and Z Gharamani, editors, Advances in Neural Information Processing Systems, vol. 14, 2002.
60. L Paninski. Estimation of entropy and mutual information. Neural Comput, 15:1191-253, 2003.
61. N Tishby, F Pereira, and W Bialek. The information bottleneck method. In B Hajek and RS Sreenivas, editors, Proc 37th Annual Allerton Conference on Communication, Control and Computing, pages 368-77. Urbana: University of Illinois, 1999.
62. N Wiener. Extrapolation, Interpolation, and Smoothing of Stationary Time Series. Cambridge, MA: MIT Press, 1964.
63. H Barlow. Sensory mechanisms, the reduction of redundancy, and intelligence. In D Blake and A Utlley, editors, Proc Symp Mechanization of Thought Processes, vol. 2, page 53774. London: HM Stationery Office, 1959.
64. H Barlow. Possible principles underlying the transformation of sensory messages. In W Rosenblith, editor, Sensory Communication, page 21734. Cambridge, MA: MIT Press, 1961.
65. J Atick and A Redlich. Toward a theory of early visual processing. Neural Comput, 2:30820, 1990.
66. C Marshall. Specificity of receptor tyrosine kinase signaling: Transient versus sustained extracellular signal-regulated kinase activation. Cell, 80:17985, 1995.
67. W Sabbagh Jr, L Flatauer, A Bardwell, and L Bardwell. Specificity of MAP kinase signaling in yeast differentiation involves transient versus sustained MAPK activation. Mol Cell, 8:68391, 2001.
68. G Lahav, N Rosenfeld, A Sigal, N Geva-Zatorsky, A Levine, M Elowitz, and U Alon. Dynamics of the p53-Mdm2 feedback loop in individual cells. Nat Genet, 36:147-50, 2004.
69. L Ma, J Wagner, J Rice, W Hu, A Levine, and G Stolovitzky. A plausible model for the digital response of p53 to DNA damage. Proc Natl Acad Sci USA, 102:14266-71, 2005.
70. P Hersen, M McClean, L Mahadevan, and S Ramanathan. Signal processing by the HOG MAP kinase pathway. Proc Natl Acad Sci USA, 105:7165-70, 2008.
71. C-Y Huang and J Ferrel. Ultrasensitivity in the mitogen-activated protein kinase cascade. Proc Natl Acad Sci USA, 93:10078, 1996.
72. N. Markevich, J Hock, and B Kholodenko. Signaling switches and bistability arising from multisite phosphorylation in protein kinase cascades. J Cell Biol, 164:354, 2004.
73. A Mugler, E Ziv, I Nemenman, and C Wiggins. Quantifying evolvability in small biological networks. IET Syst Biol, 3:379-87, 2009.
74. A Borst, V Flanagin, and H Sompolinsky. Adaptation without parameter change: Dynamic gain control in motion detection. Proc Natl Acad Sci USA, 102:6172, 2005.
75. I Nemenman. Gain control in molecular information processing: Lessons from neuroscience. Phys Biol, Volume 9, Article No. 026003, 2012.
76. P Iglesias. Feedback control in intracellular signaling pathways: Regulating chemotaxis in Dictyostelium discoideum. European J Control, 9:227-36, 2003.
77. R Normann and I Perlman. The effects of background illumination on the photoresponses of red and green cells. J Physiol, 286:491, 1979.
78. D. MacGlashan, S Lavens-Phillips, and K Miura. IgE-mediated desensitization in human basophils and mast cells. Front Biosci, 3:d746-56, 1998.
79. P Detwiler, S Ramanathan, A Sengupta, and B Shraiman. Engineering aspects of enzymatic signal transduction: Photoreceptors in the retina. Biophys J, 79:2801, 2000.
80. C Rao, J Kirby, and A Arkin. Design and diversity in bacterial chemotaxis: A comparative study in Escherichia coli and Bacillus subtilis. PLoS Biol, 2:E49, 2004.
81. T Friedlander and N Brenner. Adaptive response by state-dependent inactivation. Proc Natl Acad Sci USA, 106:22558-63, 2009.
82. D Muzzey, C Gomez-Uribe, J Mettetal, and van Oudenaardem A. A systems-level analysis of perfect adaptation in yeast osmoregulation. Cell, 138:160-71, 2009.
83. A Anishkin and S Sukharev. State-stabilizing interactions in bacterial mechanosensitive channel gating and adaptation. J Biol Chem, 284:19153-7, 2009.
84. V Belyy, K Kamaraju, B Akitake, A Anishkin, and S Sukharev. Adaptive behavior of bacterial mechanosensitive channels is coupled to membrane mechanics. J Gen Physiol, 135:641-52, 2010.
85. U Alon, M Surette, N Barkai, and S Leibler. Robustness in bacterial chemotaxis. Nature, 397:168-71, 1999.
86. C Hansen, R Enders, and N Wingreen. Chemotaxis in Escherichia coli: A molecular model for robust precise adaptation. PLoS Comput Biol, 4:e1, 2008.
87. H Salis, E Mirsky, and C Voigt. Automated design of synthetic ribosome binding sites to control protein expression. Nat Biotech, 27:946-50, 2009.
88. R Endres, O Oleksiuk, C Hansen, Y Meir, V Sourjik, and N Wingreen. Variable sizes of Escherichia coli chemoreceptor signaling teams. Mol Syst Biol, 4:211, 2008.
89. N Kashtan and U Alon. Spontaneous evolution of modularity and network motifs. Proc Natl Acad Sci USA, 102:13773-8, 2005.
90. P Reinagel and S Laughlin. Editorial: Natural stimulus statistics. Network: Comput Neura Syst, 12:237-40, 2001.
91. D Baylor and A Hodgkin. Changes in time scale and sensitivity in turtle photoreceptors. J Physiol, 242:729-58, 1974.
92. B Wark, A Fairhall, and F Rieke. Timescales of inference in visual adaptation. Neuron, 61:750-61, 2009.
93. E Knudsen. Instructed learning in the auditory localization pathway of the barn owl. Nature, 417:322-8, 2002.
94. B Andrews, T-M Yi, and P Iglesias. Optimal noise filtering in the chemotactic response of Escherichia coli. PLoS Comput Biol, 2(11):e154, Nov 2006.
95. RC Yu, CG Pesce, A ColmanLerner, L Lok, D Pincus, E Serra, M Holl, K Benjamin, A Gordon, and R Brent. Negative feedback that improves information transmission in yeast signalling. Nature, 456:75561, 2008.
96. L Yang and P Iglesias. Positive feedback may cause the biphasic response observed in the chemoattractant-induced response of Dictyostelium cells. Syst Control Lett, 55:329-37, 2006.
97. E Sontag. Remarks on feedforward circuits, adaptation, and pulse memory. IET Syst Biol, 4:39-51, 2010.
98. M DeWeese and A Zador. Asymmetric dynamics in optimal variance adaptation. Neural Comp, 10:1179-202, 1998.
99. O Shoval, L Goentoro, Y Hart, A Mayo, and E Sontag a U Alon. Fold-change detection and scalar symmetry of sensory input fields. Proc Natl Acad Sci USA, 107:15995-16000, 2010.
100. M Lazova, T Ahmed, D Bellomo, R Stocker, and T Shimizu. Response rescaling in bacterial chemotaxis. Proc Natl Acad Sci USA, 108:13870-13875, 2011.
101. S Paliwal, P Iglesias, K Campbell, Z Hilioti, A Groisman, and A Levchenko. Mapk-mediated bimodal gene expression and adaptive gradient sensing in yeast. Nature, 446(7131):46-51, 200vv7.
102. I Witten, E Knudsen, and H Sompolinsky. A hebbian learning rule mediates asymmetric plasticity in aligning sensory representations. J Neurophysiol, 100:1067-79, 2008.
103. J Melin and S Quake. Microfluidic large-scale integration: The evolution of design rules for biological automation. Annu Rev Biophys Biomol Struct, 36:213-31, 2007.
104. F Poelwijk, D Kiviet, D Weinreich, and S Tans. Empirical fitness landscapes reveal accessible evolutionary paths. Nature, 445:383-6, 2007.
105. J Mettetal, D Muzzey, C Gomez-Uribe, and A van Oudenaarden. The frequency dependence of osmoadaptation in Saccharomyces cerevisiae. Science, 319:482-4, 2008.
106. I Nemenman. Fluctuation-dissipation theorem and models of learning. Neural Comput, 17:2006-33, 2005.