Pursuit of food versus pursuit of information in a Markovian perception-action loop model of foraging

Journal of Theoretical Biology

Volumn 304, Issue , 2012, Pages 235-272

  Agarwala, Edward K ^a   Chiel, Hillel J ^a,b,c   Thomas, Peter J ^a,b  

^a CASE WESTERN RESERVE UNIVERSITY   (United States)

^b CASE WESTERN RESERVE UNIVERSITY SCHOOL OF MEDICINE   (United States)

^c Case Western Reserve University   (United States)

Author keywords

Causal bayesian network; Ideal observer; Infotaxis; Markov decision process; Perception action cycle

Indexed keywords

BAYESIAN ANALYSIS; DECISION MAKING; DNA; FORAGING BEHAVIOR; MARKOV CHAIN; OPTIMIZATION; PERCEPTION;

ACTION; ARTICLE; BEHAVIOR; DECISION MAKING; FOOD; FORAGING BEHAVIOR; INFORMATION; MATHEMATICAL ANALYSIS; MOTOR PERFORMANCE; PERCEPTION; PRIORITY JOURNAL; PROBABILITY; RECEPTIVE FIELD; SURVIVAL; TASK PERFORMANCE; THEORETICAL MODEL;

ALGORITHMS; ANIMALS; BEHAVIOR, ANIMAL; ECOSYSTEM; FEEDING BEHAVIOR; INFORMATION SEEKING BEHAVIOR; INFORMATION THEORY; MARKOV CHAINS; MODELS, BIOLOGICAL; MOTION PERCEPTION; MOVEMENT;

EID: 84860225349     PISSN: 00225193     EISSN: 10958541     Source Type: Journal    
DOI: 10.1016/j.jtbi.2012.02.016     Document Type: Article

References (133)

1. Adami C. Introduction to Artificial Life 1998, Springer-Verlag.
2. Agarwala, E.K., June 2009. Food for Thought: When Information Optimization Fails to Optimize Utility. Master's Thesis. Case Western Reserve University, Cleveland, OH.
3. Alon U., Surette M.G., Barkai N., Leibler S. Robustness in bacterial chemotaxis. Nature 1999, 397(6715):168-171.
4. Andrews B.W., Iglesias P.A. An information-theoretic characterization of the optimal gradient sensing response of cells. PLoS Comput. Biol. 2007, 3(8):e153.
5. Andrews B.W., Yi T.-M., Iglesias P.A. Optimal noise filtering in the chemotactic response of Escherichia coli. PLoS Comput. Biol. 2006, 2(11):e154.
6. Arimoto S. An algorithm for calculating the capacity of an arbitrary discrete memoryless channel. IEEE Trans. Inf. Theory IT 1972, 18:14-20.
7. Ashby W.R. An Introduction to Cybernetics 1956, Chapman & Hall LTD.
8. Attneave F. Some informational aspects of visual perception. Psychol. Rev. 1954, 61:183-193.
9. Ay N., Wennekers T. Dynamical properties of strongly interacting markov chains. Neural Networks 2003, 16(10):1483-1497.
10. Balsam P.D., Drew M.R., Gallistel C.R. Time and associative learning. Comp. Cogn. Behav. Rev. 2010, 5:1-22.
11. Balsam P.D., Gallistel C.R. Temporal maps and informativeness in associative learning. Trends Neurosci. 2009, 32(2):73-78.
12. Barlow H.B. Possible principles underlying the transformation of sensory messages. Sensory Communications 1961, MIT Press.
13. Barnett L., Barrett A.B., Seth A.K. Granger causality and transfer entropy are equivalent for gaussian variables. Phys. Rev. Lett. 2009, 103:238701.
14. Bell A.J. Levels and loops: the future of artificial intelligence and neuroscience. Philos. Trans. R. Soc. Lond. B Biol. Sci. 1999, 354(1392):2013-2020.
15. Bellman R.E. A Markovian decision process. J. Math. Mech. 1957, 6(5):679-684.
16. Bergstrom C.T., Lachmann M. Shannon information and biological fitness. Information Theory Workshop October 2004, 50-54. IEEE.
17. Bialek W., Nemenman I., Tishby N. Predictability, complexity, and learning. Neural Comput. 2001, 13(11):2409-2463.
18. Blackwell D. Finite non-homogeneous chains. Ann. Math. 1945, 46(4):594-599.
19. Blahut R. Computation of channel capacity and rate distortion functions. IEEE Trans. Inf. Theory IT 1972, 18:460-473.
20. Bourne H.R., Weiner O. A chemical compass. Nature 2002, 419(6902):21.
21. Bray D., Bourret R.B., Simon M.I. Computer simulation of the phosphorylation cascade controlling bacterial chemotaxis. Mol. Biol. Cell 1993, 4(5):469-482.
22. Brémaud P. Markov chains: Gibbs fields, Monte Carlo simulations, and queues. Texts in Applied Mathematics 1999, vol. 31. Springer.
23. Celani A., Vergassola M. Bacterial strategies for chemotaxis response. Proc. Natl. Acad. Sci. USA 2010, 107(4):1391-1396.
24. Cheong R., Rhee A., Wang C.J., Nemenman I., Levchenko A. Information transduction capacity of noisy biochemical signaling networks. Science 2011, 334(6054):354-358.
25. Chiel H.J., Beer R.D. The brain has a body: adaptive behavior emerges from interactions of nervous system, body and environment. Trends Neurosci. 1997, 20(12):553-557.
26. Clausznitzer D., Endres R.G. Noise characteristics of the Escherichia coli rotary motor. BMC Syst. Biol. 2011, 5(1):151.
27. Cover T.M., Thomas J.A. Elements of Information Theory. Wiley Series in Telecommunications 1991, John Wiley and Sons, Inc., New York.
28. Creutzig F., Globerson A., Tishby N. Past-future information bottleneck in dynamical systems. Phys. Rev. E 2009, 79(4):041925.
29. Creutzig F., Sprekeler H. Predictive coding and the slowness principle: an information-theoretic approach. Neural Comput. 2008, 20(4):1026-1041.
30. de Ruyter van Steveninck R.R., Lewen G.D., Strong S.P., Koberle R., Bialek W. Reproducibility and variability in neural spike trains. Science 1997, 275(5307):1805-1808.
31. Deweese M. Optimization principles for the neural code. Network 1996, 7(2):325-331.
32. Donaldson-Matasci M.C., Bergstrom C.T., Lachmann M. The fitness value of information. Oikos 2010, 119(2):219-230.
33. Doya K. How can we learn efficiently to act optimally and flexibly?. Proc. Natl. Acad. Sci. USA 2009, 106(28):11429-11430.
34. Engel A., van den Broeck C. Statistical Mechanics of Learning 2001, Cambridge University Press.
35. Feng X., Loparo K.A. Active probing for information in control systems with quantized state measurements: a minimum entropy approach. IEEE Trans. Automat. Control 1997, 42(2):216-238.
36. Feng X., Loparo K.A., Fang Y. Optimal state estimation for stochastic systems: an information theoretic approach. IEEE Trans. Automat. Control 1997, 42(6):771-785.
37. Fuller D., Chen W., Adler M., Groisman A., Levine H., Rappel W.-J., Loomis W.F. External and internal constraints on eukaryotic chemotaxis. Proc. Natl. Acad. Sci. USA 2010, 107(21):9656-9659.
38. Fuster J.M. Upper processing stages of the perception-action cycle. Trends Cogn. Sci. 2004, 8(4):143-145.
39. Gallistel C.R., Mark T.A., King A.P., Latham P.E. The rat approximates an ideal detector of changes in rates of reward: implications for the law of effect. J. Exp. Psychol. Anim. Behav. Process 2001, 27(4):354-372.
40. Garthe S., Benvenuti S., Montevecchi W.A. Pursuit plunging by northern gannets (Sula bassana) feeding on capelin (Mallotus villosus). Proc. Biol. Sci. 2000, 267(1454):1717-1722.
41. Gatenby R.A., Frieden B.R. Information theory in living systems, methods, applications, and challenges. Bull. Math. Biol. 2007, 69(2):635-657.
42. Geisler W.S. Chapter 52: Ideal Observer Analysis. The Visual Neurosciences 2004, MIT Press.
43. Globerson A., Stark E., Vaadia E., Tishby N. The minimum information principle and its application to neural code analysis. Proc. Natl. Acad. Sci. USA 2009, 106(9):3490-3495.
44. Green D.M., Swets J.A. Signal Detection Theory and Psychophysics 1966, Wiley, New York.
45. Griffiths B.E., Loparo K.A. Optimal control of jump-linear gaussian systems. Int. J. Control 1985, 42:791-819.
46. Grimmett G., Stirzaker D. Probability and Random Processes 2005, Oxford University Press. third ed.
47. Hajnal J. Weak ergodicity in non-homogeneous Markov chains. Math. Proc. Camb. Philos. Soc. 1958, 54:233-246.
48. Hu B., Chen W., Rappel W.-J., Levine H. Physical limits on cellular sensing of spatial gradients. Phys. Rev. Lett. 2010, 105(4):048104.
49. Hu B., Fuller D., Loomis W.F., Levine H., Rappel W.-J. Phenomenological approach to eukaryotic chemotactic efficiency. Phys. Rev. E Stat. Nonlin. Soft Matter Phys. 2010, 81(3 Part 1):031906.
50. Hu B., Chen W., Levine H., Rappel W.-J. Quantifying information transmission in eukaryotic gradient sensing and chemotactic response. J. Stat. Phys. 2011, 142(6):1167-1186.
51. Hu B., Chen W., Rappel W.-J., Levine H. How geometry and internal bias affect the accuracy of eukaryotic gradient sensing. Phys. Rev. E Stat. Nonlin. Soft Matter Phys. 2011, 83(2 Part 1):021917.
52. Iglesias P.A., Devreotes P.N. Navigating through models of chemotaxis. Curr. Opin. Cell Biol. 2008, 20(1):35-40. (Cell Structure and Dynamics).
53. Insall R., Andrew N. Chemotaxis in Dictyostelium: how to walk straight using parallel pathways. Curr. Opin. Microbiol. 2007, 10(6):578-581.
54. Jindrich D.L., Full R.J. Dynamic stabilization of rapid hexapedal locomotion. J. Exp. Biol. 2002, 205(Part 18):2803-2823.
55. Jordan M.I. Learning in Graphical Models 1998, MIT Press.
56. Jumarie G. Subjectivity, Information, Systems: An Introduction to a Theory of Relativistic Cybernetics 1986, Gordon & Breach Science Pub.
57. Jumarie G. Relative Information: Theories and Applications. Springer Series in Cybernetics August 1990, Springer-Verlag.
58. Kelly J.L. A new interpretation of information rate. IRE Trans. Inf. Theory 1956, 2(3):185-189.
59. Kimmel J.M., Salter R.M., Thomas P.J. An information theoretic framework for eukaryotic gradient sensing. Advances in Neural Information Processing Systems 2007, vol. 19:705-712. MIT Press, Cambridge, MA.
60. King J.S., Insall R.H. Chemotaxis: finding the way forward with Dictyostelium. Trends Cell Biol. 2009, 19(10):523-530.
61. Klyubin A.S., Polani D., Nehaniv C.L. Representations of space and time in the maximization of information flow in the perception-action loop. Neural Comput. 2007, 19(9):2387-2432.
62. Klyubin A.S., Polani D., Nehaniv C.L. Keep your options open: an information-based driving principle for sensorimotor systems. PLoS ONE 2008, 3(12):e4018.
63. Kollmann M., Løvdok L., Bartholomé K., Timmer J., Sourjik V. Design principles of a bacterial signalling network. Nature 2005, 438(7067):504-507.
64. Kramer D.L., McLaughlin R.L. The behavioral ecology of intermittent locomotion. Am. Zoolog. 2001, 41(2):137-153.
65. Kukillaya R., Proctor J., Holmes P. Neuromechanical models for insect locomotion: stability, maneuverability, and proprioceptive feedback. Chaos 2009, 19(2):026107.
66. Lenski R.E., Ofria C., Collier T.C., Adami C. Genome complexity, robustness and genetic interactions in digital organisms. Nature 1999, 400(6745):661-664.
67. Linsker R. Perceptual neural organization: some approaches based on network models and information theory. Annu. Rev. Neurosci. 1990, 13:257-281.
68. Linsker R. Self-organization in a perceptual network. Computer 1988, 21:105-117.
69. Loomis W.F., Insall R.H. A cell for all reasons. Nature 1999, 401(September):440-441.
70. Markin S.N., Klishko A.N., Shevtsova N.A., Lemay M.A., Prilutsky B.I., Rybak I.A. Afferent control of locomotor CPG: insights from a simple neuromechanical model. Ann. NY Acad. Sci. 2010, 1198:21-34.
71. McNamara J.M., Houston A.I. State-dependent life histories. Nature 1996, 380(6571):215-221.
72. Najemnik J., Geisler W.S. Optimal eye movement strategies in visual search. Nature 2005, 434(7031):387-391.
73. Nemenman, I., 2010. Information Theory and Adaptation. Unpublished manuscript. (To appear as In: Wall, M. E., (Ed.), Chapter 5 of Quantitative Biology: From Molecular to Cellular Systems, Taylor and Francis, 2012). http://arxiv:org/abs/1011.5466.
74. Niven J.E., Laughlin S.B. Energy limitation as a selective pressure on the evolution of sensory systems. J. Exp. Biol. 2008, 211(Part 11):1792-1804.
75. O'Brien W.J., Evans B.I., Browman H.I. Flexible search tactics and efficient foraging in saltatory searching animals. Oecologia 1989, 80:100-110.
76. Ofria C., Adami C., Collier T.C. Selective pressures on genomes in molecular evolution. J. Theor. Biol. 2003, 222(4):477-483.
77. Olshausen B.A., Field D.J. Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 1996, 381(6583):607-609.
78. Olshausen B.A., Field D.J. Natural image statistics and efficient coding. Network 1996, 7(2):333-339.
79. Olshausen B., Reinagel P. Editorial: sensory coding in the natural environment. Network: Comput. Neural Syst. 2003, 14(3):1-2.
80. Olshausen B.A., Field D.J. Sparse coding of sensory inputs. Curr. Opin. Neurobiol. 2004, 14(4):481-487.
81. Panzeri S., Senatore R., Montemurro M.A., Petersen R.S. Correcting for the sampling bias problem in spike train information measures. J. Neurophysiol. 2007, 98(3):1064-1072.
82. Parent C.A., Devreotes P.N. A cell's sense of direction. Science 1999, 284(April):765-770.
83. Pettersson J., Ninkovic V., Glinwood R., Al Abassi S., Birkett M., Pickett J., Wadhams L. Chemical stimuli supporting foraging behaviour of Coccinella septempunctata L. (Coleoptera: Coccinellidae): volatiles and allelobiosis. Appl. Entomol. Zool. 2008, 43(3):315-321.
84. Polani D. Information: currency of life?. Human Front. Sci. Prog. J. 2009, 8(September).
85. Polani D., Möller M. Models of information processing in the sensorimotor loop. Information Theory and Statistical Learning 2009, 289-308. Springer, US. F. Emmert-Streib, M. Dehmer (Eds.).
86. Rappel W.-J., Levine H. Receptor noise and directional sensing in eukaryotic chemotaxis. Phys. Rev. Lett. 2008, 100(22):228101.
87. Rappel W.-J., Thomas P.J., Levine H., Loomis W.F. Establishing direction during chemotaxis in eukaryotic cells. Biophys. J. 2002, 83(3):1361-1367.
88. Reinagel P., Laughlin S. Natural stimulus statistics. Network 2001, 12(3):237-240.
89. Reinagel P., Zador A.M. Natural scene statistics at the centre of gaze. Network 1999, 10(4):341-350.
90. Reynolds A.M., Frye M.A. Free-flight odor tracking in Drosophila is consistent with an optimal intermittent scale-free search. PLoS ONE 2007, 2(4):e354.
91. Rickert P., Weiner O.D., Wang F., Bourne H.R., Servant G. Leukocytes navigate by compass: roles of PI3Kgamma and its lipid products. Trends Cell Biol. 2000, 10(11):466-473.
92. Rieke F., Warland D., de Ruyter van Steveninck R., Bialek W. Computational neuroscience. Spikes: Exploring the Neural Code 1999, MIT Press (Bradford Book), Cambridge, MA.
93. Saloff-Coste L., Zúniga J. Convergence of some time inhomogeneous Markov chains via spectral techniques. Stochast. Process. Appl. 2007, 117(8):961-979.
94. Saloff-Coste L., Zúniga J. Merging for time inhomogeneous finite markov chains, part I: singular values and stability. Electron. J. Probab. 2009, 14(49):1456-1494.
95. Saloff-Coste L., Zúniga J. Merging for time inhomogeneous finite markov chains, part II: Nash and log-Sobolev inequalities. Ann. Probab. 2011, 39(3):1161-1203.
96. Schneider T.D. Evolution of biological information. Nucleic Acids Res. 2000, 28(14):2794-2799.
97. Schneider T.D., Stormo G.D., Gold L., Ehrenfeucht A. Information content of binding sites on nucleotide sequences. J. Mol. Biol. 1986, 188:415-431.
98. Schneider T.D. 70% efficiency of bistate molecular machines explained by information theory, high dimensional geometry and evolutionary convergence. Nucleic Acids Res. 2010.
99. Schneidman E., Bialek W., Berry M.J. Synergy, redundancy, and independence in population codes. J. Neurosci. 2003, 23(37):11539-11553.
100. Schreiber S. Measuring information transfer. Phys. Rev. Lett. 2000, 85(2):461-464.
101. Segall J.E., Block S.M., Berg H.C. Temporal comparisons in bacterial chemotaxis. Proc. Natl. Acad. Sci. USA 1986, 83(23):8987-8991.
102. Shannon C.E. A mathematical theory of communication. Bell Syst. Tech. J. 1948, 27(July):379-423. (623-656).
103. Shik M.L., Orlovsky G.N. Neurophysiology of locomotor automatism. Physiol. Rev. 1976, 56(3):465-501.
104. Simoncelli E.P., Olshausen B.A. Natural image statistics and neural representation. Annu. Rev. Neurosci. 2001, 24:1193-1216.
105. Sims D.W., Southall E.J., Humphries N.E., Hays G.C., Bradshaw C.J.A., Pitchford J.W., James A., Ahmed M.Z., Brierley A.S., Hindell M.A., Morritt D., Musyl M.K., Righton D., Shepard E.L.C., Wearmouth V.J., Wilson R.P., Witt M.J., Metcalfe J.D. Scaling laws of marine predator search behaviour. Nature 2008, 451(7182):1098-1102.
106. Slonim N., Friedman N., Tishby N. Multivariate information bottleneck. Neural Comput. 2006, 18(8):1739-1789.
107. Smith E.C., Lewicki M.S. Efficient auditory coding. Nature 2006, 439(7079):978-982.
108. Still S. Information-theoretic approach to interactive learning. Europhys. Lett. 2009, 85(2):1-6. (28005).
109. Strong S.P., de Ruyter van Steveninck R.R., Bialek W., Koberle R. On the application of information theory to neural spike trains. Pac. Symp. Biocomput. 1998, 621-632.
110. Sutton R.S., Barto A.G. Reinforcement Learning: An Introduction. Adaptive Computation and Machine Learning 1998, MIT Press (Bradford Book), Cambridge, MA.
111. Taussky O. A recurring theorem on determinants. Am. Math. Monthly 1949, 56(4):672-676.
112. Taylor, S.F., Tishby, N., Bialek, W., February 2008. Information and fitness. arXiv:0712.4382v1 [q-bio.PE].
113. Theunissen G., Roddey J.C., Stufflebeam S., Clague H., Miller J.P. Information theoretic analysis of dynamical encoding by four identified primary sensory interneurons in the cricket cercal system. J. Neurophysiol. 1996, 75(4):1345-1364.
114. Thomas P.J. Cell signaling: every bit counts. Science 2011, 334(6054):321-322.
115. Thomson E.E., Kristan W.B. Quantifying stimulus discriminability: a comparison of information theory and ideal observer analysis. Neural Comput. 2005, 17(4):741-778.
116. Tishby, N., Pereira, F.C., Bialek, W., 2000. The information bottleneck method. arXiv:physics/0004057v1, 24 April 2000.
117. Tishby N., Polani D. Perception-Reason-Action Cycle: Models, Algorithms and Systems, Information Theory of Decisions and Actions 2010, Springer-Verlag, (Chapter 19).
118. Todorov E. Efficient computation of optimal actions. Proc. Natl. Acad. Sci. USA 2009, 106(28):11478-11483.
119. Touchette S., Lloyd S. Information-theoretic limits of control. Phys. Rev. Lett. 2000, 84(6):1156-1159.
120. Touchette H., Lloyd S. Information-theoretic approach to the study of control systems. Phys. A: Statist. Mech. Appl. 2004, 331(1-2).
121. Tranquillo R.T., Lauffenburger D.A., Zigmond S.H. A stochastic model for leukocyte random motility and chemotaxis based on receptor binding fluctuations. J. Cell Biol. 1988, 106(2):303-309.
122. Van Haastert P.J.M., Veltman D.M. Chemotaxis: navigating by multiple signaling pathways. Sci. STKE 2007, (396):pe40.
123. van Hateren J.H., van der Schaaf A. Independent component filters of natural images compared with simple cells in primary visual cortex. Proc. Biol. Sci. 1998, 265(1394):359-366.
124. Vapnik V.N. The Nature of Statistical Learning Theory 2000, Springer-Verlag.
125. Vergassola M., Villermaux E., Shraiman B.I. 'Infotaxis' as a strategy for searching without gradients. Nature 2007, 445(7126):406-409.
126. Wainwright, M.J., 2002. Stochastic processes on graphs with cycles: geometric and variational approaches. PhD Thesis. Massachusetts Institute of Technology.
127. Wainwright M.J., Jordan M.I. Log-determinant relaxation for approximate inference in discrete Markov random fields. IEEE Trans. Signal Process. 2006, 54(6):2099-2109.
128. Wennekers T., Ay N. Finite state automata resulting from temporal information maximization and a temporal learning rule. Neural Comput. 2005, 17(10):2258-2290.
129. Williams, P.L., Beer, R.D., 2010. Information dynamics of evolved agents. In: From Animals to Animats 11. Lecture Notes in Computer Science, vol. 6226, Paris-Clos Lucé, France, 25-28 August 2010. 11th International Conference on Simulation of Adaptive Behavior, SAB 2010, pp. 38-49.
130. Williams, P.L., Beer, R.D., 2011. Generalized measures of information transfer. arXiv:1102.1507v1, February 2011.
131. Yi T.M., Huang Y., Simon M.I., Doyle J. Robust perfect adaptation in bacterial chemotaxis through integral feedback control. Proc. Natl. Acad. Sci. USA 2000, 97(9):4649-4653.
132. Zimmerman P.A., Mehlotra R.K., Kasehagen L.J., Kazura J.W. Why do we need to know more about mixed Plasmodium species infections in humans?. Trends Parasitol. 2004, 20(9):440-447.
133. Ziv E., Nemenman I., Wiggins C.H. Optimal signal processing in small stochastic biochemical networks. PLoS ONE 2007, 2(10):e1077.