Symmetry breaking in soft clustering decoding of neural codes

IEEE Transactions on Information Theory

Volumn 56, Issue 2, 2010, Pages 901-927

  Parker, Albert E ^a,b   Dimitrov, Alexander G ^a   Gedeon, Tomáš ^a  

^a MONTANA STATE UNIVERSITY   (United States)

^b UNIVERSITY OF AUCKLAND   (New Zealand)

Author keywords

Annealing; Bifurcations; Clustering; Information distortion; Neural coding; Symmetry breaking

Indexed keywords

ANNEALING PROCEDURES; BIFURCATION THEORY; CLUSTERING INFORMATION; CODEBOOKS; DISTORTION MEASURES; DISTORTION METHOD; HIGH-DIMENSIONAL; INFORMATION DISTORTION; LOCAL INFORMATION; LOCAL SOLUTION; NEURAL CODE; NEURAL CODING; NEURAL ENSEMBLES; NEURAL RESPONSE; NON-LINEAR OPTIMIZATION PROBLEMS; PITCHFORK BIFURCATIONS; RATE DISTORTION CURVES; SADDLE NODE BIFURCATION; SENSORY STIMULUS; SOFT CLUSTERING; SUB-CRITICAL BIFURCATIONS; SYMMETRY-BREAKING; SYMMETRY-BREAKING BIFURCATIONS;

ANNEALING; OPTIMIZATION;

BIFURCATION (MATHEMATICS);

EID: 77649323902     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIT.2009.2037045     Document Type: Article

References (69)

1. F. Rieke, D. Warland, R. R. de Ruyter van Steveninck, and W. Bialek, Spikes: Exploring the Neural Code. Cambridge, MA: The MIT Press, 1997.
2. R. E. Kass, V. Ventura, and E. N. Brown, "Statistical issues in the analysis of neural data," J. Neurophys., vol. 94, pp. 8-25, 2005.
3. L. Paninski, J. Pillow, and E. Simoncelli, "Maximum likelihood estimation of a stochastic integrate-and-fire neural model," Neur. Comp., vol. 17, pp. 1480-1507, 2005.
4. J. W. Pillow, J. Shlens, L. Paninski, A. Sher, A. Litke, E. Simoncelli, and E. Chichilnisky, "Spatio-temporal correlations and visual signaling in a complete neuronal population," Nature, vol. 454, pp. 995-999, 2008.
5. C. E. Shannon, "A mathematical theory of communication," Bell Sys. Tech. J., vol. 27, pp. 623-656, 1948.
6. H. B. Barlow, "Possible princilples underlying the transformation of sensory messages," in Sensory Communications, W. A. Rosenblith, Ed. Cambridge, MA: MIT Press, 1961.
7. T. W. Kjaer, J. A. Hertz, and B. J. Richmond, "Decoding cortical neuronal signals: Network models, information estimation and spatial tuning," J. Comp. Neurosci, vol. 1, no. 1-2, pp. 109-139, 1994.
8. T. Cover and J. Thomas, Elements of Information Theory, ser. Wiley Series in Communication. New York: Wiley, 1991.
9. N. Slonim and N. Tishby, "Agglomerative information bottleneck," in Advances in Neural Information Processing Systems, S. A. Solla, T. K. Leen, and K.-R. Müller, Eds. Cambridge, MA: MIT Press, 2000, vol. 12, pp. 617-623.
10. N. Tishby, F. C. Pereira, and W. Bialek, "The information bottleneck method," in Proc. 37th Annu. Allerton Conf. Commun., Contr., Comput., 1999.
11. A. G. Dimitrov and J. P. Miller, "Neural coding and decoding: Communication channels and quantization," Network: Computat. Neural Syst., vol. 12, no. 4, pp. 441-472, 2001.
12. T. Gedeon, A. E. Parker, and A. G. Dimitrov, "Information distortion and neural coding," Canadian Appl. Math. Quart., vol. 10, no. 1, pp. 33-70, 2003.
13. R. M. Gray, Entropy and Information Theory. New York: Springer-Verlag, 1990.
14. K. Rose, "Deteministic annealing for clustering, compression, classification, regression, and related optimization problems," Proc. IEEE, vol. 86, no. 11, pp. 2210-2239, 1998.
15. W. Bialek, R. R. de Ruyter van Steveninck, and N. Tishby, "Efficient representation as a design principle for neural coding and computation," in Proc. 2006 IEEE Int. Symp. Inf. Theory, 2006, pp. 659-663.
16. G. Chechick, A. Globerson, N. Tishby, M. Anderson, E. D. Young, and I. Nelken, "Group redundancy measures reveals redundancy reduction in the auditory pathway," in Advances in Neural Information Processing Systems, T. G. Dietterich, S. Becker, and Z. Ghahramani, Eds. Cambridge, MA: MIT Press, 2002, vol. 14.
17. A. G. Dimitrov and J. P. Miller, "Analyzing sensory systems with the information distortion function," in Proc. Pacific Symp. Biocomput. 2001, R. B. Altman, Ed., 2000.
18. A. G. Dimitrov, J. P. Miller, Z. Aldworth, T. Gedeon, and A. E. Parker, "Analysis of neural coding through quantization with an information-based distortion measure," Network: Computat. Neural Syst., vol. 14, pp. 151-176, Feb. 2003.
19. A. G. Dimitrov, J. P. Miller, Z. Aldworth, and T. Gedeon, "Non-uniform quantization of neural spike sequences through an information distortion measure," Neurocomputing, vol. 38-40, pp. 175-181, 2001.
20. B. Mumey, A. Sarkar, T. Gedeon, A. G. Dimitrov, and J. P. Miller, "Finding neural codes using random projections," Neurocomputing, vol. 58-60, pp. 19-25, 2004.
21. E. Schneidman, N. Brenner, N. Tishby, R. R. de Ruyter van Steveninck, and W. Bialek, "Universality and individuality in a neural code," in Proc. NIPS, 2000, pp. 159-165 [Online]. Available: citeseer.ist.psu. edu/305279.html
22. U. Alon, N. B. D. A. Notterman, K. Gish, S. Ybarra, D. Mack, and A. J. Levine, "Broad patterns of gene expression revealed by clustering analysis of tumor and normal colon tissues probed by oligonucleotide arrays," in Proc. Nat. Acad. Sci., 1999.
23. L. Chen, "Multiple protein structure alignment by deterministic annealing," Aug. 2003, pp. 609-610.
24. L. Chen, T. Zhou, and Y. Tang, "Protein structure alignment by deterministic annealing," Bioinformatics, vol. 21, no. 1, pp. 51-62, 2005.
25. R. M. Hecht and N. Tishby, "Extraction of relevant speech features using the information bottleneck method," in Proc. InterSpeech 2005, Lisbon, Portugal, 2005.
26. K.-M. Lee, T.-S. Chung, and J.-H. Kim, "Global optimization of clusters in gene expression data of dna microarrays by deterministic annealing," Genom. Inf., vol. 1, no. 1, pp. 20-24, 2003.
27. S. O'Rourke, G. Chechik, R. Friedman, and E. Eskin, "Discrete profile comparison using information bottleneck," BMC Bioinformatics vol. 7, pp. S8-, 2006 [Online]. Available: http://www.biomedcentral.com/1471- 2105/7/s1/s8
28. S. F. Taylor, N. Tishby, and W. Bialek, "Information and fitness," arXiv:0712.4382v1 [q-bio.PE], Dec. 2007.
29. A. Zhang, Advanced Analysis of Gene Expression Microarray Data. Singapore: World Scientific Publishing, 2006.
30. P. Andritsos, R. Miller, and P. Tsaparas, "Information-theoretic tools for mining database structure from large data sets," in Proc. 2004 ACM SIGMOD Int. Conf. Manage. Data, New York, NY, 2004, pp. 731-742 [Online]. Available: http://doi.acm.org/10.1145/1007568.1007650
31. n-invariant constrained optimization problems," J. Dynam. Diff. Equations, vol. 16, no. 3, pp. 629-678, July 2004.
32. E. T. Jaynes, "On the rationale of maximum-entropy methods," Proc. IEEE, vol. 70, pp. 939-952, 1982.
33. M. Kanou and T. A. Shimozawa, "Threshold analysis of cricket cercal interneurons by an alternating air-current stimulus," J. Comp. Physiol. A, vol. 154, pp. 357-365, 1984.
34. J. P. Miller, G. A. Jacobs, and F. E. Theunissen, "Representation of sensory information in the cricket cercal sensory system. I. Response properties of the primary interneurons," J. Neurophys., vol. 66, pp. 1680-1689, 1991.
35. F. Theunissen, J. C. Roddey, S. Stufflebeam, H. Clague, and J. P. Miller, "Information theoretic analysis of dynamical encoding by four primary sensory interneurons in the cricket cercal system," J. Neurophy., vol. 75, pp. 1345-1359, 1996.
36. F. E. Theunissen and J. P. Miller, "Representation of sensory information in the cricket cercal sensory system. II. Information theoretic calculation of system accuracy and optimal tuning curve width of four primary interneurons," J. Neurophysiol., vol. 66, pp. 1690-1703, 1991.
37. D. Bodnar, J. P. Miller, and G. A. Jacobs, "Anatomy and physiology of identified wind-sensitive local interneurons in the cricket cercal sensory system," J. Comp. Physiol. A, vol. 168, pp. 553-564, 1991.
38. G. A. Jacobs and R. K. Murphey, "Segmental origins of the cricket giant interneuron system," J. Comp. Neurol., vol. 265, pp. 145-157, 1987.
39. H. Clague, F. Theunissen, and J. P. Miller, "The effects of adaptation on neural coding by primary sensor interneurons in the cricket cercal system," J. Neurophysiol., vol. 77, pp. 207-220, 1997.
40. J. C. Roddey, B. Girish, and J. P. Miller, "Assessing the performance of neural encoding models in the presence of noise," J. Comp. Neurosci., vol. 8, pp. 95-112, 2000.
41. D. S. Reich, F. Mechler, K. P. Purpura, and J. D. Victor, "Interspike intervals, receptive fields, and information encoding in primary visual cortex," J. Neurosci., vol. 20, pp. 1964-1974, 2000.
42. J. D. Victor, "Binless strategies for estimation of information from neural data," Phys. Rev. E., vol. 66, pp. 051903-, 2002.
43. I. S. Dhillon, S. Mallela, and D. S. Modha, "Information-theoretic co-clustering," in Proc. The Ninth ACM SIGKDD Int. Conf. Knowledge Discovery Data Mining (KDD 03), 2003.
44. N. Slonim, "The Information Bottleneck: Theory and Applications," Ph.D. dissertation, Hebrew University, Jerusalem, Israel, 2002.
45. M. Golubitsky, I. Stewart, and D. G. Schaeffer, Singularities and Groups in Bifurcation Theory II. New York: Springer Verlag, 1988.
46. D. S. Dummit and R. M. Foote, Abstract Algebra. Englewood Cliffs, NJ: Prentice Hall, 1991.
47. G. James and A. Kerber, The Representation Theory of the Symmetric Group, ser. The Encyclopedia of Mathematics and Applications, G.-C. Rota, Ed. Reading, MA: Addison-Wesley, 1981, vol. 16.
48. A. Dimitrov, T. Gedeon, B. Mumey, R. Snider, A. E. Parker, and J. P. Miller, "Derivation of natural stimulus feature set using a data-driven model," in Proc. Int. Conf. Computat. Sci., P. M. A. Sloot, D. Abramson, A. V. Bogdanov, J. Dongarra, A. Y. Zomaya, and Y. E. Gorbachev, Eds., 2003, vol. 2660, Lecture Notes in Computer Science, pp. 337-345.
49. J. Nocedal and S. J. Wright, Numerical Optimization. New York: Springer, 2000.
50. A. Vanderbauwhede, "Local Bifurcation and Symmetry," Habilitation Thesis, Rijksuniversiteit Gent., 1980.
51. G. Cicogna, "Symmetry breakdown from bifurcation," Lettere Al Nuevo Cimento, vol. 31, pp. 600-602, 1981.
52. G. Cicogna, "Bifurcation and symmetries," Bollettino Un. Mat. Ital., pp. 787-796, 1982.
53. M. Golubitsky and D. G. Schaeffer, Singularities and Groups in Bifurcation Theory I. New York: Springer Verlag, 1985.
54. S. H. Strogatz, Nonlinear Dynamics and Chaos: With Applications to Physics, Biology, Chemistry, and Engineering. Cambridge, MA: Perseus Books, 1998.
55. M. Golubitsky and I. Stewart, The Symmetry Perspective: From Equilibrium to Chaos in Phase Space and Physical Space. Boston, MA: Birkhauser Verlag, 2002.
56. J. R. Schott, Matrix Analysis for Statistics. New York: Wiley, 1997.
57. J. Cohen and I. Stewart, "Polymorphism viewed as phenotypic symmetry-breaking," in Non-Linear Phenomena in Biological and Physical Sciences, S. K. Malik, M. K. Chandrashekaran, and N. Pradhan, Eds. New Delhi, India: Indian National Science Academy, 2000, pp. 1-63.
58. I. Stewart, "Self-organization in evolution: A mathematical perspective," Philosoph. Trans. Roy. Soc., vol. 361, pp. 1101-1123, 2003.
59. W. J. Beyn, A. Champneys, E. Doedel, W. Govaerts, Y. A. Kuznetsov, and B. Sandstede, "Numerical continuation and computation of normal forms," in Handbook of Dynamical Systems III. Singapore: World Scientific, 1999.
60. E. Doedel, H. B. Keller, and J. P. Kernevez, "Numerical analysis and control of bifurcation problems in finite dimensions," Int. J. Bifurcat. Chaos, vol. 1, pp. 493-520, 1991.
61. H. B. Keller, "Numerical solutions of bifurcation and nonlinear eigenvalue problems," in Applications of Bifurcation Theory, P. Rabinowitz, Ed. New York: Academic, 1977, pp. 359-384.
62. R. Gilad-Bachrach, A. Navot, and N. Tishby, "An information theoretic tradeoff between complexity and accuracy," COLT, pp. 595-609, 2003.
63. H. S. Witsenhausen and A. D. Wyner, "A conditional entropy bound for a pair of discrete random variables," IEEE Trans. Inf. Theory, vol. IT-21, pp. 493-501, Sep. 1975.
64. B. Mumey and T. Gedeon, "Optimal mutual information quantization is np-complete," in Proc. Neural Inf. Coding (NIC) Workshop, Snowbird, UT, 2003.
65. G. Elidan and N. Friednam, "Learning hidden variable networks: The information bottleneck approach," J. Machine Learning Res., vol. 6, pp. 81-127, 2005.
66. N. Slonim, N. Friedman, and N. Tishby, "Multivariate information bottleneck," Neur. Computat., vol. 18, pp. 1739-1789, 2006.
67. L. Krubitzer and K. Huffman, "Arealization in the neocortex of mammals: Genetic and epigenetic contributions to the phenotype," Brain, Behav. Evol., vol. 55, pp. 322-335, 2000.
68. L. Krubitzer and J. Kaas, "The evolution of the neocortex in mammals: How is phenotypic diversity generated?," Curr. Opin. Neurobiol., vol. 15, pp. 444-453, 2005.
69. D. J. Felleman and D. C. Van Essen, "Distributed hierarchical processing in the primate cerebral cortex," Cereb. Cortex, vol. 1, no. 1, pp. 1-47, 1991.