Toward a theory of information processing

Signal Processing

Volumn 87, Issue 6, 2007, Pages 1326-1344

  Sinanović, Sinan ^a   Johnson, Don H ^a  

^a Rice University   (United States)

Author keywords

Information processing; Information theory; Kullback Leibler distance

Indexed keywords

DISTANCE MEASUREMENT; INFORMATION THEORY; SIGNAL ENCODING; SIGNAL PROCESSING;

INFORMATION-THEORETIC DISTANCE MEASURES; KULLBACK-LEIBLER DISTANCE; NEURAL SIGNAL ANALYSIS;

DATA PROCESSING;

EID: 33847217915     PISSN: 01651684     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.sigpro.2006.11.005     Document Type: Article

References (37)

1. Shannon C. A mathematical theory of communication. Bell System Technical J. 379-423 (1948) 623-656
2. Berger T. Rate Distortion Theory (1971), Prentice-Hall, Englewood Cliffs, NJ
3. Kolmogorov A. On the Shannon theory of information transmission in the case of continuous signals. IRE Trans. Info. Th. 3 (1956) 102-108
4. Gastpar M., Rimoldi B., and Vetterli M. To code, or not to code: Lossy source-channel communication revisited. IEEE Trans. Info. Th. 49 (2003) 1147-1158
5. Tishby N., Pereira F., and Bialek W. The information bottleneck method. Proceedings of the 37th Allerton Conference on Communication, Control and Computing (1999) 368-377
6. Shannon C., and Weaver W. The Mathematical Theory of Communication (1949), U. Illinois Press, Urbana, IL
7. Ash R., and Gardner M. Topics in Stochastic Processes (1975), Academic Press, New York
8. Johnson D. Toward a theory of signal processing. Information Theory Workshop on Detection, Estimation, Classification and Imaging. Santa Fe, NM (1999)
9. S. Amari, H. Nagaoka, Methods of information geometry, Translations of Mathematical Monographs, vol. 191, American Mathematical Society, Providence, Rhode Island, 2000.
10. Basseville M. Distance measures for signal processing and pattern recognition. Signal Process. 18 (1989) 349-369
11. Ali S., and Silvey S. A general class of coefficients of divergence of one distribution from another. J. Roy. Statist. Soc., Ser. B 28 (1966) 131-142
12. Chernoff H. Measure of asymptotic efficiency for tests of a hypothesis based on the sum of observations. Ann. Math. Statist. 23 (1952) 493-507
13. Bhattacharyya A. On a measure of divergence between two statistical populations defined by their probability distributions. Bull. Calcutta Math. Soc. 35 (1943) 99-109
14. Kailath T. The divergence and Bhattacharyya distance measures in signal selection. IEEE Trans. Comm. Tech. COM-15 1 (1967) 52-60
15. N. Čencov, Statistical decision rules and optimal inference, Translations in Mathematics, vol. 14, American Mathematical Society, Providence, RI, 1982.
16. A. Dabak, A geometry for detection theory, Ph.D. Thesis, Department of Electrical & Computer Engineering, Rice University, Houston, TX, 1992.
17. Cover T.M., and Thomas J.A. Elements of Information Theory (1991), Wiley, New York
18. Chernoff H. Large-sample theory: parametric case. Ann. Math. Statist. 27 (1956) 1-22
19. Johnson D., and Orsak G. Relation of signal set choice to the performance of optimal non-Gaussian detectors. IEEE Trans. Comm. 41 (1993) 1319-1328
20. Scharf L. Statistical Signal Processing: Detection, Estimation, and Time Series Analysis (1990), Addison-Wesley, Reading, MA
21. Blahut R. Principles and Practice of Information Theory (1987), Addison-Wesley, Reading, MA
22. Kullback S. Information Theory and Statistics (1959), Wiley, New York
23. Johnson D., and Dudgeon D. Array Signal Processing: Concepts and Techniques (1993), Prentice-Hall, Englewood Cliffs, NJ
24. Johnson D., Gruner C., Baggerly K., and Seshagiri C. Information-theoretic analysis of neural coding. J. Comput. Neuroscience 10 (2001) 47-69
25. Miller C., Johnson D., Schroeter J., Myint L., and Glantz R. Visual responses of crayfish ocular motoneurons: an information theoretical analysis. J. Comput. Neuroscience 15 (2003) 247-269
26. Rozell C., Johnson D., and Glantz R. Measuring information transfer in crayfish sustaining fiber spike generators: methods and analysis. Biological Cybernet. 90 (2004) 89-97
27. Jeffreys H. An invariant form for the prior probability in estimation problems. Proc. Roy. Soc. A 186 (1946) 453-461
28. Arandjelovic O., and Cipolla R. Face recognition from face motion manifolds using robust kernel resistor-average distance. Conference on Computer Vision and Pattern Recognition Workshops (2004) 88
29. Spiess T., Wrede B., Kummert F., and Fink G.A. Data-driven pronunciation modeling for ASR using acoustic subword units. Proceedings of the European Conference on Speech Communication and Technology. Geneva (2003) 2549-2552
30. Varshney P. Distributed Detection and Data Fusion (1997), Springer, New York
31. Johnson D. Neural population structures and consequences for neural coding. J. Comput. Neuroscience 16 (2004) 69-80
32. Popoli R., and Mendel J. Relative sufficiency. IEEE Trans. Automat. Control 38 (1993) 826-828
33. Lexa M., Rozell C., Sinanović S., and Johnson D. To cooperate or not to cooperate: detection strategies in sensor networks. ICASSP '04. Montreal, Canada (2004)
34. M. Lexa, D. Johnson, Broadcast detection structures with applications to sensor networks, IEEE Trans. Signal Process., submitted.
35. van Trees H. Detection, Estimation, and Modulation Theory, Part I (1968), Wiley, New York
36. Leang C., and Johnson D. On the asymptotics of M-hypothesis Bayesian detection. IEEE Trans. Info. Th. 43 (1997) 280-282
37. Cramer H. Mathematical Methods of Statistics (1946), Princeton University Press, Princeton, NJ