Bootstrap methods for the empirical study of decision-making and information flows in social systems

Entropy

Volumn 15, Issue 6, 2013, Pages 2246-2276

  DeDeo, Simon ^a   Hawkins, Robert X D ^a,b   Klingenstein, Sara ^a   Hitchcock, Tim ^c  

^a SANTA FE INSTITUTE   (United States)

^b INDIANA UNIVERSITY   (United States)

^c UNIVERSITY OF HERTFORDSHIRE   (United Kingdom)

Author keywords

Bayesian estimation; Biological systems; Bootstrap; Cognition; Information theory; Social systems; Statistical estimation

Indexed keywords

EID: 84880008032     PISSN: None     EISSN: 10994300     Source Type: Journal    
DOI: 10.3390/e15062246     Document Type: Article

References (41)

1. Shannon, C. A mathematical theory of communication. Bell Syst. Tech. J. 1948, 27, 379-423, 623-656.
2. Paninski, L. Estimation of entropy and mutual information. Neural Comput. 2003, 15, 1191- 1253.
3. Collaboration "Institution Formation, Semantic Phenomena, and Three Hundred Years of the British Criminal Court System", with participants Simon DeDeo, Sara Klingenstein, and Tim Hitchcock.
4. This is not exactly true, since we use part-of-speech information for our semantic classifier, and the means (a hidden Markov model) for identifying a word's lexical category are sensitive to order and context: compare "the dogs bit the sailor" ['dogs' as plural noun] and "the sailor dogs the waitress" ['dogs' as verb]. It is an interesting and open question the extent to which syntactical features convey semantic information.
5. It is possible to correct the magic word problem post hoc, merging categories until no magic words remain. If this rule is specified with sufficient generality, it allows for bootstrap estimation. We do not consider this approach here.
6. Lin, J. Divergence measures based on the Shannon entropy. IEEE Trans. Inform. Theory 1991, 37, 145-151.
7. Nielsen, F. A family of statistical symmetric divergences based on Jensen's inequality. arXiv 2010, arXiv:1009.4004.
8. Hellman, M.; Raviv, J. Probability of error, equivocation, and the Chernoff bound. IEEE Tran. Inform. Theory 1970, 16, 368-372.
9. Jaynes, E.; Bretthorst, G. Probability Theory: The Logic of Science; Cambridge University Press: Cambridge, UK, 2003.
10. Cover, T.; Thomas, J. Elements of Information Theory; Wiley: Hoboken, NJ, USA, 2006.
11. Chen, C.H. On information and distance measures, error bounds, and feature selection. Inform. Sci. 1976, 10, 159-173.
12. Ito, T. Approximate error bounds in pattern recognition. Mach. Intell. 1972, 7, 369-376.
13. Hashlamoun, W.A.; Varshney, P.K.; Samarasooriya, V.N.S. A tight upper bound on the Bayesian probability of error. IEEE Trans. Inform. Theory 1994, 16, 220-224.
14. We can estimate ρ itself by means of the statistical bootstrap. Resampling suggests that ρ itself may suffer from bias which can be corrected for.
15. Consider, for example, the case where the supports of p1 and p2 only partially overlap: p1(x) is zero for some x, but p2(x) is non-zero, and vice versa for some y. The draw fx; yg is not possible for either p1 or p2, but is possible for the average.
16. O'Loughlin, J.; Witmer, F.; Linke, A.; Thorwardson, N. Peering into the Fog of War: The Geography of the WikiLeaks Afghanistan War Logs 2004-2009. Eurasian Geogr. Econ. 2010, 51, 472-95.
17. Zammit-Mangion, A.; Dewar, M.; Kadirkamanathan, V.; Sanguinetti, G. Point process modelling of the Afghan War Diary. Proc. Natl. Acad. Sci. USA 2012, 109, 12414-12419.
18. There are more than 78,000 SIGACTs in the original data set. We removed approximately 20,000 because they expressed a level of ambiguity about what happened ("unknown-initiated action" or "suspicious incident"), because their GPS coordinates did not match the description, because they were not time-sensitive ("weapons cache found") or because they were irrelevant to our study ("IED hoax" or "show of force"). Information on initiative is explicitly included in the SIGACT data and is extracted by the methods of Ref. [16].
19. Sanín, F.G.; Giustozzi, A. Networks and Armies: Structuring Rebellion in Colombia and Afghanistan. Stud. Confl. Terror. 2010, 33, 836-853.
20. Gutierrez Sanin, F. Telling the Difference: Guerrillas and Paramilitaries in the Colombian War. Polit. Soc. 2008, 36, 3-34.
21. Green, A.H. Repertoires of Violence Against Non-combatants: The Role of Armed Group Institutions and Ideologies. Ph.D. Thesis, Yale University, New Haven, CT, USA, 2011.
22. Collaboration "The Emergence of Insurgency in Afghanistan: An Information Theoretic Analysis", with participants Simon DeDeo and Robert Hawkins.
23. Hutter, M. Distribution of Mutual Information. arXiv 2001, arXiv:cs/0112019.
24. Zaffalon, M.; Hutter, M. Robust Feature Selection by Mutual Information Distributions. In Proceedings of the 18th Conference in Uncertainty in Artificial Intelligence, Edmonton, Canada, 1-4 August 2002; pp. 577-584.
25. Williams, P.L.; Beer, R.D. Generalized Measures of Information Transfer. arXiv 2011, arXiv:1102.1507.
26. Schreiber, T. Measuring Information Transfer. Phys. Rev. Lett. 2000, 85, 461-464.
27. THOTH. Available online: http://thoth-python.org (accessed on 22 May 2013).
28. Nemenman, I.; Shafee, F.; Bialek, W. Entropy and Inference, Revisited. In Advances in Neural Information Processing Systems 14; Dietterich, T.G., Becker, S., Ghahramani, Z., Eds.; MIT Press: Cambridge, MA, USA, 2002.
29. Nemenman, I.; Bialek,W.; de Ruyter van Steveninck, R. Entropy and information in neural spike trains: Progress on the sampling problem. Phys. Rev. E 2004, 69, 56111:1-56111:6.
30. Wolpert, D.H.; Wolf, D.R. Estimating functions of probability distributions from a finite set of samples. Phys. Rev. E 1995, 52, 6841-6854.
31. Nijenhuis, A.; Wilf, H.S. Combinatorial Algorithms for Computers and Calculators, 2nd ed.; Academic Press: New York, NY, USA, 1978.
32. Care needs to be taken, however, since a general stochastic function of the underlying process will not, in general, preserve independence in the empirical distribution. It is always possible, for example, to choose an instantiation of a stochastic re-mapping post hoc that magnifies accidental correlations found in the original empirical distribution. We thank John Geanakoplos for pointing this out to us.
33. DeDeo, S. Effective theories for circuits and automata. Chaos 2011, 21, arXiv:1106.5778.
34. Olshausen, B.A.; Field, D.J. Sparse coding with an overcomplete basis set: A strategy employed by V1? Vision Res. 1997, 37, 3311-3325.
35. Olshausen, B.A.; Field, D.J. Sparse coding of sensory inputs. Curr. Opin. Neurobiol. 2004, 14, 481-487.
36. Daniels, B.C.; Krakauer, D.C.; Flack, J.C. Sparse code of conflict in a primate society. Proc. Natl. Acad. Sci. USA 2012, 109, 14259-14264.
37. Plato. Phaedrus; Harvard University Press: Cambridge, MA, USA, 1925; Plato in Twelve Volumes, Volume 9. Translated by Harold N. Fowler. Available online: http://www.perseus.tufts.edu/hopper/text?doc=Plat.+Phaedrus+265e (accessed on 22 May 2013).
38. Afghan War Diary. Available online: http://wikileaks.org/afg/event/2005/04/AFG20050403n68. html (accessed on 4 June 2013).
39. Miller, G. Note on the Bias of Information Estimates. In Information Theory in Psychology II-B; Quastler, H., Ed.; Free Press: Glencoe, IL, USA, 1955.
40. MacKinnon, J.G.; Smith, A.A., Jr. Approximate bias correction in econometrics. J. Econometrics 1998, 85, 205-230.
41. MacKinnon, J.G. Bootstrap Methods in Econometrics. Econ. Rec. 2006, 82, S2-S18