Conservation of information in search: Measuring the cost of success

IEEE Transactions on Systems, Man, and Cybernetics Part A:Systems and Humans

Volumn 39, Issue 5, 2009, Pages 1051-1061

  Dembski, William A ^a   Marks II, Robert J ^b  

^a Southwestern Baptist Theological Seminary ^*  (United States)

^b Baylor University   (United States)

Author keywords

Active information; Asymptotic equipartition property; Brillouin active information; Conservation of information (COI); Endogenous information; Evolutionary search; Genetic algorithms; Kullback Leibler distance; No free lunch theorem (NFLT); Partitioned search

Indexed keywords

ACTIVE INFORMATION; ASYMPTOTIC EQUIPARTITION PROPERTY; BRILLOUIN ACTIVE INFORMATION; ENDOGENOUS INFORMATION; EVOLUTIONARY SEARCH; KULLBACK-LEIBLER DISTANCE; NO FREE LUNCH THEOREM (NFLT); PARTITIONED SEARCH;

ASYMPTOTIC ANALYSIS; LEARNING ALGORITHMS; SPACE PLATFORMS; TARGETS;

GENETIC ALGORITHMS;

EID: 69649099279     PISSN: 10834427     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSMCA.2009.2025027     Document Type: Article

References (51)

1. H. M. Abbas and M. M. Bayoumi, "Volterra-system identification using adaptive real-coded genetic algorithm," IEEE Trans. Syst., Man, Cybern. A, Syst., Humans, vol. 36, no. 4, pp. 671-684, Jul. 2006.
2. S. Agrawal, Y. Dashora, M. K. Tiwari, and Y.-J. Son, "Interactive particle swarm: A pareto-adaptive metaheuristic to multiobjective optimization," IEEE Trans. Syst., Man, Cybern. A, Syst., Humans, vol. 38, no. 2, pp. 258-277, Mar. 2008.
3. M. Aigner, Discrete Mathematics. Providence, RI: Amer. Math. Soc., 2007.
4. J. Bernoulli, "Ars conjectandi (the art of conjecturing)," Tractatus De Seriebus Infinitis1713.
5. L. Brillouin, Science and Information Theory. New York: Academic, 1956.
6. J. P. Burg, "Maximum entropy spectral analysis," Ph.D. dissertation, Stanford Univ., Stanford, CA, May, 1975.
7. S. Christensen and F. Oppacher, "What can we learn from no free lunch? A first attempt to characterize the concept of a searchable," in Proc. Genetic Evol. Comput., 2001, pp. 1219-1226.
8. T.-Y. Chou, T.-K. Liu, C.-N. Lee, and C.-R. Jeng, "Method of inequality-based multiobjective genetic algorithm for domestic daily aircraft routing," IEEE Trans. Syst., Man, Cybern. A, Syst., Humans vol. 38, no. 2, pp. 299-308, Mar. 2008.
9. T. M. Cover and J. A. Thomas, Elements of Information Theory, 2nd ed. New York: Wiley-Interscience, 2006.
10. J. C. Culberson, "On the futility of blind search: An algorithmic view of 'no free lunch'," Evol. Comput., vol. 6, no. 2, pp. 109-127, 1998.
11. J. D. Cutnell and J. W. Kenneth, Physics, 3rd ed. New York: Wiley, 1995.
12. R. Dawkins, The Blind Watchmaker: Why the Evidence of Evolution Reveals a Universe Without Design. New York: Norton, 1996.
13. S. Droste, T. Jansen, and I.Wegener, "Perhaps not a free lunch but at least a free appetizer," in Proc. 1st GECCO, pp. 833-839.
14. R. C. Eberhart, Y. Shi, and J. Kennedy, Swarm Intelligence. San Mateo, CA: Morgan Kaufmann, 2001.
15. T. M. English, "Some information theoretic results on evolutionary optimization," in Proc. CEC, Jul. 6-9, 1999, vol. 1, p. 795.
16. M. S. Fadali, Y. Zhang, and S. J. Louis, "Robust stability analysis of discrete-time systems using genetic algorithms," IEEE Trans. Syst., Man, Cybern. A, Syst., Humans, vol. 29, no. 5, pp. 503-508, Sep. 1999.
17. L. K. Grover, "A fast quantum mechanical algorithm for data search," in Proc. ACM Symp. Theory Comput., 1996, pp. 212-219.
18. P. Guturu and R. Dantu, "An impatient evolutionary algorithm with probabilistic tabu search for unified solution of some NP-hard problems in graph and set theory via clique finding," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 38, no. 3, pp. 645-666, Jun. 2008.
19. T. D. Gwiazda, Genetic Algorithms Reference: Tomasz Gwiazda, 2006.
20. J. S. Gyorfi and C.-H.Wu, "An efficient algorithm for placement sequence and feeder assignment problems with multiple placement-nozzles and independent link evaluation," IEEE Trans. Syst., Man, Cybern. A, Syst., Humans, vol. 38, no. 2, pp. 437-442, Mar. 2008.
21. M. J. Healy, The Boeing Company, Personal Communication.
22. S.-Y. Ho, H.-S. Lin, W.-H. Liauh, and S.-J. Ho, "OPSO: Orthogonal particle swarm optimization and its application to task assignment problems," IEEE Trans. Syst., Man, Cybern. A, Syst., Humans, vol. 38, no. 2, pp. 288-298, Mar. 2008.
23. Y.-C. Ho and D. L. Pepyne, "Simple explanation of the no free lunch theorem," in Proc. 40th IEEE Conf. Decision Control, Orlando, FL, 2001, pp. 4409-4414.
24. Y.-C. Ho, Q.-C. Zhao, and D. L. Pepyne, "The no free lunch theorems: Complexity and security," IEEE Trans. Autom. Control, vol. 48, no. 5, pp. 783-793, May 2003.
25. C. A. Jensen, R. D. Reed, R. J. Marks, II, M. A. El-Sharkawi, J.-B. Jung, R. T. Miyamoto, G. M. Anderson, and C. J. Eggen, "Inversion of feedforward neural networks: Algorithms and applications," Proc. IEEE, vol. 87, no. 9, pp. 1536-1549, Sep. 1999.
26. M. Koppen, D. H. Wolpert, and W. G. Macready, "Remarks on a recent paper on the 'no free lunch' theorems," IEEE Trans. Evol. Comput. vol. 5, no. 3, pp. 295-296, Jun. 2001.
27. G. Korodi, I. Tabus, J. Rissanen, and J. Astola, "DNA sequence compression," IEEE Signal Process. Mag., vol. 47, no. 1, pp. 47-53, Jan. 2007.
28. C.-Y. Lee, "Entropy - Boltzmann selection in the genetic algorithms," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 33, no. 1, pp. 138-149, Feb. 2003.
29. R. E. Lenski, C. Ofria, R. T. Pennock, and C. Adami, "The evolutionary origin of complex features," Nature, vol. 423, no. 6936, pp. 139-144, May 8, 2003.
30. Y. Li and C. O. Wilke, "Digital evolution in time-dependent fitness landscapes," Artif. Life, vol. 10, no. 2, pp. 123-134, Apr. 2004.
31. B. Liu, L. Wang, and Y.-H. Jin, "An effective PSO-based memetic algorithm for flow shop scheduling," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 37, no. 1, pp. 18-27, Feb. 2007.
32. D. J. C. MacKay, Information Theory, Inference and Learning Algorithms. Cambridge, U.K.: Cambridge Univ. Press, 2002.
33. R. J. Marks, II, Handbook of Fourier Analysis and Its Application. London, U.K.: Oxford Univ. Press, 2009.
34. A. Papoulis, Probability, Random Variables, and Stochastic Processes 3rd ed. New York: McGraw-Hill, 1991, pp. 537-542.
35. U. S. Putro, K. Kijima, and S. Takahashi, "Adaptive learning of hypergame situations using a genetic algorithm," IEEE Trans. Syst., Man, Cybern. A, Syst., Humans, vol. 30, no. 5, pp. 562-572, Sep. 2000.
36. R. D. Reed and R. J. Marks, II, Neural Smithing: Supervised Learning in Feedforward Artificial Neural Networks. Cambridge, MA: MIT Press, 1999.
37. K. Takita and Y. Kakazu, "Automatic agent design based on gate growth-application to wall following problem," in Proc. 37th Int. Session Papers SICE Annu. Conf., Jul. 29-31, 1998, pp. 863-868.
38. F. Rojas, C. G. Puntonet, M. Rodriguez-Alvarez, I. Rojas, and R. Martin-Clemente, "Blind source separation in post-n[Online]ar mixtures using competitive learning, Simulated annealing, and a genetic algorithm," IEEE Trans. Syst., Man, Cybern. C, Appl. Rev., vol. 34, no. 4, pp. 407-416, Nov. 2004.
39. C. Schaffer, "A conservation law for generalization performance," in Proc. 11th Int. Conf. Mach. Learn., H. Hirsh and W. W. Cohen, Eds., 1994, pp.259-265.
40. T. D. Schneider, "Evolution of biological information," Nucleic Acids Res., vol. 28, no. 14, pp. 2794-2799, Jul. 2000.
41. C. E. Shannon, "A mathematical theory of communication," Bell Syst. Tech. J., vol. 27, pp. 379-423, Jul./Oct. 1948, and 623-656.
42. R. Srinivasan, Importance Sampling. Berlin, Germany: Springer-Verlag, 2002.
43. B. Weinberg and E. G. Talbi, "NFL theorem is unusable on structured classes of problems," in Proc. CEC, Jun. 19-23, 2004, vol. 1, pp. 220-226.
44. A. M. Turing, "On computable numbers with an application to the entscheidungs problem," Proc. Lond. Math. Soc. Ser. 2, vol. 42, pp. 230-265, 1936, vol. 43. pp.544-546.
45. G. S. Tewolde and W. Sheng, "Robot path integration in manufacturing processes: Genetic algorithm versus ant colony optimization," IEEE Trans. Syst., Man, Cybern. A, Syst., Humans, vol. 38, no. 2, pp. 278-287, Mar. 2008.
46. D. Wolpert and W. G. Macready, "No free lunch theorems for optimization," IEEE Trans. Evol. Comput., vol. 1, no. 1, pp. 67-82, Apr. 1997.
47. D. H. Wolpert and W. G. Macready, "Coevolutionary free lunches," IEEE Trans. Evol. Comput., vol. 9, no. 6, pp. 721-735, Dec. 2005.
48. E. V. Wright, Gadsby. New York: Lightyear Press, 1997.
49. Y.-C. Xu and R.-B. Xiao, "Solving the identifying code problem by a genetic algorithm," IEEE Trans. Syst., Man, Cybern. A, Syst., Humans vol. 37, no. 1, pp. 41-46, Jan. 2007.
50. H. P. Yockey, Information Theory, Evolution, and the Origin of Life Cambridge, U.K.: Cambridge Univ. Press, 2005.
51. F. Yu, F. Tu, and K. R. Pattipati, "A novel congruent organizational design methodology using group technology and a nested genetic algorithm," IEEE Trans. Syst., Man, Cybern. A, Syst., Humans, vol. 36, no. 1, pp. 5-18, Jan. 2006.