Fast learning neural networks using Cartesian genetic programming

Neurocomputing

Volumn 121, Issue , 2013, Pages 274-289

  Mahsal Khan, Maryam ^a   Masood Ahmad, Arbab ^a   Muhammad Khan, Gul ^a   Miller, Julian F ^b  

^a UNIVERSITY OF ENGINEERING AND TECHNOLOGY   (Pakistan)

^b UNIVERSITY OF YORK   (United Kingdom)

Author keywords

Artificial neural network; Breast cancer; Neuroevolution; Pole balancing; Recurrent networks

Indexed keywords

BENCHMARK CONTROL PROBLEMS; BREAST CANCER; CARTESIAN GENETIC PROGRAMMING; FEEDFORWARD AND RECURRENT NETWORKS; NEURO EVOLUTIONS; PATTERN RECOGNITION PROBLEMS; POLE BALANCING; RECURRENT NETWORKS;

DIAGNOSIS; DISEASES; EVOLUTIONARY ALGORITHMS; GENETIC PROGRAMMING; LEARNING ALGORITHMS; PATTERN RECOGNITION; POLES;

NEURAL NETWORKS;

ALGORITHM; ARTICLE; ARTIFICIAL NEURAL NETWORK; BREAST CANCER; CALCULATION; CANCER STAGING; COMPARATIVE STUDY; COMPUTER PROGRAM; CONTROLLED STUDY; DIAGNOSTIC ACCURACY; FINE NEEDLE ASPIRATION BIOPSY; GENOTYPE; HUMAN; HUMAN CELL; INFORMATION PROCESSING; INFORMATION RETRIEVAL; MUTATION RATE; PRIORITY JOURNAL; SIMULATION;

EID: 84884125432     PISSN: 09252312     EISSN: 18728286     Source Type: Journal    
DOI: 10.1016/j.neucom.2013.04.005     Document Type: Article

References (82)

1. Wong P.M., Gedeon T., Taggart I.J. An improved technique in porosity prediction. a neural network approach. IEEE Trans. Geosci. Remote Sensing 1995, 33(4):971-980.
2. Wu Y., Huang T.S. Nonstationary color tracking for vision-based human-computer interaction. IEEE Trans. Neural Netw. 2002, 13(4):948-960.
3. Tetsuji Tani S.M., Umano M. Neuro-fuzzy hybrid control system of tank level in petroleum plant. IEEE Trans. Fuzzy Syst. 1996, 4(3):360-368.
4. Farrell J.A., Polycarpou M.M. Adaptive Approximation Based Control. Unifying Neural, Fuzzy and Traditional Adaptive Approximation Approaches 2006, Wiley-Blackwell.
5. K. Stanley, R. Sherony, N. Kohl, R. Miikkulainen, Neuroevolution of an automobile crash warning system, in: Proceedings of GECCO-2005, 2005.
6. K. Tumer, Coordinating multi-rover systems: evaluation functions for dynamic and noisy environments, in: GECCO-2005, ACM Press, 2005, pp. 591-598.
7. J. Koutnik, F. Gomez, J. Schmidhuber, Evolving neural networks in compressed weight space, in: GECCO '10, ACM, 2010, pp. 619-626.
8. M.M. Khan, G.M. Khan, J.F. Miller, Evolution of optimal ANNs for non-linear control problems using cartesian genetic programming, in: Proceedings of IEEE. ICAI, 2010, pp. 339-346.
9. Gomez F., Schmidhuber J., Miikkulainen R. Accelerated neural evolution through cooperatively coevolved synapses. J. Mach. Learn. Res. 2008, 9:937-965.
10. D. Moriarty, Symbiotic Evolution of Neural Networks in Sequential Decision Tasks, Ph.D. Dissertation, University of Texas at Austin, 1997.
11. García-Pedrajas N., Hervás-Martínez C., Muñoz-Pérez J. Covnet. a cooperative coevolutionary model for evolving artificial neural networks. IEEE Trans. Neural Netw. 2003, 14(3).
12. Koza J. Genetic Programming. On the Programming of Computers by Means of Natural Selection 1992, MIT Press.
13. Banzhaf W., Nordin P., Keller R.E., Francone F.D. Genetic Programming-An Introduction; On the Automatic Evolution of Computer Programs and its Applications 1998, Morgan Kaufmann.
14. Poli R., Langdon W.B., McPhee N.F. A Field Guide to Genetic Programming 2008, Lulu.
15. J.F. Miller, P. Thomson, T.C. Fogarty, Designing electronic circuits using evolutionary algorithms, arithmetic circuits: a case study, in: Genetic Algorithms and Evolution Strategies in Engineering and Computer Sciences, Wiley, 1998, pp. 105-131.
16. J.F. Miller, An empirical study of the efficiency of learning Boolean functions using a cartesian genetic programming approach, in: Proceedings of GECCO'99, vol. 2, Morgan Kaufmann, 1999, pp. 1135-1142.
17. J.F. Miller, P. Thomson, Cartesian genetic programming, in: Proceedings of the 3rd European Conference on Genetic Programming, vol. 1802, 2000, pp. 121-132.
18. S. Harding, J.F. Miller, Evolution of robot controller using cartesian genetic programming, in: Genetic Programming, 8th European Conference, EuroGP2005, vol. 3447, 2005, pp. 62-73.
19. J.A. Walker, J.F. Miller, Solving real-valued optimisation problems using cartesian genetic programming, in: Proceedings of GECCO'07, 2007, pp. 1724-1730.
20. Walker J.A., Miller J. Automatic acquisition, evolution and re-use of modules in cartesian genetic programming. IEEE Trans. Evol. Comput. 2008, 12:397-417.
21. Walker J.A., Völk K., Smith S.L., Miller J.F. Parallel evolution using multi-chromosome cartesian genetic programming. Genet. Program. Evolvable Mach. 2009, 10:417-445.
22. Harding S.L., Miller J.F., Banzhaf W. Developments in cartesian genetic programming. self-modifying cgp. Genet. Program. Evolvable Mach. 2010, 11:397-439.
23. J.C. Bongard, R. Pfeifer, Repeated structure and dissociation of genotypic and phenotypic complexity in artificial ontogeny, in: Spector et al. (Ed.), 2001, pp. 829-836.
24. C. Lee, J.-H. Kim, Evolutionary ordered neural network with a linked-list encoding scheme, in: Proceedings of IEEE. CEC, 1996, pp. 665-669.
25. Yao X., Liu Y. Towards designing artificial neural networks by evolution. Appl. Math. Comput. 1996, 91(1):83-90.
26. D. Dasgupta, D. McGregor, Designing application-specific neural networks using the structured genetic algorithm, in: Proceedings of ICCGANN, 1992, pp. 87-96.
27. Opitz D.W., Shavlik J.W. Connectionist theory refinement. genetically searching the space of network topologies. Journal of Artificial Intelligence Research 1997, 6:177-209.
28. Zhang B., Muhlenbein H. Evolving optimal neural networks using genetic algorithms with occam's razor. Complex Systems 1993, 7:199-220.
29. Maniezzo V. Genetic evolution of the topology and weight distribution of neural networks. IEEE Trans. Neural Netw. 1994, 5(1):39-53.
30. H. Braun, J. Weisbrod, Evolving feedforward neural networks, in: Proceedings of ICANNGA93, Springer-Verlag, 1993.
31. Gruau F. Automatic definition of modular neural networks. Adaptive Behav. 1994, 3:151-183.
32. Hornby G.S., Pollack J.B. Creating high-level components with a generative representation for body-brain evolution. Artif. Life 2002, 8.
33. Mandischer M. Artificial Neural Nets and Genetic Algorithms 1993, Springer.
34. F. Gruau, D. Whitley, L. Pyeatt, A comparison between cellular encoding and direct encoding for genetic neural networks, in: Proceedings of First Annual Conference on GP, MIT Press, 1996, pp. 81-89.
35. X. Yao, Evolving artificial neural networks, in: Proceedings of the IEEE, vol. 87, no. 9, 1999, pp. 1423-1447.
36. F.J. Gomez, R. Miikkulainen, Solving non-Markovian control tasks with neuroevolution, in: Proceedings of International Joint Conference on Artificial Intelligence, Morgan Kaufmann Publishers Inc., 1999, pp. 1356-1361.
37. A.P. Wieland, Evolving neural network controllers for unstable systems, in: Proceedings of International Joint Conference on Neural Networks, 1991, pp. 667-673.
38. K.O. Stanley, R. Miikkulainen, Efficient reinforcement learning through evolving neural network topologies, in: Proceedings of GECCO'02, Morgan Kaufmann, 2002, pp. 567-577.
39. Y. Tsoy, V. Spitsyn, Using genetic algorithm with adaptive mutation mechanism for neural networks design and training, in: Proceedings of 9th KORUS'05, IEEE, 2005,pp. 709-714.
40. K. Ohkura, T. Yasuda, Y. Kawamatsu, Y. Matsumura, K. Ueda, Mbeann: mutation-based evolving artificial neural networks, in: Proceedings of ECAL'07, Springer-Verlag, 2007, pp. 936-945.
41. Angeline P.J., Saunders G.M., Pollack J.B. An evolutionary algorithm that constructs recurrent neural networks. IEEE Transactions on Neural Networks 1993, 5:54-65.
42. C.G. Miguel, C.F.d. Silva, M.L. Netto, Structural and parametric evolution of continuous-time recurrent neural networks, in: SBRN '08: Proceedings of 10th Brazilian Symposium on Neural Networks, IEEE Computer Society, 2008, pp. 177-182.
43. D.B. D'Ambrosio, K.O. Stanley, A novel generative encoding for exploiting neural network sensor and output geometry, in: Proceedings of the 9th Annual Conference on Genetic and Evolutionary Computation Series GECCO '07, ACM, New York, NY, USA, 2007, pp. 974-981. [Online]. Available: http://doi.acm.org/10.1145/1276958.1277155.
44. Stanley K.O., D'Ambrosio D.B., Gauci J. A hypercube-based encoding for evolving large-scale neural networks. Artif. Life 2009, 15(April (2)):185-212. [Online]. Available: http://dx.doi.org/10.1162/artl.2009.15.2.15202
45. Z. Buk, J. Koutník, M. Šnorek, Neat in hyperneat substituted with genetic programming, in: Proceedings of the 9th International Conference on Adaptive and Natural Computing Algorithms, ICANNGA'09, Springer-Verlag, Berlin, Heidelberg, 2009, pp. 243-252. [Online]. Available http://dl.acm.org/citation.cfm?id=1813739.1813766.
46. S. Risi, J. Lehman, K.O. Stanley, Evolving the placement and density of neurons in the hyperneat substrate, in: Proceedings of the 12th Annual Conference on Genetic and Evolutionary Computation, GECCO'10, ACM, New York, NY, USA, 2010, pp. 563-570. [Online]. Availablehttp://doi.acm.org/10.1145/1830483.1830589.
47. R. Poli, Evolution of graph-like programs with parallel distributed genetic programming, in: Proceedings of ICGA'97, Morgan-Kaufmann, 1997, pp. 346-353.
48. Pujol J., Poli R. Evolving the topology and the weights of neural networks using a dual representation. Appl. Intell. J. 1998, 8:73-84.
49. R. Poli, M. Graf, Free lunches for neural network search, in: Proceedings of GECCO'09, ACM, 2009, pp. 1291-1298.
50. I. Rechenberg, Evolutionsstrategie-Optimierung Technischer Systeme nach Prinzipien der Biologischen Evolution, Ph.D. Dissertation, Technical University of Berlin, Germany, 1971.
51. V.K. Vassilev, J.F. Miller, The advantages of landscape neutrality in digital circuit evolution, in: Proceedings of ICES, Lecture Notes in Computer Sciences, vol. 1801, Springer Verlag, 2000, pp. 252-263.
52. T. Yu, J.F. Miller, Neutrality and the evolvability of boolean function landscape, in: Proceedings of EuroGP, Lecture Notes in Computer Sciences, vol. 2038, Springer-Verlag, 2001, pp. 204-217.
53. Miller J.F., Smith S.L. Redundancy and computational efficiency in cartesian genetic programming. IEEE Transactions on Evolutionary Computation 2006, 10(2):167-174.
54. M.I. Jordan, Attractor dynamics and parallelism in a connectionist sequential machine, in: J. Diederich (Ed.), Artificial Neural Networks, IEEE Press, Piscataway, NJ, USA, 1990, pp. 112-127. [Online]. Available: http://dl.acm.org/citation.cfm?id=104134.104148.
55. A.P. Wieland, Evolving controls for unstable systems, in: Proceedings of Connectionist Models Summer School, 1991, pp. 91-102.
56. J. Gomez, F. Schmidhuber, R. Miikkulainen, Efficient non-linear control through neuroevolution, in: ECML 2006: Proceedings of the 17th European Conference on Machine Learning, Springer, 2006.
57. Wolberg W.H., Mangasarian O.L. Multisurface method of pattern separation for medical diagnosis applied to breast cytology. Proceedings of the National Academy of Sciences of the United State of America 1990, 87:9193-9196.
58. M. Iranpour, S. Almassi, M. Analoui, Breast cancer detection from FNA using SVM and RBF classifier, in: 1st Joint Congress on Fuzzy and Intelligent Systems, 2007.
59. Túlio C.S.S.A., Rangayyan R.M. Classification of breast masses in mammograms using neural networks with shape, edge sharpness and texture features. J. Electron. Imaging 2006, 15(1):013019.
60. R. Santo, R. Lopes, R. Rangayyan, Classification of mammographic masses using radial basis functions and simulated annealing with shape, acutance, and texture features, in: Proceedings of the IASTED Conference on Biomedical Engineering, 2005, pp. 458-175.
61. Cheng H.D., Cui M. Mass lesion detection with a fuzzy neural network. Pattern Recognition 2004, 37(6):1189-1200.
62. Palmer G., Zhu C., Breslin T., Xu F., Gilchrist K., Ramanujam N. Comparison of multi excitation fluorescence and diffuse reflectance spectroscopy for the diagnosis of breast cancer. IEEE Trans. Biomed. Eng. 2003, 50(11):1233-1242.
63. Markey M.K., Lo J.Y., Tourassi G.D., Floyd C.E. Self-organizing map for cluster analysis of a breast cancer database. Artif. Intell. Med. 2003, 27:113-127.
64. Ho S., Chen H., Ho S., Chen T. Design of accurate classifiers with a compact fuzzy-rule base using an evolutionary scatter partition of feature space. IEEE Trans. Syst. Man Cybern. 2004, 34(2):1031-1044.
65. Wei L., Yang Y., Nishikawa R.M., Jiang Y. A study on several machine-learning methods for classification of malignant and benign clustered microcalcifications. IEEE Trans. Med. Imaging 2005, 24:371-380.
66. M.M. Khan, G.M. Khan, J.F. Miller, Evolution of neural networks using cartesian genetic programming, in: IEEE Congress on Evolutionary Computation, 2010, pp. 1-8.
67. Guo H., Nandi A.K. Breast cancer diagnosis using genetic programming generated feature. Pattern Recognition 2006, 39(5):980-987.
68. H. Jianchao, N. Cercone, uleViz: a model for visualizing knowledge discovery process, in: KDD, 2000, pp. 244-253.
69. J.C. Werner, T.C. Fogarty, Genetic programming applied to severe diseases diagnosis, in: Proceedings of Intelligent Data Analysis in Medicine and Pharmacology (IDAMAP-2001), 2001.
70. Nandi R., Nandi A., Rangayyan R., Scutt D. Classification of breast masses in mammograms using genetic programming and feature selection. Med. Biol. Eng. Comput. 2006, 44(8):683-694.
71. W.H. Land Jr., L. Albertelli, Y. Titkov, P. Kaltsatis, G. Seburyano, Evolution of neural networks for the detection of breast cancer, in: Proceedings of IEEE International Joint Symposia on Intelligence and Systems, INTSYS '98, 1998.
72. T. Mu, A.K. Nandi, Breast cancer detection from FNA using SVM with different parameter tuning systems and SOM-RBF classifier, J. Franklin Inst. (2013) 285-311, in press. http://dx.doi.org/10.1016/j.jfranklin.2006.09.005.
73. West D., West V. Model selection for a medical diagnostic decision support system. a breast cancer detection case. Artif. Intell. Med. 2000, 20(3):183-204.
74. Abbass H.A. An evolutionary artificial neural networks approach for breast cancer diagnosis. Artif. Intell. Med. 2002, 25:265-281.
75. Ho S.-Y., Chen H.-M., Ho S.-J., Chen T.-K. Design of accurate classifiers with a compact fuzzy-rule base using an evolutionary scatter partition of feature space. IEEE Trans. Syst. Man Cybern. B 2004, 34(2):1031-1044.
76. Lam W., Keung C.-K., Ling C.X. Learning good prototypes for classification using filtering and abstraction of instances. Pattern Recognition 2002, 35(7):1491-1506.
77. R. Janghel, A. Shukla, R. Tiwari, R. Kala, Intelligent decision support system for breast cancer, in: Advances in Swarm Intelligence, Springer, Berlin, Heidelberg, 2010, vol. 6146, pp. 351-358 (Chapter 46).
78. B.A. Garro, H. Sossa, R.A. Vzquez, Artificial neural network synthesis by means of artificial bee colony (abc) algorithm, in: IEEE Congress on Evolutionary Computation, IEEE, 2011, pp. 331-338.
79. B.A. Garro, H. Sossa, R.A. Vazquez, Design of artificial neural networks using a modified particle swarm optimization algorithm, in: Proceedings of the 2009 International Joint Conference on Neural Networks, IJCNN'09, IEEE Press, Piscataway, NJ, USA, 2009, pp. 2363-2370.
80. William H.W., Street W., Mangasarian O. Image analysis and machine learning applied to breast cancer diagnosis and prognosis. Anal. Quant. Cytol. Histol. 1995, 17(2):77-87.
81. Dam H.H., Abbass H.A., Lokan C., Yao X. Neural-based learning classifier systems. IEEE Trans. Knowl. Data Eng. 2008, 20(January (1)):26-39. [Online]. Available: http://dx.doi.org/10.1109/TKDE.2007.190671
82. M.M. Islam, X. Yao, S.M.S. Nirjon, M.A. Islam, K. Murase, Bagging and Boosting Negatively Correlated Neural Networks, 2008.